We usually test that our code does what we expect. Pete Barber tries to prove that his code fails to compile.
There is an issue with the use of Window’s COM types being used with STL containers in versions of Visual Studio prior to 2010. The crux of the matter is that various COM types define the
address-of
operator and instead of returning the real address of the object they return a logical address. If the STL container re-arranges its contents this can lead to corruption as it can unwittingly manipulate an object at the incorrect address.
The prescribed solution to this problem is to not place the COM types directly into the STL container but instead wrap them with the ATL
CAdapt
1
class. The merit or not of this approach is not the subject of this article. Instead, assuming this path is being followed, there are two important aspects to address:
- Firstly, is there a way to prevent incorrect usage of the containers?
- Secondly, assuming this is possible, is there a way to make sure it covers the various permutations of STL containers and COM types that are being used?
It turns out that the first issue is easily solved using template specialization. As shall be shown, this works very well to the extent that it prevents compilation. This is fine except that to write a set of unit tests for this requires them to handle the success case which is compilation failure!
The particular issue required finding uses of the following classes of which all but the first are class templates:
-
CComBSTR -
CComPtr<> -
CComQIPtr<> -
_com_ptr_t<>
When used with the following STL containers:
-
std::vector -
std::deque -
std::list -
std::map -
std::multimap -
std::set -
std::multiset
It wasn’t necessary to explicitly handle
std::stack
,
std::queue
and
std::priority_queue
as these are implemented in terms of the previous classes.
In order to simplify the explanation the scope of the problem will be reduced to the use of the following custom type:
class Foo {};
used in
std::vector
, i.e.
2
std::vector<Foo> vecOfFoo;
Part 1 – Detecting the problem
Preventing usage of
std::vector<Foo>
is fairly simple. All that’s required is to provide a specialization of
std::vector
for
Foo
that contains a private constructor so it can’t be constructed (for example, see Listing 1).
template<typename A> class std::vector<Foo, A>
{
private:
vector();
};
|
| Listing 1 |
This now causes a compilation error, e.g.
error C2248: 'std::vector<_Ty>::vector' : cannot access private member declared in class 'std::vector<_Ty>'
The specialization needs to be present before any usage. This is achieved by creating a new file containing the specialization called
prevent.h
that also includes
vector
. It also contains the declaration class
Foo;
to avoid circular dependencies.
Foo.h
is then modified to include
prevent.h
.
The specialization is in fact a partial specialization as the type parameter for the allocator remains. If it were fully specialized then the specialization would be
std::vector<Foo, std::allocator<Foo> >
This would not be sufficient to catch an instantiation that uses a custom allocator or, bizarrely,
std::allocator
instantiated with a type different to
Foo
.
In the real world situation prevent.h would be extended to include all of the specified containers and contain specializations for all the combinations of container and payload. This has the downside that every inclusion of prevent.h pulls in the headers for all the STL containers and requires parsing all the specializations.
This isn’t a complete solution. The
Foo
class is equivalent to
CComBSTR
above as they’re both concrete classes. Preventing the use of a class template in a container which is also a template is a little more complex. The scope of the example problem can be extended to a class template, e.g.
template<typename T> class Bar {};
To prevent this being used as a type parameter to
std::vector
the partial specialization in Listing 2 can be used. This will detect the use of any type used with
Bar
(for example, as shown in Listing 3).
template<typename T, typename A>
class std::vector<Bar<T>, A>
{
private:
vector();
};
|
| Listing 2 |
class Dummy {};
void OtherUsesOfBar()
{
std::vector<Bar<int*> > vecOfBarIntPtr;
std::vector<Bar<Dummy> > vecOfBarDummy;
std::vector<Bar<const Dummy*> >
vecOfBarDummyConstPtr;
}
|
| Listing 3 |
Part 2 – Testing the detection mechanism
Some basic manual testing with the previous samples (well, the real types in question) gives a general feeling that the mechanism is working, i.e. any use causes a compilation failure. However, when all the different combinations are considered there are too many to test manually. This is very true if a test program is written that contains examples of each, as compilation will fail at the first usage: a manual cycle of compile, fail, comment-out failed code and move onto the next failure is required.
This calls for some form of automated testing. At this level the most familiar and applicable is Unit Testing. Unit Testing is generally considered to be automatic testing, which causes a problem as automating the manual tests is not just a case of creating a set of fixtures. This is because a successful test results in a compilation failure!
Using various template based mechanisms and techniques, it is possible to check that the detection mechanism does indeed capture all of the combinations without explicitly compiling – well, failing to compile – each combination.
As it seems impossible at the normal compilation level to detect code that will not compile, the next best thing is to attempt to detect the presence of code that will prevent compilation. In this case it means testing for the presence of the specializations.
SFINAE
The first mechanism required is Substitution Failure Is Not An Error (SFINAE) [ Wikipedia ]; a concept introduced by Davide Vandevoorde [ Vandevoorde02 ]. To facilitate this, a line of code must be added to the partial specialization. Any other specialization (full or partial) would also require this line adding (see Listing 4).
template<typename A>
class std::vector<Foo, A>
{
private:
vector();
public:
typedef std::vector<Foo, A> self_t;
};
|
| Listing 4 |
A class is created with an overloaded method which returns
true
if a specialized version of
std::vector
is detected and
false
otherwise (see Listing 5).
class DetectPreventionWithSFINAE
{
public:
template<typename U>
static bool Detect(const typename U::self_t)
{
return true;
}
template<typename U>
static bool Detect(...)
{
return false;
}
};
|
| Listing 5 |
Its use is as follows:
typedef std::vector<Bar<int>> BarInt_t;
const bool detectVectorOfBarInt =
DetectPreventionWithSFINAE::Detect<BarInt_t>(
BarInt_t());
If the type passed to
DetectPreventionWithSFINAE
does not contain the
typedef self_t
then the first version of
Detect
is invalid code. Rather than this being an error, the compiler just removes this overload from the available overload set and the method is not instantiated. This leaves the other overload, which as it has ellipses for its argument will match anything, but only if nothing more specific is available. In this case it is the only method, meaning it will be used and when invoked it will return
false
. The removal of the invalid code rather than a compilation error is SFINAE.
In the case where the type passed does contain the
typedef self_t
, i.e. it is one of the specializations, then both overloads will be valid but as the first one is the better match it will be chosen. When this method is invoked it will return
true
.
NOTE: The use of
self_t
is not fool proof. If its definition is neglected from a specialization then an instance of that specialization will not match that case. Conversely if an altogether different specialization of
std::vector
is created which contains the same named
typedef
to itself this will also match.
sizeof
Using SFINAE the presence of the specialization can be detected, but unfortunately this requires creating an instance of it, which of course can’t be done. In fact the example above is misleading as the code will not compile. Additionally, assuming the code could compile, passing an object to a method with ellipses as its argument has undefined results so this code couldn’t be reliably used.
This seems to be almost back at square one: the presence of the prevention code can be detected but it requires an instance of the specialization that can’t be compiled in order to do so! However, examining the use of the instance it becomes clear that the instance isn’t actually used. It’s only needed in order to interrogate its type. This is the realm of Template Meta Programming (TMP). A fair amount of TMP can be performed without creating instances of types but in this case – as SFINAE is being used, which is tied to the overloading of function templates – an instance that can be passed as a parameter to a function template is required.
Fortunately the innovative use of
sizeof
as introduced by Andrei Alexandrescu [
Alexandrescu01
] in his seminal work
Modern C++ Design
can be used.
3
,
4
This is shown in an updated version of the
DetectPrevention
class (see Listing 6).
template<typename T> class DetectPrevention
{
typedef char Small;
class Big { char dummy[2]; };
public:
static bool Detect()
{
return sizeof(DetectImpl<T>(MakeT())) ==
sizeof(Small);
}
private:
template<typename U> static
Small DetectImpl(const typename U::self_t);
template<typename U> static
Big DetectImpl (...);
static T MakeT();
};
|
| Listing 6 |
The key point is that if a type is used within the
sizeof
operator then the compiler does not have to actually create an instance of it in order to determine its size. As shown by Alexandrescu, the real magic is that
sizeof
is not just limited to taking a type. It can be given an entire expression which, as long as it results in a type (so a type name, a function invocation, i.e. calling not just referencing a method that returns a value, so non-void or a logical expression resulting in a Boolean), works.
Another essential bit of magic is the
MakeT
method. Even though an instance is required for the overloaded
DetectImpl
methods, as these are only ever referenced within the
sizeof
operator the instance doesn’t actually need creating. If
T()
rather than the call to
MakeT
were specified in the definition of
Detect
, e.g.
static bool Detect()
{
return sizeof(DetectImpl<T>(T())) ==
sizeof(Small);
}
then this won’t compile as the compiler checks that a constructor is available, and this is not the case as the specialization deliberately makes it private. However, as
sizeof
only needs to calculate the size as opposed to constructing an instance of
T
, the
MakeT
method provides the
potential
to create one, which is sufficient. This is demonstrated by the fact there is no definition.
The ‘no definition’ theme is extended to the
DetectImpl
methods, as these too are never executed but purely evaluated at compile time. This is important given the earlier statement that passing an object, in this case
T
, to a method taking ellipses could lead to undefined results.
In the previous non-working example, the
Detect
methods return a Boolean to indicate whether the specialization was detected or not. As the
DetectImpl
methods have no implementation and are not invoked, it is not possible for them to return a value. Instead, another technique from Alexandrescu is used. This again returns to the use of
sizeof
, which in this case evaluates the size of the return type of whichever overload of
DetectImpl
was chosen. This is then compared to the size of the type returned by the chosen method. If these are equal then it means the overload of
DetectImpl
that requires the specialization was chosen; if not, the other method was chosen. This use of the size comparison then converts the application of SFINAE into a Boolean result, which can be used at compile time.
Any differently sized types could have been used for return values; however, as some can vary on a platform basis explicitly defining them avoids any future problems, hence the use of
Big
and
Small
.
5
At this point the presence or lack of the specialization can be detected as shown in Listing 7.
#include <iostream>
#include <vector>
#include "prevent.h"
#include "bar.h"
#include "DetectPrevention.h"
int main()
{
bool isPreventer =
DetectPrevention<std::vector<Bar<int>>>
::Detect();
std::cout << "isPreventer:" << isPreventer
<< std::endl;
}
|
| Listing 7 |
Part 3 – What about the other STL Containers and types?
The problem faced was the prevention and detection of a limited number of types against a limited number of STL Containers. For the purposes of the example the types to be placed in the containers are:
-
Foo -
Bar<>
The partial specialization will catch every use of
Bar<T>
in production code. As the set of
T
is effectively infinite, testing the detection of
Bar<T>
used within the containers is limited to a representative set of types. In the real world scenario, as the problematic template types were all wrappers for COM objects, just using them with
IUnknown
sufficed. For this example
int
will continue to be used. If a warmer feeling is required that different instantiations are being caught then the examples can be extended to use additional types.
At the moment, the use of
Foo
and
Bar<>
can be prevented and detected in
std::vector
. This can be easily extended to the remaining applicable STL Containers:
deque
,
list
,
map
,
multimap
,
set
and
multiset
, by creating specializations for
Foo
and
Bar<>
in a similar manner to that of
std::vector
.
The source is too long to show here for all the containers but is included in the example code. In Listing 8 are the partial specializations for
std::vector
and
std::map
to handle
Foo
and
Bar<>
.
// Prevent use of std::vector with Foo
template<typename A>
class std::vector<Foo, A>
{
private:
vector();
public:
typedef std::vector<Foo, A> self_t;
};
// Prevent use of std::vector with Bar<T>
template<typename T, typename A>
class std::vector<Bar<T>, A>
{
private:
vector();
public:
typedef std::vector<Bar<T>, A> self_t;
};
// Prevent use of std::map with Foo
template<typename T1, typename T2, typename T3>
class std::map<Foo, T1, T2, T3>
{
private:
map();
public:
typedef std::map<Foo, T1, T2, T3> self_t;
};
template<typename T, typename T2, typename T3>
class std::map<T, Foo, T2, T3>
{
private:
map();
public:
typedef std::map<T, Foo, T2, T3> self_t;
};
template<typename T2, typename T3>
class std::map<Foo, Foo, T2, T3>
{
private:
map();
public:
typedef std::map<Foo, Foo, T2, T3> self_t;
};
// Prevent use of std::map with Bar<T>
template<typename T, typename T1,
typename T2, typename T3>
class std::map<Bar<T>, T1, T2, T3>
{
private:
map();
public:
typedef std::map<Bar<T>, T1, T2, T3> self_t;
};
template<typename T, typename T1,
typename T2, typename T3>
class std::map<T, Bar<T1>, T2, T3>
{
private:
map();
public:
typedef std::map<T, Bar<T1>, T2, T3> self_t;
};
template<typename T, typename T1,
typename T2, typename T3>
class std::map<Bar<T>, Bar<T1>, T2, T3>
{
private:
map();
public:
typedef std::map<Bar<T>, Bar<T1>, T2, T3> self_t;
};
|
| Listing 8 |
To avoid repeating lots of test code another template mechanism is used. This is template-templates. The code sample in Listing 9 shows a reduced set of tests for detecting the prevention of
Foo
and
Bar<int>
when used to instantiate a
vector
,
deque
,
list
,
map
and
multimap
.
#include <iostream>
#include <vector>
#include <deque>
#include <list>
#include <set>
#include <map>
#include "prevent.h"
#include "foo.h"
#include "bar.h"
#include "DetectPrevention.h"
using namespace std;
template<template<typename, typename> class CONT>
bool TestSimpleContainer()
{
const bool detectFoo =
DetectPrevention<CONT<Foo,
allocator<Foo> > >::Detect();
const bool detectBarT =
DetectPrevention<CONT<Bar<int>,
allocator<Bar<int> > > >::Detect();
return detectFoo && detectBarT;
}
template<template<typename, typename,
typename, typename> class MAP>
bool TestMap()
{
typedef int Placeholder_t;
typedef MAP<Foo, Placeholder_t, less<Foo>,
allocator<pair<Foo,
Placeholder_t> > > FooAsKey_t;
typedef MAP<Placeholder_t, Foo,
less<Placeholder_t>,
allocator<pair<Placeholder_t,
Foo> > > FooAsValue_t;
typedef MAP<Foo, Foo, less<Foo>,
allocator<pair<Foo,
Foo> > > FooAsKeyAndValue_t;
const bool detectFooAsKey =
DetectPrevention<FooAsKey_t>::Detect();
const bool detectFooAsValue =
DetectPrevention<FooAsValue_t>::Detect();
const bool detectFooAsKeyAndValue =
DetectPrevention<FooAsKeyAndValue_t>
::Detect();
// Bar<int>
typedef MAP<Bar<int>, Placeholder_t,
less<Bar<int> >, allocator<pair<Bar<int>,
Placeholder_t> > > BarAsKey_t;
typedef MAP<Placeholder_t, Bar<int>,
less<Placeholder_t>,
allocator<pair<Placeholder_t,
Bar<int> > > > BarAsValue_t;
typedef MAP<Bar<int>, Bar<int>, less<Bar<int>>,
allocator<pair<Bar<int>,
Bar<int> > > > BarAsKeyAndValue_t;
const bool detectBarAsKey =
DetectPrevention<BarAsKey_t>::Detect();
const bool detectBarAsValue =
DetectPrevention<BarAsValue_t>::Detect();
const bool detectBarAsKeyAndValue =
DetectPrevention<BarAsKeyAndValue_t>
::Detect();
return detectFooAsKey &&
detectFooAsValue &&
detectFooAsKeyAndValue &&
detectBarAsKey &&
detectBarAsValue &&
detectBarAsKeyAndValue;
}
int main()
{
cout << "vector:"
<< TestSimpleContainer<vector>() << endl;
cout << "deque:"
<< TestSimpleContainer<deque>() << endl;
cout << "list:"
<< TestSimpleContainer<list>() << endl;
cout << "map:"
<< TestMap<map>() << endl;
cout << "multimap:"
<< TestMap<multimap>() << endl;
}
|
| Listing 9 |
The tests for the
vector
,
deque
and
list
are identical so rather than repeating the code for each container type the container type is a template parameter. This requires the use of template-template parameters as rather than passing an instantiated template, i.e. a concrete type, a container to test is passed instead. This is then instantiated with different types within the test method, e.g.
Foo
and
Bar<int>
.
Conclusion
Attempting to create a Unit Test (well some form of automated test) to determine if code wouldn’t compile without actually stopping it from compiling was a daunting task. The application of various template techniques showed that in combination this problem could be addressed. Some of these techniques, despite being fairly old, are still relatively unknown and their use somewhat obscure. This article shows just the simple application of them can be used to solve practical problems.
References
[Alexandrescu01] Alexandrescu, Andrei. Modern C++ Design: Generic Programming and Design Patterns Applied. 2001.
[Vandevoorde02] Vandevoorde, David & Nicolai M. Josuttis. C++ Templates: The Complete Guide . 2002.
[Wikipedia] ‘Substitution failure is not an error.’ Wikipedia. http://en.wikipedia.org/wiki/Substitution_failure_is_not_an_error
- http://msdn.microsoft.com/en-us/library/bs6acf5x(v=VS.90).aspx
-
This class does not exhibit the
address-ofproblem of the COM classes. This doesn’t matter as the techniques to be shown do not detect the presence of theaddress-ofoperator but instead detect named classes. -
The explanation of how
sizeofworks in this context is derived from this book, quite possibly almost verbatim. The application is different. -
The combination of SFINAE and
sizeofas used here was first discovered by Paul Mensonides. -
Typedefs of
yesandnoare also common.
