This article originally appeared in the February 1996 issue of the C++ Report. I have made it available here due to the widespread interest in COM smart pointers. It is a follow-up piece to Interface Pointers Considered Harmful which appeared in the September 1995 issue.

COM Smart Pointers Even More Harmful

Don Box

The September 95 issue of C++ Report contained an article that explored various issues surrounding COM interface pointers. In that article I used an excerpt from a class library to demonstrate how using smart pointers could hide many of the intricacies of IUnknown behind a convenient type- and exception-safe firewall. Since the publication of that article, I and others have had an opportunity to exercise various versions of the SmartInterface class, and this article presents some of the more interesting issues that have come to the surface since the original article was published.

To Leak Or Not To Leak

One of the primary goals of using a smart pointer to wrap IUnknown was to prevent resource leaks due to exceptions or unbalanced AddRef/Release calls. This goal conflicts somewhat with the goals of efficiency and source-level transparency. Consider the following simple example that uses pure COM interface pointers:
void SayHello(IFoo *pfoo)
{
    IBar *pbar = 0;
    if (SUCCEEDED(pfoo->GetBar(&pbar)))
    {
        pbar->Hello();
        pbar->Release();
    }
 }
Note that the interface is released only when the call to IFoo::GetBar succeeds. This is the correct behavior. Also note that if the call to Release is missing or if an exception is thrown prior to reaching it, an object reference will be leaked.

To be usable, smart pointers need to exhibit a high degree of source-level transparency. To achieve this, one could implement a minimal smart pointer as follows:

template <class Interface>
class SmartInterface {
  Interface *m_pif;
public:
  SmartInterface(Interface *pif = 0) : m_pif(pif) {
    if (m_pif) m_pif->AddRef();
  }

  ~SmartInterface() { if (m_pif) m_pif->Release(); }

  Interface ** operator & () { return &m_pif; }
  Interface * operator -> () { return m_pif; } 

// remaining members deleted for clarity
};
Given the smart pointer above, the original example could easily be transformed to look like this:
void SayHello(IFoo *pfoo)
{
    SmartInterface<IBar> pbar;
// (1) operator & yields an IBar**
    if (SUCCEEDED(pfoo->GetBar(&pbar)))
    {
// (2) operator -> yields something with Hello
        pbar->Hello();
// (3) operator -> yields something with Release
        pbar->Release();
    }
 }
The three annotations above help to illustrate the tension that exists between correctness and transparency. For the smart interface class to be correct, it must ensure that it properly maintains the reference count on the interface. For this to happen, the call to IFoo::GetBar which overwrites the smart pointer's native interface pointer must be matched by exactly one call to Release on the same interface. To guarantee that this happens in the face of exceptions or programmer error, the destructor of the smart pointer will call Release. This behavior is necessary to maintain the integrity of the reference count.

Unfortunately, the code example above results in Release being called twice, once implicitly in the destructor, and once explicitly via the arrow operator (annotation 3). Depending on the object that is on the other side of the interface, this extraneous release will either be silently ignored, or cause the client application to die a horrible death. Ideally, one would like the call to Release to not only release the interface pointer, but also inform the smart pointer that it is no longer responsible for releasing the interface (this would imply that the arrow operator could return something that would forward all of the members of IBar to the actual interface, but would intercept the calls to AddRef and Release and map them onto something reasonable).

The problem of trapping individual members in smart pointers is a long-standing one. One solution would require the creation of an adaptor object that could be returned from SmartInterface's arrow operator. This object would explicitly forward all members through the original interface pointer and would implement AddRef and Release by manipulating the smart pointer. This approach is lacking in that the implementation of the adaptor object must be done by hand for each interface. An alternative solution would be to return a pointer that prohibits the use of AddRef and Release. This can be done by deriving a new interface from the base interface that adds no new methods (which results in a binary-compatible vtable) but does change the protection level of AddRef and Release from public to private, thus preventing AddRef and Release calls on the smart interface. The following is an example of this technique:

template <class Interface>
class SmartInterface {
  Interface *m_pif;
public:
  class INoAddRefOrRelease : public Interface {
  private:
    virtual ULONG AddRef() = 0;
    virtual ULONG Release() = 0;
  };
  INoAddRefOrRelease * operator -> () { 
    return (INoAddRefOrRelease * )m_pif; 
  } 
// remaining members deleted for clarity
}
With the smart interface above, this line of code would work exactly as before:
pbar->Hello(); // public member of INoAddRefOrRelease
while the call to Release is now a compile-time error.

The advantage of simply hiding the offending methods is that there is no additional runtime cost for methods that are publicly visible in the derived interface. The disadvantage is that the use of the smart pointer is no longer completely transparent in the client code. Raw pointer compatibility could be increased somewhat by adding AddRef and Release as member functions of the smart pointer class. This of course would require the client to use . instead of -> to invoke these two methods. More problematically, the presence of an AddRef method on the smart pointer implies that either (1) we are trusting clients to always balance calls to AddRef with an equal number of Release calls, or (2) the smart pointer will keep track of the number of additional references and will call Release an equal number of times in the destructor to offset unbalanced AddRefs.

Adding a reference count data member to the smart pointer is certainly possible, but not worth doubling the footprint of what should be a very simple class. To understand why, we need to examine where calls to AddRef and Release take place. According to the COM specification, Release must be called prior to overwriting an interface pointer variable or when the scope of a pointer variable ends. A proper implementation of the smart pointer's assignment operator and destructor will ensure this behavior. Release can also be used to explicitly indicate that an object is no longer in use. Assuming the following assignment operator:

template <class Interface> SmartInterface<Interface>& 
SmartInterface<Interface>::operator = (Interface *pif)
{
  if (m_pif) m_pif->Release();
  m_pif = pif;
  if (m_pif) m_pif->AddRef();
  return *this;
}
one could easily force a release explicitly by assigning the null pointer to the smart pointer. If this usage is not obvious enough, an explicit method could be added as well.

One other common usage of Release is when an interface pointer is to be stabilized. Stabilization is used to artificially bump the reference count on an interface for some short duration of time, often when there is the possibility of a nested call to Release. The following is an example of stabilization:

void DoSomething(IBar *pbar)
{
  pbar->AddRef();  // artificially lock interface
  pbar->RiskyOperation();
  pbar->Release(); // offset artificial lock
}
While stabilization is not required for most operations, it occurs often enough to address in a smart pointer. The following helper class supports pointer stabilization:
template <class Interface>
class SmartInterface {
  Interface *m_pif;
public:
  friend class Stabilize;
// ...
};
class Stabilize {
  IUnknown *m_punk;
public:
  template <class Interface>
  Stabilize(SmartInterface<Interface>& si) 
    : m_punk(si.m_pif) 
  { if (m_punk) m_punk->AddRef();  }

  Stabilize(IUnknown *punk) : m_punk(punk) 
  { if (m_punk) m_punk->AddRef(); }

  ~Stabilize() { if (m_punk) m_punk->Release(); }
};
This class implies the following usage:
void DoSomething(IBar *pbar) {
  Stabilize lock(pbar);
  pbar->RiskyOperation();
}
or
void DoSomething(SmartInterface<IBar> &pbar) {
  Stabilize lock(pbar);
  pbar->RiskyOperation();
}
either one of which ensures that each call to AddRef will be balanced by exactly one call to Release.

To Cast Or Not To Cast

Dealing with Release in a smart pointer is relatively straight-forward. In contrast, the rules surrounding AddRef make it difficult to implement a smart pointer correctly. Most of the problems surface when addressing the type-cast operator.

Assuming that the smart pointer class provides a typecast operator to expose the native interface pointer, the question of whether or not to AddRef for each cast must be addressed. As one might fear, there are occasions where an AddRef is necessary and occasions where it is prohibited. When the cast operator is invoked to pass the interface as an input parameter to a method or function, then no AddRef is required:

void f(SmartInterface<IBar> & pbar)
{
  ::CoSomeFunction(pbar); // no addref is needed
}
However, when the interface is used on the right hand side of an assignment or as the result of a function, it must be AddRefed:
void GetBar(IBar **ppbarOut)
{
  SmartInterface<IBar> pbar = SomeHelperProc();
  *ppbarOut = pbar; // here, an AddRef is needed
}
Since both forms are common, the presence of a cast operator leaves the client code susceptible to either leaked references or over-released objects, both of which are unacceptable. Given this, the only acceptable alternative is to provide an explicit accessor function that indicates the usage:
template <class Interface> Interface *
SmartInterface<Interface>::GetInterface (BOOL bAddRef) {
  if (bAddRef && m_pif)
    m_pif->AddRef();
  return m_pif;
}
One could also write convenience wrappers that don't require the client to remember what value to use for bAddRef:
template <class Interface> Interface *
SmartInterface<Interface>::AsInParam() {
 return GetInterface(FALSE);
}
template <class Interface> Interface *
SmartInterface<Interface>::AsRightHandSide() {
 return GetInterface(TRUE);
}
These members make it very clear to the maintainer of the code what the original intention was.

The problems that surface with the cast operator exist because the usage of the operator's result is ambiguous. This ambiguity also occurs when dealing with unary operator &. Consider the following code fragment that uses native interface pointers:

void f(IUnknown * &punk1, IUnknown * &punk2)
{
// (1) pass the address of punk1 as an [out] parameter
  g(&punk1);
// (2) pass the address of punk2 as an [in, out] parameter
  h(&punk2);
}
In the first statement, the interface pointer is being passed by reference and is an [out] parameter, indicating that its value is considered indeterminate on input and will be valid or null on output. COM requires that the function simply overwrite the parameter and call AddRef on the new interface pointer as follows:
void g(IUnknown **ppunk)
{
// on input, *ppunk is meaningless, so ignore it
  *ppunk = new CSomeClass;
  (*ppunk)->AddRef();
}
In the second statement, the interface pointer is being passed by reference and is an [in, out] parameter, indicating that its value is considered valid on input and will be valid or null on output. COM requires that the function Release the initial interface pointer prior to overwriting it. The call to AddRef is required as before:
void h(IUnknown **ppunk)
{
// on input, *ppunk is meaningful, so release prior to 
// overwriting it
  if (*ppunk) (*ppunk)->Release();
  *ppunk = new CSomeClass;
  (*ppunk)->AddRef();
}
In both cases it is expected that the caller will make the offsetting Release call.

As shown above, the [in, out] form of unary & assumes that the contents of the variable will be manipulated according to normal assignment rules. In this case, no special treatment is required and the following would be the correct implementation:

template <class Interface> Interface **
SmartInterface<Interface>::operator & () {
  return &m_pif;
}
In the simple [out] parameter case, the unary & operator needs to ensure that the data member is ready to be overwritten, which means that a call to Release is needed:
template <class Interface> Interface **
SmartInterface<Interface>::operator & () {
  if (m_pif) m_pif->Release();
  return &m_pif;
}
Of course, only one form is allowed in the actual implementation. In the case of the type-cast operator, it is unclear which of the two possible contexts were more common, so to avoid possible errors, no cast operator is provided, forcing the client to be explicit about the usage. This approach could also be used for unary &:
template <class Interface> Interface **
SmartInterface<Interface>::AsOutParam () {
  if (m_pif) m_pif->Release();
  return &m_pif;
}
template <class Interface> Interface **
SmartInterface<Interface>::AsInOutParam () {
return &m_pif;
}
However, in scanning the standard interface suite, the usage of [in, out] parameters is very rare, and one could make a argument for providing the operator in the [out] form.

Testing, Testing

One negative side-effect of eliminating the cast operator is the loss of this familiar usage:
void f(SmartInterface<IBar> & pbar) {
  if (pbar)   // relies on cast to integral or ptr type
    pbar->Hello();
}
Requiring an explicit member function to test the status is possible, but further compromises the transparency of the smart pointer.

The standard practice of providing a cast to an int, void *, or boolean is possible, but allows heterogeneous comparisons to silently occur:

template<class Interface> 
SmartInterface<Interface>::operator bool () {
  return m_pif != 0;
}
void f(SmartInterface<IFoo> & pfoo, 
       SmartInterface<IBar> & pbar) {
  if (pbar == pfoo) // this compiles but makes no sense 
    pbar->Hello();
}
One indirect way of solving the problem is to use operator !. Providing operator ! does not weaken the type safety of the smart pointer, and allows tests for valid and invalid pointers to take place by using ! twice:
template<class Interface> bool
SmartInterface<Interface>::operator ! () {
  return m_pif == 0;
}
void f(SmartInterface<IFoo> & pfoo) {
  if (!!pfoo) // correct, but not obvious
    cerr << "the pointer is good";
}
While functional, this approach is less than perfect, as many users of the original smart pointer class had pointed out. The ideal solution would allow transparent usage in both success and failure tests and would not allow heterogeneous comparisons without explicit support. This can be achieved by providing a cast operator that yields a pointer to a nested class that is not type-compatible with any other class:
template<class Interface>
class SmartInterface {
  Interface *m_pif;
public:
  class PlaceHolder {}; 
  operator PlaceHolder * () {
    return (PlaceHolder *)m_pif;
  }
  bool operator ! () {
    return m_pif == 0;
  }
};
The use of the placeholder class allows normal validity tests:
void f(SmartInterface<IBar> & pbar) {
  if (pbar) // operator SmartInterface<IBar>::PlaceHolder *
    pbar->Hello();
}
but prohibits heterogeneous comparisons:
bool IsSame(SmartInterface<IBar> & pbar,       
            SmartInterface<IFoo> & pfoo,
  if (pbar == pbar) // error: PlaceHolder<T>'s incompatible
    pbar->Hello();
}
The implementation shown has the side effect of implicitly supporting homogeneous comparisons correctly.

Missing The Obvious

Writing technical articles forces one to pay attention to both correctness and completeness. While I have been guilty of lapses on both fronts, it is the latter goal that I find most elusive. In the
September article, I tried to illustrate why using the delete operator on an IUnknown interface pointer was a bad idea. While it was and still is a bad idea to try to delete an interface pointer, my thinking (and subsequent writing) was incomplete in coming up with ways to prevent it. I had fleshed out several possibilities revolving around private and pure virtual destructors, none of which would work. In trying to completely exhaust the possibilities for destructor-based solutions, I became so focused that I missed the obvious solution. Had Microsoft declared a private class-specific version of operator delete in IUnknown, the problem would be solved. Assume the following modified definition IUnknown:
class IUnknown {
  void operator delete(void *); // private, never called
public:
  virtual HRESULT QueryInterface(REFIID, void**) = 0;
  virtual ULONG AddRef() = 0;
  virtual ULONG Release() = 0;
};
With the definition above, one still gets the same vtable layout as the prior definition of IUnknown, and it is now a compile-time error to delete an interface pointer. Hopefully subsequent versions of the IDL compiler and the system headers will use this safer version.

TRAILER

This article illustrates the complexities surrounding COM and smart pointers. While necessary, the rules of COM reference counting result in certain compromises when trying to balance transparency, efficiency and correctness. The current state of the smart pointer discussed is available via http://www.develop.com/combind.htm.
Back to Don's Home Page