More on overloading new/delete

In my previous post I highlighted some of the pitfalls encountered while adapting memory management functions in a middleware library.

In this post I’ll continue the discussion of overloading new/delete in the context of writing a middleware library. The specifics here are that when you write middleware you have to be very careful not to inadvertently stomp the client application’s domain – memory, rendering state etc.

Overloading new/delete with some sugar on-top

In a new library we are writing at Coherent Labs, we went with the standard memory tracking technique of overriding all new/delete operators with a memory tag.

It looks something like this:

inline void* operator new(size_t size, MemoryTag memtag) {
  return gAllocator->Allocate((unsigned)size, memtag.Tag);
}

Here gAllocator is a pointer to an interface the client provides to hook her own allocation methods. Adding a memory tag really helps debugging and shows how memory is distributed in subsystems. On-top of these overloaded allocators we have macros that simplify the syntax.

I prefer overloading new/delete as it more or less allows us to continue using the syntax most developers are used to, instead of going for completely custom memory allocation templates.

In C++ you can overload delete but can’t call it. The overloaded instances are only called by the compiler when throwing an exception in a constructor. We solve this by substituting delete with a custom function like this:

template <typename T>
inline void DestroyMemory(T* ptr, MemoryTag tag) {
  if (!ptr)
    return;
  ptr->~T();
  operator delete(ptr, tag);
}

The function will call our overload with a tag. In C++ there is a difference between “delete operator” and “operator delete”. We prefer tagging deallocations too because it helps debugging and you can have accurate per-subsystem stats even if you don’t track every single allocation.
If you’ve read my previous post you know that overriding new[]/delete[] has some issues so we simply disable them and go for custom functions for array allocation and deallocation.

Guarding against default new

Now that we have custom macros and functions for all allocations, it is important that everybody always uses them instead of the default new/delete. Developers have to be careful and reviewers should be extra cautious, but this is not a very good way to make sure the policy is enforced.

We make extensive use of STL containers with custom allocators. People have again to use the specialized templates instead of the generic ones.

Things become even more tricky with C++ smart pointers. It is good policy (and important for performance) to use std::make_shared when creating a shared_ptr. Unfortunately std::make_shared calls the default new. When you need to allocate the pointer through you own custom allocator you have to go with std::allocate_shared. It is easy to make a mistake and call the wrong function due to habit.

We need a way to make sure the default new/delete are never called. A relatively straightforward way is overriding them and asserting their calls. This will fail in debug (although on runtime) and is a good solution if you library is dynamically linked. If the library is statically linked however, the overridden functions will ‘leak’ to the client application! Our library has to be linked statically on some platforms, so this method becomes problematic. Even if we only do the check in Debug, we’d have to disable default allocations in our test applications which is quite inconvenient.

In the end I came up with post-processing step that checks if new/delete are ever called in the library. We do this on POSIX platforms as those were the ones where we had to statically link on. On Windows things should be analogous.

When the library is compiled in the final “.a” file, it contains a symbol list with everything it uses so that the linker can then properly link external symbols. The memory operators are such functions. We use the “nm” tool to inspect what symbols are referenced in the library. The names are mangled, but can be un-mangled with “c++filt”.

If anywhere in the library default “new” is called, it will be reflected by “nm” along with the object where this was done. We wrote a  tool that runs after each build to inspect the library for unwanted calls.

The method can be extended to ‘guard’ against other forbidden functions too.