reference(instead of ptr) to smart ptr instance

I usually pass smart pointers by value (copy-constructor or move-constructor), just like copying a raw ptr.  Therefore the code below looks unnatural:

unique_ptr<Trade> & ref2smartPtr

Actually it is rather common because

  • As Herb Sutter suggested, when we need to put pointer into containers, we should avoid raw ptr. Unique ptr is the default choice, and the first choice, followed by shared_ptr
  • I often use unique_ptr as map value . The operator[] return type is a reference to the value type i.e. reference to unque_ptr
  • I may also put unique_ptr into a vector…. ditto for vector operator[]

buggy RAII smart_ptr #SIG

Q1: given a well-behaved class X, someone wrote a RAII wrapper below. What problems do you see?

Q1b: how would you fix them?

%%A1: memory leak of x1, due to the synthesized op=()
%%A1: double-delete on x2

Q1c: OK Good. Now what if X has subclass Y,  all with virtual dtor?
%%A1c: X and Y each create virtual clone() to return a pointer to the host type

Q2: what’s the outcome of double-delete?
AA: undefined behavior. You are lucky if it crashes.

struct A{
  A(X* x): myX_(x){}
  ~A(){delete myX_;}
  X* myX_;
void demo(){
  A a1(new X()); //unnamed x1 object on heap
  A a2(new X()); //unnamed x2 object on heap
  a1 = a2;
/////////// above is original code; below is my Q1b answer -- Add op=()
A & operator=(A const & rhs){
  if (this != &rhs){
    auto tmp = this->myX_; //delete only after successful "new"
    this->myX_ = new X(*rhs.myX_); //clone the X object of rhs
    // if new or ctor throws, the memory is not really allocated 🙂
    delete tmp; //let's assume no exception here.
  return *this;

make_shared() cache efficiency, forward()

This low-level topic is apparently important to multiple interviewers. I guess there are similarly low-level topics like lockfree, wait/notify, hashmap, const correctness.. These topics are purely for theoretical QQ interviews. I don’t think app developers ever need to write forward() in their code. touches on a few low-level optimizations. Suppose you follow Herb Sutter’s advice and write a factory accepting Trade ctor arg and returning a shared_ptr<Trade>,

  • your factory’s parameter should be a universal reference. You should then std::forward() it to make_shared(). See gcc source code See make_shared() source in
  • make_shared() makes one allocation for a Trade and an adjacent control block, with cache efficiency — any read access on the Trade pointer will cache the control block too
  • if the arg object is a temp object, then the rvr would be forwarded to the Trade ctor. Scott Meryers says the lvr would be cast to a rvr. The Trade ctor would need to move() it.
  • if the runtime object is carried by an lvr (arg object not a temp object), then the lvr would be forwarded as is to Trade ctor?

Q: What if I omit std::forward()?
AA: Trade ctor would receive always a lvr. See ScottMeyers P162 and my github code is my experiment.


populate map<string,list> no list cloning #make_shared

I feel a few things are fairly impressive for a short coding interview:

  • no leak.
  • no explicit new(). make_shared allocates the control block efficiently

In c++11, we could probably return the list by value, and rely on move semantics.

Here’s an inefficient nested container. Saving pointer-to-list in the map is more common, because inserting or reading the map doesn’t need to clone the list!

//tested with -std=c++0x
unordered_map<string, list<size_t> > lookup;

shared_ptr<list<size_t> > myfunc(){
  shared_ptr<list<size_t> > ret= make_shared<list<size_t> >();
  return ret; //return shared_ptr by clone, cheaply

lookup.insert(make_pair(word, *myfunc())); //dereference to get the list

Thesys QnA IV

Q: what’s the average O() complexity of insertion sort
AA: N^2 in most implementations — comparison-based, but using contiguous memory. NlogN if using skip list

Q: what’s the average O() complexity of heap sort
AA: NlogN, since it’s comparison-based.

Q: what are the synchronization primitives on the operating systems you know

Q: how many levels of caches are in a typical Intel processor
%%A: at least 3 possibly 4.
AA: some intel processors do have 4 levels

Q: name some IPC mechanisms
%%A: 1) shared mem 2) sockets 3) pipes

–2nd round (Don Bush?)

Q: Requirement: insert into a std::map only if key is absent
AA: the insert() function does that by default! Returns a pair<Iterator-where-inserted, bool-isInserted>. I used this same technique in my handleInputInternal()

Q: if my class Student has a private dtor, will it compile?
%%A: yes it’s possible to define a member function to invoke the dtor on a pointer passed in
%%A: deleting a ptr-to-Student won’t compile. Correct!
AA: have a friend call the dtor!

Q: Any guess why STL stack designer didn’t combine top() and void pop() like in other languages? hint: what if the copy ctor throws?
%%A: it’s more efficient if a function doesn’t need to return anything, so we don’t want to force the pop() users to pay a price for something they don’t need.

Q: do you create your own templates?
A: I should name the SFINAE hack!

Q: any restriction on a non-type template parameter’s type?
%%A: int and bool are common. Pointers are allowed. shows even arrays and ptr-to-functions are allowed.
AA: has to be a discrete type, not a float

%%Q: what’s the usage of this non-type parameter?
A: std::array uses it for the size. It can’t be a ctor argument since if you construct an array on stack the size must be known at compile time.

Q: can you use new-arrays with shared_ptr? I think he wanted vector<shared_ptr>
%%A: if I have a new-array to store somewhere, then I think shared_array or scoped_array
A: how about shared_ptr<vector>?

Q: what if ctor failes in Foo * p = new Foo() // will memory leak?
%%A: i know it’s common practice to throw from ctor…

Q: name some c++11 features you used in your recent projects
%%A: unique_ptr, shared_ptr, auto, unordered_map, initializer list for vector/map

make_shared syntax #Gotcha’s

Needed for take-home coding test.

#include <iostream>
#include <memory> // make_shared

struct C{
  float f;
  C(float arg): f(arg){}
int main() {
  std::shared_ptr<int> a(new int(13));
  std::shared_ptr<int> b = std::make_shared<int>(12);

  std::shared_ptr<C> c = std::make_shared<C>(1.1); //must pass in ctor args
  //shared_ptr<C> b= make_shared<C>(new C(1.1)); //can't use "new" with make_shared.


housekeeping^payload fields: vector,string,shared_ptr

See also std::string/vector are on heap; reserve() to avoid re-allocation

std::vector — payload is an array on heap. Housekeeping fields hold things like size, capacity, pointer to the array. These fields are allocated either on stack or heap or global area depending on your variable declaration.

  • Most STL (and boost) containers are similar to vector in terms of memory allocation
  • std::string — payload is a char-array on heap, so it can expand both ways. Housekeeping data includes size…
  • shared_ptr — payload includes a ref counter and a raw-pointer object [1] on heap. This is the control-block shared by all “club members”. There’s still some housekeeping data (pointer to the control block), typically allocated on stack if you declare the shared_ptr object on stack and then use RAII.

If you use “new vector” or “new std::string”, then the housekeeping data will also live on stack, but I find this practice less common.

[1] this is a 32-byte pointer object, not a pure address. See 3 meanings of POINTER + tip on q(delete this)

##shared_ptr thr-safety: 3 cases

This topic is worthwhile as it is about two high-value topics … threading + smart ptr

At least 2 interviewers pointed out thread safety issues … first answer shows many correct MT usages and incorrect MT usages. Looking at that answer, I see at least 3 distinct”objects” that could be “shared-mutable”:

  1. control block — shared by different club members. Any one club member, like global_instance could be a shared mutable object. Concurrent access to the control block is internally managed by the shared_ptr implementation and probably thread-safe.
  2. pointee on heap — is shared  mutable. If 2 threads call mutator methods on this object, you can hit race condition.
  3. global_instance variable —  a shared mutable instance of shared_ptr. race condition 😦




thread-unsafe shared_ptr: tiny examples

##shared_ptr thr-safety: 3 cases is a “framework”. gave simple examples of unsafe access.

//— Example 3, where the shared mutable is a club member p3 ——————
// thread A
p = p3; // reads p3, writes p

// thread B
p3.reset(); // writes p3: Undefined, simultaneous read/write

===> In the same clock cycle, p3 Object is accessed on 2 threads. This is nothing to do with the ref count, or the pointee object.

//— Example 4,  where the shared mutable is a club member p2 ——————
// thread A
p3 = refToP2; // reads p2, writes p3

// thread B
// p2 goes out of scope: Undefined, simultaneous read/write since the destructor is considered a “write access”

===> In the same clock cycle, p2 Object gets write-accessed.

unique_ptr and move()

When I use a unique_ptr instance, I frequently copy it around. Unique_ptr ‘s copying is actual moving.

The are different ways to code it.

  • sometimes you need to use someFunc(move(myUniquePtr));
  • sometimes you can omit move() and the semantics remain the same. has some examples. Note none of the functions have a parameter/return type showing “&&”. That’s because there is pbclone in play. The copying uses the move-constructor, which does have a && parameter.

I think some developers simply copy sample working code, without understanding why, like an ape. Some use threading constructs the same way. Nothing shame. I feel interviewers are interested in your understanding.


unique_ptr shared_ptr auto_ptr #less important scoped_ptr #least important
container more popular than boost::ptr_container! can put into container forbidden illegal
instance field yes [1] but rare yes good replacement
for raw ptr
? Yes!
stack obj YES yes popular OK? Yes
copyable no. move-only yes releases ownership illegal
as return type YES but rare yes

[1] and

shared_ptr [+unique_ptr] pbclone^pbref#Sutter

Practical question. In practice, the safe and lazy choice is pbclone. In IV, I think pbclone is “acceptable”. I feel pbref (const ref) is a risky micro-optimization, but Herb Sutter advised differently:

  • Express that a function will store and share ownership of a heap object using a by-value shared_ptr parameter
  • Use a const shared_ptr& as a parameter only if you’re not sure whether or not you’ll take a copy and share ownership. Perhaps the default choice IMO.

In my apps, the  receiving function often saves the smart_ptr in some container. Both options above actually work fine.

If you just need to access the underlying raw pointer, then Sutter said just pass in the raw pointer. I feel a pbref is also acceptable.

Smart pointer objects are designed to mimic raw pointers, which are usually passed by clone.

see and

—-With unique_ptr, rules are simpler. Pass by clone as much as you want. It will be move-constructed. I don’t think pbref is needed.

See other posts about unique_ptr, such as

%% autoPtr class template

#include <iostream>
using namespace std;

struct Dummy {
	void play() {
		cout << "play " << payload << endl;
	float payload = 0.40490;
ostream& operator<<(ostream& os, Dummy const & d){
	os << "[ Dummy object "<<d.payload<<" ]";
	return os;
template<typename T> class autoPtr {
	autoPtr(): autoPtr(nullptr){}
	autoPtr(T* raw): needle(raw) {
		needle = raw;
		if (raw)
			cout << "ctor called with *raw == " << *raw << endl;
	autoPtr(autoPtr<T> & other) { //steal the resource
		cout << "copier called with other.real ";
		needle = other.needle;
		if (other.needle) {
			cout << "pointing to "<<*needle;
			other.needle = nullptr;
		}else {
			cout << "== nullptr";
		cout << endl;
	autoPtr<T> & operator=(autoPtr<T> & other) {
		if (&other == this) return *this;
		cout << "op= called ...";
		if (needle) {
			cout << " ... deleting original raw ptr *real = " << *needle;
			delete needle;
		cout << endl; this->needle = other.needle;
		other.needle = nullptr;
		return *this;
	T* operator->() {
		cout << "calling operator->"<<endl;
		return needle;
	T& operator* () {
		cout << "dereferencing this ...";
		if (needle)
			cout << ".... *real == " << *needle << endl;
		else {
			cout << ".... returning a null ptr" << endl; throw "dereferencing a null autoPtr!"; } return *needle; //how do we return a T when real is nullptr? } void set(T * const raw) { delete this->needle;
		this->needle = raw;
	~autoPtr() {
		cout << "Dtor called for *real == ";
		if (needle)
			cout << *needle;
			cout << "(null ptr)";
		cout<< endl;
		delete needle;
	T* needle;
int main() {
	autoPtr<Dummy> pdummy(new Dummy);
	autoPtr<int> p1 = (autoPtr<int>)new int(12);
		autoPtr<int> p2(new int(35));
		cout << "Testing deference: *ptr == "<<*p2<< endl;
		autoPtr<int> p3(p2);
		p1 = p3;
	return 0;

retreat to raw ptr from smart ptr ASAP

Raw ptr is in the fabric of C. Raw pointers interact/integrate with countless parts of the language in complex ways. Smart pointers are advertised as drop-in replacements but that advertisement may not cover all of those “interactions”:

  • double ptr
  • new/delete/free
  • ptr/ref layering
  • ptr to function
  • ptr to field
  • 2D array
  • array of ptr
  • ptr arithmetics
  • compare to NULL
  • ptr to const — smart ptr to const should be fine
  • “this” ptr
  • factory returning ptr — can it return a smart ptr?
  • address of ptr object

Personal suggestion (unconventional) — stick to the known best practices of smart ptr (such as storing them in containers). In all other situations, do not treat them as drop-in replacements but retrieve and use the raw ptr.

factory returning smart ptr^pbclone #Sutter2013

Background — This is one example of “too many variations”. A simple idea become popular. People start mix-n-match it with other ideas and create many, many interesting questions. In practice we should stick to a few best practices and avoid the hundreds of other variations.

Update: is a little too dogmatic as an article. A few take-aways:

  • returning raw ptr can indeed leak in real life
  • For a polymorphic type, return a unique_ptr by default. See also [[effModernC++]]. (Return shared_ptr only if the factory saves it in internal state, but who does?)
  • For a non-polymorphic type, return a clone. Safely assume it’s copyable or movable in c++11.

I think it’s quite common (see [[safe c++]] P51) to have a factory function creating a heap “resource”. Given it’s on heap, factory must return the object by pointer. Easy to hit memory leak. Standard solution is return by ref count smart ptr.

Q: what if I forget to destruct the obj returned?
A: I feel it’s uncommon and unnatural. (More common to forget deletion.) Often the object returned i.e. the smart pointer object is on the stack – no way to “forget”.

Note if a raw pointer variable is on the stack, then the stack unwinding will deallocate/reclaim the memory, but won’t call delete on it!

Q: what if the object returned is saved as a field like myUmbrella.resource1
A: I feel this is fine because the myUmbrella object will be destructed by someone’s responsibility. That would automatically destruct the smart pointer field this->resource1.

Alternative to the ref count smart ptr, we can also return a scoped pointer. See [[safe c++]].

Q: what if factory need to return a pointer to stack?
A: I don’t think so. Pointee goes out of scope….
A: I feel most “resources” are represented by a heapy thingy. Alternatively, it’s also conceivable to hold a “permanent” resource like globally. Memory management is non-issue.

By the way, [[safe c++]] also mentions the __common__ pattern of a (non-factory) function returning a pointer to an existing object, rather than a “new SomeClass”. The challenges and solutions are different.

barebones RAII smart ptr classes

If RAII and memory management is the main purpose, I feel a better word for the class template is a “capsule class”. I feel the boost implementations are too complicated and hard to modify. Here are some small but realistic implementations

[[c++FAQ]] has a one-pager RAII capsule class.

[[safe c++]] has a 2-page-short source code for a real-world capsule. Includes both ref count and scoped versions. introduces a very simple RAII capsule. One of the simplest yet useful smart pointers. I think the entire class needs nothing but … big3 + ctor.

A few observations —

* It’s not reliable enough if the capsule ctor can throw. For RAII to provide the ironclad guarantee, the ctor of the “capsule” class (smart ptr or not) should be simple and robust, and no-throw. If the capsule is not even locked during construction, then it offers no protection no guarantee.

* (Smart pointer always needs a copy policy) It’s meaningless unless every client “goes through” the wrapper to get to the raw pointer. Raw pointer direct access is dangerous and should be completely avoided.

* It’s meaningless without the exception risk. RAII is mostly for exceptions and programmer forgetfulness.
* It’s meaningless if there’s no need for an ironclad “guarantee“.
* It’s meaningless if there’s no need for “automatic” cleanup.
* It relies on the dtor feature. RAII relies on dtor. Smart pointer relies on dtor.
* It relies on local stack variables,but …
* It’s unusable unless the original pointer is for a heap object.
… These keywords are the heart of RAII idiom.

Given the heavy reliance on dtor, the dtor should be fairly simple. I don’t think it should be virtual.

STL containers and smart ptr – pbclone on the outside

STL containers should store only values and smart ptr in containers, so said [[c++ coding standards]].

Essentially, store “value types” not reference types, borrowing c# terminology. Containers pass element objects (including 32-bit ptr objects) by value, as a rule. By-reference is less supported and a bit dangerous.

Now let’s look at smart ptr. Except inside the smart ptr class itself, do we ever pass smart ptr Instances by reference/ptr, or we always pass smart ptr Instances by value? I feel by-value is absolutely essential otherwise the special logic in copiers is bypassed.
I feel for STL containers and smart ptr, internal operations may pass element objects by reference or by ptr , but public API is mostly pass-by-value (pbclone) or pass-by-ptr, which is the C tradition. C++ style pass-by-reference is popular mostly in standard constructs such as copier and assignment operator or passing large objects without pointers.

smart ptr = a class instance; raw ptr = holder of an address

(perhaps merge with another post on raw ptr vs smart ptr)

A raw pointer is exactly 32 bit (assuming a 32bit machine). To be precise, we mean a “pointer object”. Given the monolithic nature of this /construct/, its behavior is transparent and well-understood.

A smart ptr acts like a raw pointer but is a class instance, a more sophisticated /construct/. Each standard raw-ptr “operation” is carefully emulated.

One of the operations is “copying a ptr”. A smart ptr is designed for pass by value (pbclone). Designed for cheap copy.

RAII – heap alloc/deleted in separate functions

I think the standard pattern of RAII smart ptr is via a local variable. When it goes out-of-scope RAII kicks in. (P199 [[Essential C++]] is authoritative…)

Q: Now suppose a function is designed to new up an object on the heap, and pass (by several means) it out of the function. The pointee object on heap needs to be cleaned up in another method such as mymain(). So the smart ptr is not local variable. How can we use RAII smart ptr for this?

1) A tested technique – in some outer function such as mymain(), declare/create the smart ptr local variable and pass it (by ref NOT by value — tested in MSVS) into the factory function, which puts the heap ptr into the smart ptr. After using it in some other function, eventually we reach the end of mymain() and the local variable goes out of scope. Exception-safe.

2) A tested technique – a global vector holding a bunch of smart pointers. From anywhere we can save a “resource pointer” into the vector. From anywhere we can also clear the vector, and those pointee objects will receive the delete call. However, there’s no automatic, fool proof clean-up because a careless developer could forget to call clear(). Also, no exception-safety.

boost intrusive smart ptr phrase book #no IV

* MI — P35 [[beyond c++ standard lib]] shows your pointee CLASS can be written to multiple-inherit from a general-purpose ref_count holder class. This is a good use case of multiple inheritance, perhaps in Barclays QA codebase.

* real-estate — The 32-bit ref counter lives physically on the real estate of the pointee. Pointee type can’t be a builtin like “float”. In contrast, a club of shared_ptr instances share a single “club-count” that’s allocated outside any shared_ptr instance.

* legacy — many legacy smart pointer classes were written with a ref count in the pointee CLASS like QA YieldCurve. As a replacement for the legacy smart pointer, intrusive_ptr is more natural than shared_ptr.

* builtin — pointee class should not be a builtin type like float or int. They don’t embed ref count on their real estate; They can’t inherit; …

* TR1 — not in TR1 (, but popular

* ref-count — provided by the raw pointee Instance, not the smart pointer Instance. Note the Raw pointER Instance is always 32 bit (assuming 32-bit bus) and can never host the reference count.

* same-size — as a raw ptr

* expose — The pointee class must expose mutator methods on the ref count field

smartPtr – remember the 3 types involved

I feel it’s easy to confuse the 3 data types involved in even the simplest smart pointer. It’s less confusing when you instantiate the class template and slightly more confusing when you have to program in templates.

Say you use smartPtr, there are really 3 data types involved here
1) T
2) pointer to T
3) smartPtr

There are also at least 3 objects involved
1) a single pointee object, of type T
2) 1 or more 32-bit raw-pointer objects of type T*
3) Multiple instances of smartPtr

At clean-up time, we need to
1) deallocate the pointee object, assuming it’s on heap
2) don’t worry about the raw pointer objects. Someone would take care of it. However, we assume no one calls delete() on one of them!
3) the smartPtr instances are typically low-maintenance. They are often stack-allocated or are (non-ref) fields of bigger objects — non-ref meaning a field of type smartPtr not smartPtr* or smartPtr&. When such a smartPtr instance is cleaned up, we don’t need to worry about anything.

Now we are ready to understand circular reference.
– A smartPtr instance myCar holding a raw pointer to a Car instance, which has a field which is a smartPtr instance;
– this smartPtr instance holds raw pointer to a Driver instance.
– the driver instance has a field that is myCar.

All 6 objects are unreachable and unusable, but look at the 2 reference counts.

smart_ptr < double* > i.e. smart ptr of ptr

Should or can we ever put ptr-to-ptr into a smart ptr like smart_ptr?

My problem is the delete in the smart_ptr dtor — “delete ptr2ptr2dbl” where ptr2ptr2dbl is pointing at a 32-bit object (let’s call it “Dog123”). Dog123 is a 32-bit pointer to a double object. Is Dog123 on heap or on stack or in global area?

If dog123 is not in heap, the delete will crash (UndefinedBehavior) — deleting a pointer to stack (or global)

if dog123 is in heap, then how is it allocated? (Now it’s best to draw memory layout.) I feel it is almost always a field in a class/struct. Supposed the 32-bit object dog123 is also referred to as student1.pointerToFee, like

class student{
   double * pointerToFee;
// in this case, our smart pointer is perhaps set up like smart_ptr sptr( &(student1.pointerToFee) )

Now, if our “delete ptr2ptr2dbl” works, then the memory location occupied by Dog123 is reclaimed.
– that means the 64-bit double object is never reclaimed — memory leak
– also, student1.pointerToFee is in trouble. It’s not a wild pointer (its 64-bit pointee is ok), but the 32-bit memory location holding pointerToFee is reclaimed. On the real estate of student1instance, there’s now a 32-bit hole — 4 bytes de-allocated.

Q: how can we safely use this kind of smart pointer?
%%A: I guess we must use a custom deleter

Q: When would you need this kind of smart poitner?
%%A: I don’t think we need it at all.

#1 famous undefined behavior C++

(See below for smart ptr, template, non-RTTI)

Deleting [1] a derived[3] object via a base[4] pointer is undefined behavior if base[6] class has non-virtual dtor.

This is well-known but it applies to a very specific situation. Many similar situations aren’t described by this language rule —
[1a] This rule requires pointer delete. In contrast, automatic destruction of a non-ref “auto” variable (on stack) is unrelated.

[1b] This rule requires a heap object. Deleting a pointee on stack is a bug but it’s outside this rule.

[1c] This rule is about delete-expression, not delete[]

[3] if the object’s run-time type is base, then this rule is Inapplicable

[4] if the pointer is declared as pointer-to-derived, then Inapplicable

[3,4] if the object run time type is base, AND pointer is declared pointer-to-derived? Inapplicable — compiler or runtime would have failed much earlier before reaching this point.

[6] what if derived class has non-virtual dtor? Well, that implies base non-virtual too. So Yes applicable.

*) P62 [[effC++]] points out that even in the absence of virtual functions (i.e. in a world of non-RTTI objects), you can still hit this UB by deleting a subclass instance via a base pointer.

**) The same example also shows a derived class-template is considered just like a derived class. Let me spell out the entire rule — deleting an instance of a derived-class-template via a pointer to base-class-template is UB if the base class-template has a non-virtual dtor.

What if the pointee is deleted by a smart_ptr destructor? I think you can hit this UB.

iterator = simple smart ptr

– Iterators are *simple* extensions of raw pointers, whereas
– smart pointers are *grand* extensions of pointers.

They serve different purposes.

If an iterator is implemented as a class (template) then it usually defines
– operator=
– operator==, >=
operator* i.e. dereference
– operator++, —
– operator+, += i.e. long jumps
—-However, iterator class (template) won't define any operation unsupported by raw pointers, therefore no member functions! Smart pointers do define public ctor, get() etc

shared_ptr doesn’t like a pointee on Stack or global memory

Boost documentation says shared_ptr class template stores a pointer to a dynamically allocated object (not an “auto” object), typically allocated with a C++ new-expression (or array-new). The object pointed to is guaranteed to be deleted when…
If you feed it a stack object, then the eventual deletion will probably be undefined behaviour.

Q: Can you finish your task and exit the program before the eventual deletion? Maybe you don’t care?

Anyways, it’s unwise to create a library routine accepting a pointer argument and return a share_ptr. The library routine has no way to test if it points to the heap or the stack (or global area). Such a routine (if you must make one) had better force callers to pass in an explicit flag “isOnHeap”.

As stated in, the function should be designed to receive a shared_ptr as argument, not a raw ptr.

boost::serialization – learning notes

My learning notes based on the very clear and detailed tutorial

boost serialization features

* deep copy of pointer fields
* properly deserialize pointers to shared data
* proper support if a field is an array i.e. bunch of polymorphic pointers to Base and Der
* derived class serialization will include base instance state
* data and code portable across platforms – depend only on ANSI C++ facilities.
* when a class definition changed, older files can still be imported to the new version of the class. Similar to a hardcoded java serailVersion
—- More advanced —-
* fine if a field is a STL-container
* special considerations (tutorials) for shared_ptr — if a pointer field gives memory issues, convert it to a shared_ptr.
* special considerations (tutorials) for pimpl
* special considerations (tutorials) for xml serialization

In the simple GPS class (a serializable DTO), the class author must intrusively introduce a serialize() method, to be called by Hollywood during serialize/deserialize.

Intrusive solution is tailor-made by member function. Alternatively, one might opt for one-size-fit-all — non-intrusive serialization for classes without changing class definition. Class must expose enough information to save/reconstruct instance state. In the simple GPS example, we presumed that the class had public members – not a common occurrence.

The actual rendering of serialized data is implemented in the archive classes (text or xml or binary…). Thus the stream of serialized data is a joint effort between 1) serialization magic and 2) the archive class selected. It is a key design decision that these two components be independent. Interfaces to all archive classes are all identical. Once serialization has been defined for a class, that class can be serialized to any type of archive.

3 + N ways to create a shared_ptr

Background — Like any useful software component, Shared_ptr offers a clean api but has a sophisticated implementation, and many non-trivial interactions when integrated into an application — Complexity. Many ways to wrap your mind around the complexity. This is one of the best Perspectives.

The various ways to construct a shared_ptr clearly illustrates the fundamental use cases. An experienced user is probably familiar with all these use cases. These use cases are so important that you should be able to explain each constructor. Here’s what I remember.

1) pass in a raw ptr — starting a new club as the first member.
2) copier — joining an existing club.
3) pass in a weak_ptr
) pass in an auto_ptr

Note the good old auto_ptr is a club of 1. When “joining” a club you must remove the existing “owner”.

smart pointer casts – using shared_ptr for illustration

Smart pointers are the infrastructure of infrastructures for a quant library.

Minor performance issues become performance critical in this foundation component. One example is the allocation of the reference count. Another example is the concurrent update of the reference count. But here my focus is yet another performance issue — the lowly pointer-cast. (I don’t remember why performance is a problem … but I believe it is.)

C++ support 4 specific casts but smart pointers need to -efficiently- implement const_cast and dynamic_cast, in addition to the implicit upcast.

Q1: why can’t I assign a smartPtr-of-int instance to a variable declared as smartPtr of Dog?
A: compiler remembers the type of each pointer variable.

Q2: in that case, should c++ compiler allow me to assign a SP-of-Derived instance to a variable declared as SP of Base?
%%A: I don’t think so. I don’t think every smart pointer automatically supports upcast, but boost shared_ptr does, by design. See below.

For Boost, a Barcap expert said there’s an implicit cast from shared_ptr of T (RHS) to (LHS) shared_ptr of const T. See the special casting methods in

Specifically, boost says shared_ptr can be implicitly converted to shared_ptr whenever T* can be implicitly converted to U*. This mimics the behavior of raw pointers. In particular, shared_ptr is implicitly convertible to shared_ptr (i.e. add constness), to shared_ptr where U is an public/protected base of T (i.e. upcast), and to shared_ptr.I believe this is the same “member template” magic in the smart pointer chapter of [[more effC++]]

Allow me to explain the basics — in basic C++, a raw-ptr-to-T instance can be assigned to these variables —
T const *  my_ptr_to_const_T;
U * my_ptr_to_a_T_parent_class_U;
void * my_ptr_to_void;

Therefore a raw ptr supports these 3 (among others) casts implicitly. So does shared_ptr.

auto_ptr is still worth learning

auto_ptr was about the only smart pointer in the standard library (Until boost shared_ptr was added to TR1) and has been used in commercial apps for years. Obviously, it’s not designed for STL. Well, competent programmers won’t use auto_ptr in STL, so it’s basically robust and battle tested.

People say auto_ptr can lead to bugs in the hands of a novice, but what c++ features can’t?

The essential features of auto_ptr are very much relevant even if you use an alternative smart ptr. Numerous wall street interviews still reference auto_ptr.

In MTS, I simply replaced every “auto_ptr” in source code with “unique_ptr”, because auto_ptr has been deprecated in c++11.

DIY auto_ptr — NittyGrittyC++

P200 [[NittyGrittyC++]] shows that you can create your own wrapper class to ensure a local heap-allocated object always, always gets deallocated when the variable goes out of scope, even if exceptionally.

Once you wrap your local heap variable into this wrapper object, you can effectively forget deallocation. Just make sure you don't deallocate manually.

Then you realize — This is a partial auto_ptr, but it's actually better to forget about auto_ptr at this point. Auto_ptr has too many other features. Here we are only interested in the RAII dtor.

auto_ptr Achilles’ heel

auto_ptr would be fine if you don’t implicitly use copy-ctor or copy-assignment. This is the true Achille’s heel of this otherwise useful smart pointer.

Q: auto_ptr would be fine if you don’t ever use stl??? Well, non-STL functions also assume “normal” copy semantics??
A: I would grudgingly agree. auto_ptr users should simply avoid all such libraries.

The fact that auto_ptr is widely banned is testimony to the entrenched status of STL, and to the widespread use of implicit copy.

1) The c++11 unique_ptr is an improvement. Name suggests sole ownership.

2) Boost  scoped_ptr is also designed to be an improvement. Its name signal its intent to retain ownership solely within the current scope. According to boost documentation, it is “safer than shared_ptr or std::auto_ptr”. It has no more space overhead that a built-in pointer, so probably no other field than the built-in pointer. Almost bitwise const, except the reset(T*) and swap() methods.

Both these improvements address auto_ptr Achille’s heel.

smart ptr is 50% similar to a raw ptr

I’d say it’s 90% different and 10% similar to a raw ptr. Smart pointers are class objects, way beyond 32-bit pointers. smart pointers overload de-referencer and many other operators to “look like” a raw pointer, but it’s really a class-template (Avoid the non-standard “template-class” jargon)

– pbclone? A raw pointer is always passed by clone (bitwise clone), just as in java. Smart pointers override copier (and op=), so a new instance is created based on the RHS smart pointer instance. Is this pbclone or pbref? I’d say more like pbclone.
– When you use a star to dereference a raw ptr, you simply dereference (unwrap) the real pointer. Smart pointer dereference is a high-level user-defined operation. There’s a get() method to return the  real pointer, but I think we seldom need it.
– Similarly, you use an arrow to access a member of the pointee object, without the get() method.
– You can up/down cast raw pointers, not smart pointers.
– a raw ptr can be cast to void ptr. A smart ptr can’t.
– Raw pointers are key to virtual functions, not smart pointers.
– Creation and initialization is simple for raw pointers.
– size? A Smart pointer exceeds 32 bits except intrusive_ptr
– raw ptr has tight integration with arrays, with pointer arithmetic. Not smart ptr.
– double ptr sematics with 2 stars is natural for raw ptr, not smart ptr
– delete
– new? we can put raw ptr as lvalue of a “new” expression
– null assignment

In conclusion, raw pointers are such a part of the fabric. As a crude analogy, an e-book can feel like a paper book, but you can’t fold corners; can’t write anywhere using any pen; can’t spread 3 books out on your table; can’t tear off a page; can’t feel the thickness

smart ptr ^ weak reference ^ atomic ref

There’s some interesting resemblance between sptr and weakref. (Will address atomic ref later ..)Background — dominant construct in both c++ and java is the 32-bit pointer, assuming 32-bit address bus.

sptr and weakref
– both “wrap” the raw ptr
– both occupy more than 32 bits
– both introduce an extra level of indirection when accessing pointee
– both were designed for clever heap memory de-allocation
– when you need a raw ptr, you can’t pass in such a super ptr.
– in each case, 2 addresses are involved –
** pointee address and sptr’s own address
** pointee address and the weakref’s own address. When you pass the weakref, you don’t clone the weakref but you clone the 32-bit address of the weakref object.

auto_ptr vs scoped_ptr, in essence

(See also post on auto_ptr vs shared_ptr.)

Boost scoped_ptr ++forbids++ copy and assignment[2] whereas aptr permits these 2 accidents to happen.

scptr gives up ownership carefully, via reset, where as aptr does so accidentally and freely.

A const auto_ptr (i am not saying “auto_ptr to const”) is very similar to a scoped_ptr. There’s a small difference — P 13 [[boost]]

[2] Note some text explicitly says a scoped_ptr can’t be the LHS of assignment due to compilation error.

drop-in replacement for a raw ptr – barebones

(see also my little post on auto_ptr implementation, and Any Smart pointer must be a drop-in replacement for a raw ptr. At the minimum this requires

– overload deref operator
– overload arrow operator
– when an existing func requires a raw ptr argument, you can pass in smart ptr — Implicit conversion by …. [1]
– when an existing func returns raw ptr, but you receive it in a smart ptr — Implicit conversion by …… [2]

Above the minimum, it needs
+ [optional] overload increment/decrement? This is indeed part of the drop-in deal. Not easy. Not widely implemented.
+ ctor, dtor, copier, op=
+ how about explicit cast between smart ptr and the corresponding [3] raw ptr? Not part of drop-in replacement.
+ overload address-of operator? I feel no need
+ overload reference to the smart ptr? I feel no need

However, there’s a more stringent requirement that shared_ptr, auto_ptr don’t meet … same size as a raw ptr. Which smart ptr can achieve this? [5]

How about const_cast, dynamic_cast and upcast on raw pointer??

[3] smart ptr is parametrized so each concrete smart ptr type covers one type of raw ptr
[2] conversion ctor??
[1] operator-overload conversion method
[5] intrusive_ptr

boost weak_ptr can access inner ptr only through a shared_ptr

Unlike shared_ptr, scoped_ptr and auto_ptr, a boost weak_ptr doesn’t overload operator-} and operator*. Ditto for std::weak_ptr.

There’s an important implication.[2]

“If I want to veto deletion of internal ptr, then I must join the club, i.e. the club must register me as –veto– member”.

As a passive observer, weak_ptr is helpless to prevent the internal pointer’s demise. There’s an important implication again [3].

[2] No direct means to get the raw pointer from a weak_ptr. You must create and go through a shared_ptr, either this->lock() or shared_ptr conversion ctor.  I feel a weak_ptr instance is an observer using a shared_ptr instance as a lens.

[3] User of weak_ptr can’t reliably use the referenced resource, since the resource could disappear any time.

Crucially, a std::weak_ptr can be empty, as explained on P96 [[std c++lib]]. We can detect weak_ptr emptiness when the inner ptr is deleted. In contrast, raw ptr doesn’t offer this feature — if the underlying heap object is reclaimed, the raw ptr becomes dangling and we have absolutely no way to detect that, according to my friend Paul M.

boost shared_ptr implementation choices
describes linked list was once considered in Boost team, but the boost shared_ptr authors said in :

“The default behavior of shared_ptr is to allocate its control block
using new.” No linked list.

However, it was then argued that op-new is not always appropriate to allocate memory for the ref counter.

auto_ptr implementation notes – eSpeed IV

Someone asked me to implement a basic auto_ptr class template. It should be usable like

void foo(int * ip){…}

int* intP = new int(3);
AutoPtr aptr = ….. // create an AutoPtr object to encapsulate intP;

Implement dtor, copier, op=… for the AutoPtr class,

Overload dereference, arrow to mimic pointer behavior.

Q: And what else?
A: cvctor to convert int ptr to an AutoPtr

Q: how about an operator-overload-converter (OOC) to convert from an AutoPtr to an int ptr? Why do we need it?
A: an auto ptr should be usable as an int ptr, ie foo(aptr). This requires the OOC.

Now you see the need to implicit convert both ways between AutoPtr and raw ptr!

shared_ptr (sptr) vs auto_ptr (aptr)

Both of them delete intelligently. Both overload -> to mimic ptr syntax. See

aptr has unconventional copier/assignment, therefore unacceptable to STL containers; sptr is the only solution I know when storing pointers into containers — not scoped_ptr, not weak_ptr.

sptr is ref-counting; aptr isn’t as each aptr’s ref count is always 1.

aptr is sole-ownership; shared_ptr is shared ownership — that’s why there’s a ref count. shared ref count

shared_ptr ref counter implementation — a non-intrusive doubly-linked list to glue all sister shared_ptr instance. See

Q: can you delete a smart ptr as you do a raw pointer?
%%A: no. a smart ptr is a class template. u can’t call “delete someNonrefVariableOfAnyClass”

Q: if a smart ptr is a field, what happens during destruction?
%%A: see post on DCB. Field destructors are automatically invoked.

Key weakness of ref-count is the cycle. Use weak_ptr instead.

For container usage with inheritance, see

ref-count` && smart pointers in general

Java GC embraced and surpassed the ref counting capability of smart pointers. I doubt smart pointers handle islands, but do check out boost weak_ptr.

#1 motivation for smart pointers is memory issues like

– double-delete (ok if null)
– dangling pointer access (not “null” nor uninitialized), either read or write
– heap leak [1] — least dramatic but most common

As in any pbref environment, pointers are usually shared among functions, so a standard solution is ownership. Owner of a pointer takes care of deletion, using ref counting.

Another solution to the above memory issues — c++ references.

These memory issues are so common and serious that people come up many variations of these solution…

[1] not stack, so another solution to the list of memory issues — stackVars

container of (polymorphic) pointers (double-pointer)

If you have a vector of pointers (fairly common in practice), the iterator is nothing but a double pointer.

– If you have a deque of pointers …?
– If you have a multiset of pointers, the iterator is a pseudo-pointer of pointers.

Now shift gear to smart pointers. Container of smart pointer is extremely common in quant and other apps, perhaps more common than container of values.
– If you have a vector of smart pointers, the iterator is a pointer-to-smart-pointer.
– If you have a priority queue (binary heap) of smart pointers, the iterator is a pesudo-pointer-to-smart-pointer

P186 [[c++coding standards]] says to store polymorphic objects, best solution is
– avoid array
– avoid storing the objects directly
+ Use container of smart pointer to base class

Here are the 2nd-choice but acceptable alternatives
* array of raw   pointer to base class
* array of smart pointer to base class
* container of raw pointer to base class

shared_ptr QnA

I think we should learn shared_ptr (sptr) before weak_ptr, scoped_ptr or intrusive_ptr.

A shared_ptr object is a an object — NOT a variable. It can either be a stackVar or a field (or a global). As a stackVar, the var will get out of scope, and the shared_ptr’s destructor will run and do its job. As a field, will the sptr dtor do its job? yes see P22.

A shared_ptr (and its sisters) keeps the shared counter somewhere on heap. Allocation of this counter is considered expensive, since you may allocate millions of counters each second. See first answer

Some people may say a shared_ptr can be used much like a raw ptr, but i don’t agree. (For one thing, a raw ptr is not simple in terms of usage.) I feel you need to see enough (simple and non-trivial) examples.

Note some sptr ctor creates the “1st ref” in the ref-counting pointer group; other ctors create “2nd ref”. Make sure you know which ctor is which type.

Q: Are both the smae — mySp.get() vs *mySp
A: No. operator*() is equivalent to *mySp.get()

Q8a: is it normal to replace a ptr field with a sptr field?
A8a: yes. See eg on P21
Q8b: in the destructor, do you need to handle the sptr field as you would the raw ptr field?
A8b: i think u can relax. Dtor of the sptr field will be invoked in the DCB order. If there’s a raw ptr field, its dtor will run but it won’t call reclaim the pointee.

Q: do we ever call operator new and assign it to a sptr?
A: yes see p21

Q: is a sptr always cleaned up upon exception?
A: i think so. stack unwinding calls sptr object’s dtor

Q: if a method expects a particular ptr type, can you pass in a sptr?
A: i don’t think so. Compiler error. Compiler needs the type for memory allocation.

Q: if a method expects a particular type of sptr, can you pass in a raw ptr?
A: i don’t think so

Q: if a sptr stackVar goes out of scope, will the destructor get called?
A: yes P21

Q: deleting a sptr vs destructing a sptr?
A: a sptr is not a 4-byte thingy created with NEW, so u can’t call delete on it.

Q: do we ever destruct a shared_ptr explicitly?
A: i think so

auto_ptr – top 5 operations for a novice

An auto_ptr object wraps a heap[3] object, and deletes it judiciously. You don’t need to delete the heap object.

– If I go out of scope, I delete referent.
– release() ? detach from the referent without deletion, and hold null. A simple but important method.

– reset(p) ? delete current referent, and attach to incoming referent. see [1]. Note referent is a regular ptr, not auto_ptr

– auto_ptr as rval (ie RHS) in copier or assignment? I will hold null after ownership transfer. See [2]

– auto_ptr as lval ? delete current referent (if any), then become owner of the incoming referent, and release the RHS auto_ptr. see [1]

[1] before taking on a new referent object, I always delete current referent to avoid leak
[2] after giving up ownership, I hold null
[3] never wrap a non-heap object, or the delete() will crash

! not safe as elements in containers — non-standard copier and assignment
! if you independently put the same object into 2 auto_ptr, and one of them subsequently deletes the pointee, making the other a stray dog

standard auto_ptr is a template class …

intrusive_ptr is 32-bit; shared_ptr is at least 64-bit

First, remember a smart_ptr instance is a full-fledged object whose class often has methods or fields.

In contrast, a raw ptr instance is monolithic — not an instance of a class. Its pointee can often be an instance of a class with members, and you can access them like rawPtr->pointeeMethod2().

Since the counter is in the pointee object, an intrusive_ptr object needs no state. An intrusive_ptr do need methods. (Look, if a class has no, including inherited, field or method, then it’s useless.) Therefore, minimum members of an intrusive_ptr are

– internal pointer to the pointee object
– no other field
– ctor, dtor …
– no virtual functions

If an intrusive_ptr class uses no virtual function, then its size can be the size of the internal pointer, i.e. 32 bit, smaller than sizeof shared_ptr can’t. A shared_ptr object needs

– internal pointer to the pointee object
– some pointer that (indirectly) references the ref counter object somewhere in memory

Therefore, at least 64 bits, assuming a 32-bit machine.

intrusive_ptr and java interface

For a beginner to intrusive_ptr, we won’t be too far off to remember that a pointee class could (not required) implement a java-style interface with 2 public methods addRef and delRef(). The pointee class typically uses these methods to change a private field refCount.

In Boost, the intrusive_ptr actually uses 2 stipulated free functions, but they could (and often do) call methods of the pointee class, like our 2 methods.

intrusive ptr — intrusive access to pointee state

Q: Why the world “intrusive” in intrusive_ptr?

A: because this pointer manipulates state of the pointee object. Typically the pointee object provides/allocates/embeds the ref counter. When an intr pointer joins the “club” of intr pointers, the pointer initiates a call[1] to increment the counter.

A: because the more common smart pointers don’t mess with the state of pointee object. Therefore, the intrusiveness is a defining characteristic of this type of pointer.

[1] intrusive_ptr_add_ref(). P33 [[beyond the c++ standard library]] says that ==often== this function calls a *member* method of the pointee class, in order to manipulate the counter *field*. I feel this is a common solution.

intrusive/shared_ptr — warrants/options/fan-club

a payload-class, or pointee-class or “managed-class” is fully aware of the intrusive_ptr instances, like a warrant, where the issuer (say IBM) is aware of how many warrants are out there.

a managed-class is unaware of the shared_ptr instances circling around like sharks (ready to KILL it), a bit like an option — IBM doesn’t know how many IBM options are out there.

shared_ptr is also like a Fan club for a celebrity — celebrity doesn’t know about the fans.

“Payload” is clearer in a phone conversation.

conversion ctor ^ conversion method – pervasive

I feel the various type-converters are essential for code TRACING. Compiler implicitly uses conversion ctor (cvctor) and operator-overload-converter (OOC) whenever it needs to.

Note converter methods without “operator something” are explicitly invoked — much easier to spot. They often look like

   XXX asXXX(void)

As pointed out in other posts, cvctor and OOC can be invoked implicitly by static_cast into a nonref type.

Here are some converters —

Eg: scoped_lock myLock(myMutex);
Eg: string ( const char * s );
Eg: string.operator vector(), and string cvctor from vector. P197-198 [[STL tut]]
Eg: the conversions between auto_ptr, shared_ptr and raw ptr
Eg: conversion between vector iterators
Eg: back_inserter(myVector) returns a back_insert_iterator object