reference(instead of ptr) to smart ptr instance

I usually pass smart pointers by value (copy-constructor or move-constructor), just like copying a raw ptr.  Therefore the code below looks unnatural:

unique_ptr<Trade> & ref2smartPtr

Actually it is rather common because

  • As Herb Sutter suggested, when we need to put pointer into containers, we should avoid raw ptr. Unique ptr is the default choice, and the first choice, followed by shared_ptr
  • I often use unique_ptr as map value . The operator[] return type is a reference to the value type i.e. reference to unque_ptr
  • I may also put unique_ptr into a vector…. ditto for vector operator[]
Advertisements

ptr cast Can return a different addr !

Q: With dynamic_cast() on a ptr, when would the new address differ from the original address?

See https://github.com/tiger40490/repo1/blob/cpp1/cpp/88miscLang/downCast_MultiInherit.cpp.

Note the comments on pure-interface

RVO^move on return value

Let’s set the stage. A function returns a local Trade object “myTrade” by value. Will RVO kick in or move-semantic kicks in? Not both!

I had lots of confusions about these 2 features. P23 [[c++stdLib]] has 2 lines

  1. if Trade class has a suitable copy or move ctor, then compiler may choose to “elide the copy”. This was long implemented as RVO optimization in most compilers before c++11. https://github.com/tiger40490/repo1/blob/cpp1/cpp/rvr/moveOnlyType_pbvalue.cpp is my experiment.
  2. otherwise, if Trade class has a move ctor, the myTrade object is robbed

So if condition for RVO is present, then most likely your move-ctor will NOT run.

q[inline] to avoid a.. jump^stackFrame

Dino of BBG FX team asked me — when you mark a small f1()function inline (like manually copying the code into main()), you save yourself a jump or a new stack frame?

It turns out a new stack frame would require a jump, because after the new stack frame is created, thread jumps to the beginning of f1().

However, there’s something to set up before the jump — Suppose f1() is on Line 5 in main(), then Line 6’s address has to be saved to CPU register, or the thread has no idea where to jump back after returning from f1(). According to my codebashing training, this Line 6’s address is saved in the old stack frame, not the f1() stack frame!

Note the Line 6’s address is not a heap address not a stack address but an address in the code area.

concurrent lazy singleton using static-local var

https://stackoverflow.com/questions/26013650/threadsafe-lazy-initialization-static-vs-stdcall-once-vs-double-checked-locki has 20 upvotes and looks substantiated. It also considers double-checking, std::call_once, atomics, CAS…

GCC uses platform-specific tricks to ensure a static local variable is initialized only once, on first use. 

If it works, this is the easiest solution.

As explained in another blog post, static local is a shared mutable.

pbclone large obj(eg:vector)rely`@move

This is impressive in QQ interviews + coding questions

GotW #90 Solution: Factories

has a good illustration of move semantics put to good use.

  • Before c++11, a function returning a large vector (or any large object) by value incurs expensive deep copying of all vector elements.
  • With c++11 move features added to std::vector class, returning a vector by value is cheap and recommended.
  • RVO may kick in but (i feel) less reliable than move semantic. For the specific rules see RVO^move-semantics

housekeeping^payload fields: vector,string,shared_ptr

See also std::string/vector are on heap; reserve() to avoid re-allocation

std::vector — payload is an array on heap. Housekeeping fields hold things like size, capacity, pointer to the array. These fields are allocated either on stack or heap or global area depending on your variable declaration.

  • Most STL (and boost) containers are similar to vector in terms of memory allocation
  • std::string — payload is a char-array on heap, so it can expand both ways. Housekeeping data includes size…
  • shared_ptr — payload includes a ref counter and a raw-pointer object [1] on heap. This is the control-block shared by all “club members”. There’s still some housekeeping data (pointer to the control block), typically allocated on stack if you declare the shared_ptr object on stack and then use RAII.

If you use “new vector” or “new std::string”, then the housekeeping data will also live on stack, but I find this practice less common.

[1] this is a 32-byte pointer object, not a pure address. See 3 meanings of POINTER + tip on q(delete this)

memset: a practical usage #Gregory

  • memset is a low-level C function.
  • memset takes a void pointer.
  • Fast and simple way to zero out an array of struct, having primitive data members. No std::string please. No ptr please. Use sizeof to get the byte count.
  • Useful in low level wire coding
// illustrates packed and memset
#include <iostream>
using namespace std;

struct A{
  unsigned int i1; //4 bytes
  bool b; //1 byte
  char cstr[2];
  int* ptr; //8 bytes
} __attribute__((packed));
size_t const cnt = 3;
A arr[cnt];
int main(){
  cout<<sizeof(A)<<endl;
  size_t sz = sizeof(arr);
  cout<<sz<<endl;
  memset(arr, 0, sz);
  for(size_t i=0; i<cnt; ++i){
    A* tmp = &arr[i];
    cout<<"i1 = "<<tmp->i1<<"; b = "<<tmp->b<<" ; cstr[1] = "<<(int)tmp->cstr[1]<<" ptr = "<<tmp->ptr<<endl;
  }
}

memory leak demo; memset(); std::string

valgrind proves the leak.

using namespace std;
size_t const len=15;
struct A{
        string s4;
        A(string s="default std::string"): s4(s){ cout<<"ctor"<<endl; }
};
size_t const  cnt=2;
size_t siz=cnt * sizeof(A);
A arr[cnt], ar2[cnt];

char fname[] = "/tmp/,.dat";
void leakDemo1() {
        {
                arr[0]=A("grin");
                arr[1]=A("frown"); //somehow skipping this causes core dump
        }
        cout<<"before write()"<<endl;

        int fd = open(fname, O_CREAT | O_WRONLY, S_IRUSR | S_IWUSR);
        write(fd, arr, siz);
        close(fd);

        if(1){
                int fd2 = open(fname, O_RDONLY);
                read(fd2, ar2, siz);
                close(fd2);
        }
        for (int idx = 0; idx < cnt; ++idx){
                A * tmp = ar2 + idx;
                cout<<tmp->s4<<endl;
        }
}
void leakDemo2(){
        int * intp = new int(11); //not deleted. valgrind detected the leak
        memset(&intp, 0, 8); //overwrite the 8-byte pointer object
        delete intp;  //deleting on the fake address from memset
}
void leakDemo3(){
        string s="hello";
        cout<<"sie of string == size of pointer = " << sizeof(string)<<endl; //inside the string object, there's nothing but a poitner!
        memset(&s, 0, 1); // overwite pointer object itself, so it now points to somewhere else ... leak

        //somehow, memset(....,2) causes seg fault?
}
int main() {
        leakDemo1();
}

c++variables: !! always objects

Every variable that holds data is an object. Objects are created either with static duration (sometimes by defining rather than declaring the variable), with automatic duration (declaration alone) or with dynamic duration via malloc().

That’s the short version. Here’s the long version:

  • heap objects — have no name, no host variable, no door plate. They only have addresses. The address could be saved in a “pointer-object”, which is a can of worm.
    • In many cases, this heap address is passed around without any pointer object.
  • stack variables (including function parameters) — each stack object has a name (multiple possible?) i.e. the host variable name, like a door plate on the memory location. This memory is allocated when the stack frame is created. When you clone a stack variable you get a cloned object.
    • Advanced — You could create a reference to the stack object, when you pass the host variable by-reference into another function. However, you should never return a stack variable by reference.
  • static Locals — the name myStaticLocal is a door plate on the memory location. This memory is allocated the first time this function is run. You can return a reference to myStaticLocal.
  • file-scope static objects — memory is allocated at program initialization, but if you have many of them scattered in many files, their order of initialization is unpredictable. The name myStaticVar is a door plate on that memory location, but this name is visible only within this /compilation-unit/. If you declare and define it (in one step!) in a shared header file (bad practice) then you get multiple instances of it:(
  • extern static objects — Again, you declare and define it in one step, in one file — ODR. All other compilation units  would need an extern declaration. An extern declaration doesn’t define storage for this object:)
  • static fields — are tricky. The variable name is there after you declare it, but it is a door plate without a door. It only becomes a door plate on a storage location when you allocate storage i.e. create the object by “defining” the host variable. There’s also one-definition-rule (ODR) for static fields, so you first declare the field without defining it, then you define it elsewhere. See https://bintanvictor.wordpress.com/2017/05/30/declared-but-undefined-variable-in-c/

Note: thread_local is a fourth storage duration, after 1) dynamic, 2) automatic and 3) static

c++non-void function +! a return value

Strictly, undefined behavior not a compiler error. https://stackoverflow.com/questions/9936011/if-a-function-returns-no-value-with-a-valid-return-type-is-it-okay-to-for-the explains the rationale.

However, in practice,

  • For an int function the compiler could return any int value.
  • For functions returning type AA, I don’t know what is returned. Could it be a default-constructed instance of AA?
    • My specific case — I modified a well-behaving function to introduce an exception. I then added a catch-all block without a return value. Actually worked fine. So some instance of AA is actually returned!

 

use Unsigned-char-array(i.e. ByteArray)to transfer binary chunk

If you intend to store or transfer arbitrary binary data, you should use …. unsigned char. ANSI-C has no “byte” data type.

It is the only data type that is guaranteed (by the ANSI C Standard) to have no padding bits. So all 8 bits in an unsigned char contribute to the value. None of them is a padding bit.

Contrary to some online posts, unsigned-char type is different from “char” —

op-new : no DCBC rule

B’s op-new is bypassed by D’s op-new [1]
B’s ctor is always used (never bypassed) by D’s ctor.

This is a interesting difference.

Similarly, an umbrella class’s op-new [1] would not call a member object’s op-new. See [[more effC++]]

These issues are real concerns if you want to use op-new to prohibit heap instantiation of your class.

See http://bigblog.tanbin.com/2012/01/dcbc-dtor-execution-order.html

[1] provided these two classes each define an op-new()

By the way, op-new is a static member operator, but is still inherited.

mv-semantics: MSDN article

http://blogs.msdn.com/b/vcblog/archive/2009/02/03/rvalue-references-c-0x-features-in-vc10-part-2.aspx is one of the best articles to shed lights on this confusing topic, written by someone in the Visual c++ core team.

One comment says “…this is mainly a feature for use in library-code. It’s mostly transparent to client-code which will just silently benefit from it”. How true!

I always believed move-semantic is a non-trivial topic. Many authors try to dumb it down and make it accessible to the mere mortals, but I feel a correct understanding takes effort.

Scott Meyers articles are also in-depth but seem to skip the basics.

As I mentioned earlier, [[c++ primer]] has the best deep-intro on this confusing topic. Again, the author is another legend in the c++ community.

lval(i.e. L-value) – my definition, incl c++ func CALL

See also post on rval reference

int& foo();
int* p1 = &foo(); // ok, foo() is an lvalue
foo(5) = 42; // (A) ok, foo() is an lvalue. Java doesn’t allow it but does allow …
java ##  foo[5]=42; // (B)

p1  = … //ok
*p1 = … //ok
*(…) = …//ok. Most deference expressions can be L-value expressions

C++ lets you overload the subscript operator i.e. brackets so (B) can be implemented as a shorthand for (A).
————–
C# indexer and property both look like fields of an object (fields are L-values), but both get converted to getter/setter functions. Therefore they don’t refer to memory locations. Therefore an indexer/property can’t be ref or out arguments. They can be used as regular method arguments though, because they are good enough as R-values.

field^param^local-variable — C++ allocation

A field is always allocated memory, since a (including static) field is part of an object.

A function parameter is always allocated on the stack.

Local variables are supposed to be allocated on stack, but may not be allocated at all. Compiler can often optimize them away.

Sometimes a variable is just a token/alias in source code’s symbol table. A constant variable can be replaced by the constant value at compile time.

How about initialization?
Rule 1: class instances are never UNinitialized
Rule 2: static variables are never UNinitialized
Rule 3: local vars and class fields are uninitialized except as part of Rule 1.

non-dummy-type template parameters

(Note this topic is not related to template Partial specialization)

First, let’s distinguish a template parameter (like “T”) vs a template argument (like a class “Account”). For a NDTTP, the template parameter has a) concrete type and b) a parameter name, typically “size_t value” vs the template argument like “31”.

Simplest example: std::array template has 2 template parameters —

  1. a dummy type T
  2. a non-dummy type “size_t length”

std::array<Account, 31> concretizes the template with a concrete type Account and a value of 31.

Majority of class templates in practice have a dummy type (or multiple, but let’s stay focused) to signify an Unknown-type. For example, the STL vector can be “concretized” in memory to become a real CLASS when given a real type like “Account”. The real type replaces the dummy type. [[C++ primer]] is the first book I read that mentions a very powerful type of class template. I call it NDTTP i.e. non-dummy-type template parameter.

Background — What appears inside angle brackets after the keyword “template” is always a list of “tokens” or “thingies”. In most cases in practice, each thingy is a “typename T” or “class C”.

Now, the thingy can also be something like “int size”. This is known as a non-type template parameter. I call it a non-dummy-type template parameter, or NDT template parameter. a NDT is not a dummy type, but a real type with a parameter name. NDT declaration syntax is like a regular function parameter. NDT represents a dummy type pinned to a real type.  But how is NDT used when the push comes to the shove i.e. template instantiation (I prefer “template concretization”)?

Remember a real type argument like “AccountAllocator” in STL vector is a piece of information at the class level, not Instance level. When the vector is concretized in memory to become a real/concrete Class, the real class uses AccountAllocator Class. In the same vein, the NDT value is at class-level, not class-instance level nor at template level. That implies the “int size” value of 55 is a constant for the class, across all class instances.

In other words, when we convert a GENERIC “unconcretized” template matrix_double into a real class, the “int size” template parameter (NDT template parameter) is replaced by a value like 55, and treated as a class-level static constant. If we construct 9999 instances of the concrete matrix_double class, all of them share the same size=55.

A class template can have all its “tokens” be dummy types (standard practice), or all of them NDT (P861 [[c++primer]]), or a mixture.

See other posts about When the NDT arg is a specific func ptr like “revalue”. In contrast, given a functor class, you use a regular type param (more common), not a NDTTP.

implement vtbl using ANSI-C

See also [[understanding and using c pointers]], which has about 5 pages (with sample code) on how to implement polymorphism. It covers some non-trivial details of the implementation.

Basic techniques and principles…. Emphasis in this write-up is clarity through simplicity, not rigor or correctness.

_instance_field_ — implemented as field of struct.

** Let’s say the struct type is MyClass.

_instance_method_ — function pointer field in the struct. Each function is an ordinary free functions taking ptr-to-MyClass as 1st parameter. Compiler converts all instance-method calls to function calls with “this” as 1st argument.

_static_method_ — instance methods without that 1st parameter.
_this_ — special hidden read-only field in MyClass of “ptr-to-MyClass” type. Through this pointer, Each instance MyClass knows the address of its own real-estate. As explained in other posts such as Object.java size, such a 32-bit real estate usage in every MyClass instance is rather costly and probably avoided in a c++ compiler. Can we avoid it in our home-made class?
_virtual_methods_ — a bunch of identically named free functions each taking a different type of 1st parameter. Note overload isn’t allowed in C.
— ptr-to-ClassB vs ptr-to-ClassD
_vptr_ — another hidden field in the struct, pointing to an array of function pointers.
_inheritance_ — MyClassD struct encloses/embeds (not a pointer to but) an entire MyClassB struct. Note the instances of MyClassB and MyClassD have the same address, permitting pointer cast.
_private_ — a compile-time access check on members of the struct

return value optimization, again

See also http://bigblog.tanbin.com/2010/09/return-value-optimization.html. There’s also a brief but complete item in [[More eff c++]]

Q : what are required for RVO? (From MS CVA interview. Obscure detail but a halo)
A: copy/move must be available (not private not deleted)

Q: when would RVO kick in?
%%A:
– object created as a nonref auto/local/stackVar in the called function
– return by value (pbclone)
– return value is assigned to a new variable.

Q: what conditions would suppress RVO?
??

Such performance features are good to know, but I feel we don’t need to predict, count on or avoid RVO. In a given context, it may or may not happen and our app should work correctly nevertheless.

piecemeal vs one-gulp – c++ object memory allocation

– some operations must be piecemeal
– some operations must be one-gulp

* ctor and dtor are always piece-meal. Reason — each subclass and embedded component class can define its own ctor and dtor.
** For example, dtor execute in the DCBC sequence — http://bigblog.tanbin.com/2010/01/d-c-b-destructor-execution-order.html

* “operator new” (I don’t mean a new-expression) is always one-gulp. Reason — the memory (“real-estate”) of an entire object must be a contiguous piece of “real estate”, but one allocation for one part of the real estate and another allocation for another part will be dis-contiguous.
** There’s an important implication on P157 [[more eff C++]]

Background —
+ if you are a derived class instance, Your base class instance will occupy part of “your” real estate
+ if you have a nonref[1] field of a another class, That instance occupies part of “your” real estate

[1] “nonref” is something other than a pointer or reference, something not dereferencible. No such field in java, because a non-primitive field is always a reference field in any java class.

when Must RHS be a L-value expression

Advanced!

Q: when must the RHS be a L-value expression?
%%A: it must if LHS is a non-const reference variable, as the var must bind a Location. This applies to function arguments, too.

C++ true-blue call-by-reference requires the argument (i.e. RHS) be an L-value. RHS can’t be a literal “3” or a math expression. An L-value has an address so can be passed by reference.

Q: can RHS be a new-expression? Does it qualify as a RHS L-value?
A: YES tested — qq/ *(new int(8)) /, presumably because this object has an address

Q: what if the LHS is a ref-to-const?
A: then RHS can be literals like “3”. Tested.

Q: how about a function call returning an object? Can this be an RHS L-value?
A: YES if the function returns by reference. Tested. Such a function-call IS a true-blue L-value-expression.
A: NO if the function returns by value. Such a function-call ISN’T an L-value-expression.

return value optimization

I got a surprise when I run the program below with g++ (mingw port).

* returnLocalVar() should return by value, by calling the copy constructor, but copy constructor didn't get called.
* localCat object should be destructed when it goes out of scope at end of the function, but no it was kept alive till end of main()
* ret is a local variable in main(), not a reference, so i expect ret to be a copy of localCat, with a separate address, but no they 2 variables have the same address.

#include
#define S(x) std::cout<<#x" address =\t"<<&x<<"\n";
using namespace std;
struct CAT {
    CAT() {
        cout < no-arg constructor with addr = ” << this << "\n";
    } // default constructor
    ~CAT() {
        cout << this << " Destructor called!\n";
    }
    CAT(const CAT& rhs) {
        cout < copy constructor ” << this << " <- " << &rhs << "\n";
    }
};
CAT returnLocalVar() {
    CAT localCat;
    cout << "leaving returnCat\n";
    return localCat;
    // localCat is not destructed if result is assigned
}
int main() {
    CAT ret = returnLocalVar();
    S(ret);
    cout << "done!\n";
}

Answer from a friend:

return value optimization (RVO). It is an optimization specifically blessed by the C++ specification. While the compiler is creating the returnLocalVar function, it is allowed for it to create the CAT temporary in main’s stack space instead of returnLocalVar’s stack space. Then, instead of returning it by making a copy, it just leaves it there on the stack where main can find it. The compiler can’t always figure out whether that would be safe, so it only does it in predictable circumstances.

prefix/postfix increment – my cheatsheet

Based on [[moreEffC++]].

Q (important): why return by reference in one scenario and return by value in another?

It’s by necessity, not by preference. We _always_ prefer return by reference — efficient. But one of them (quiz – which one?) must return the “Before image” (database lingo), so it has to copy the object and return the copy, while the original object is updated.

Implication — the return-by-reference operator (quiz – which one?) is always More efficient, therefore we must Change the old habit of (ab)using “myIterator++” —- must be changed.

Q (non-trivial): why const?
This is a simple (not the only) solution to disallow myVar++++

Q (trivial): which operator takes an argument?
A: in fact neither needs argument, but one of them takes a fake argument of int. I guess it’s not really important to remember which one

"this" as implicit arg to non-static c++ methods

Compiler adds an implicit param to each non-static method. Note in both java and c++, non-static method invocation absolutely[2] needs a receiving object ie the “this” pointer. C++ merely add that pointer as the implicit argument. In the example below, the object handle [3] is passed as argument to the method as if (suppose j1 is an instance of class J)

m(this, other_args) — binds to –> m(J*, other_params)

Runtime parameter passing mechanism includes an implicit pbref

J* this_ptr = & j1;

Since constness radiates left, a const j1 (RHS) requires the param (LHS) declared const. You can mark the method m() const. It’s interpreted as

m(J const *, other_params), so now the method call does a pbref

J const * this_ptr = & j1;

In conclusion —- you can use a non-const “handle”[3] to invoke a const method.

[2] There’s simply no exception. A non-static method invocation is a message, and needs an recipient object.
[3] handle can be a ptr, a reference, or a nonref variable

template struct J{
	J(T & rhs){}
	void m(T a) {
	  this->intVar = a; // legal when method is const but no one calls.
	}
	T intVar;
};
int main()
{
	int a=22;
	const J j1(a);
	j1.m(89);
	return 0;
}

c++ inheriting private fields^methods

* Private fields – onions
* Private methods – utility methods on a single onion-layer

As [[Absolute C++]] P598 puts it, private methods are completely unavailable to subclasses. Private methods are useful as utility methods on-that-layer of the onion, not outside, not inside.

A base class (B) object is wrapped in a subclass (D) object like onions. The physical molecules of each layer are the instance fields. When you construct a new D, every single B instance field must be allocated memory (Java == C++). However, private B fields are inherited but inaccessible in D’s methods [java == c++]. Often D can access B’s private field privfield1 using accessors [java == c++].

Q: can D methods access qq(B::privfield1)?
A: I think so

return mem cells to freelist, then fill your pointer’s 32 bits with 0

Suppose a 32-bit pointer p2 contains address 0x1234,and 22==sizeof(Derived)

******* q[base p2 = new Derived()] implements —
– grab 22 bytes from freelist
– initialize by ctor
– return 0x1234 i.e. starting address of the 22 bytes
–> from now on, “system” won’t give those 22 bytes to other malloc requests, since those are ALLOCATED to us.

****** q(delete p2) performs
– call dtor, possibly virtual
– determine size of the (polymorphic) pointee object, possibly via vtbl. Suppose it’s 22.
(The compile-time declared type of p2 doesn’t matter. If we did a Base p2 = new Derived(), then deletion will reclaim all the mem cells of Derived.)
– mark that many bytes starting from Address 0x1234
– return these bytes to freelist
–> Any time from now on, those 22 bytes can be allocated to anyone. Note p2 continues to be seated at 0x1234. The deletion performs no state change in the 32-bit object p2. If you read/write/delete on p2, you die.

Rule — after you call q(delete p2), always fill the 32-bit object p2 with zeros, by calling q(p2 = 0)

local variable declaration IS allocation@@

– local nonref (i.e. stackVar) declaration always allocate-create the object — the C tradition. (java made a bold departure.) This is the big deal in this post.

MyClass c; // allocates without initializing. calls noArg ctor? I doubt it?
MyClass c= …. // ditto

– local ptr variable declaration — always allocates the 32 bits for the pointer. [1]
– local ref variable declaration — always allocates and initializes the reference. Compare to ptr.
– function param declaration (nonref) always allocate-create the object, on stack.
– A field declaration doesn’t allocate-create the field object — not until host class construction-time. Same as in java.
^ obviously, if you see new … then you know it calls constructor, new-expression is fundamentally different from nonref variable declarations because
^^ heap
^^ returns pointer
^^ object created is nameless. In contrast, nonref variable is a name-plate carved on a memory location.

[1] It may initialize it to 0 or seat it to a real pointee object

implicit low-level non-OO operations — in-depth lesson #1 on c++

This self-contained, focused, in-depth lesson covers copier, assignment, destructors, heap/stack, new/delete, pbclone/pbref (see separate post). I feel it pays to slow down and get the concepts right before moving on.

This lesson doesn’t need any OO feature.

The following implicit operations are “part of the fabric” ie implicit and pervasive
* assignment
* constructor calls at var declarations
* pbclone/pbref is even more part-of-the-fabric than others
* heap/stack is a prerequisite of pbclone/pbref

—– 2nd lesson —–
What would be the 2nd “focus lesson”? inheritance + redefinition/overriding + slicing.

————————
After the inheritance lesson, we could tackle static member and templates in inheritance.

How many such lessons needed before becoming “experienced”? I guess about 3 or 4.