##java8 features #MethodRef

  • lambda
  • default methods
  • streaming
  • method references? Supposed to be an important feature in the core language/syntax. related to lambda syntax, presumably a syntactic sugar coating.
  • java.util.Optional<T> introduced to reduce NPE. Compare boost::optional
  • CompletableFuture<T> to promote async, event-driven programming
  • PermGen disappears in java8
  • –not really java8 features:
  • java7 introduced G1 garbage collector though default GC remains ParallelGC, for both java8 and java7
Advertisements

need a safe invalid value for a c++float@@ NaN

— For c++ float, if you need a safe “invalid” value, there’s NaN, with standard support like std::isnana() etc

— For c++ int, you need to pick a actually-valid number like INT_MAX.

Q: How do you find a special value to indicate “variable has an invalid value”?
%%A: I think you need separate boolean flag.
A: boost::optional #NaN #a G9 boost construct is exactly designed for this

posix^SysV-sharedMem^MMF

http://www.boost.org/doc/libs/1_65_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.sharedmemory.xsi_shared_memory points out

  • Boost.Interprocess provides portable shared memory in terms of POSIX semantics. I think this is the simplest or default mode of Boost.Interprocess. (There are at least two other modes.)
  • Unlike POSIX shared memory segments, SysV shared memory segments are not identified by names but by ‘keys’. SysV shared memory mechanism is quite popular and portable, and it’s not based in file-mapping semantics, but it uses special system functions (shmgetshmatshmdtshmctl…).
  • We could say that memory-mapped files offer the same interprocess communication services as shared memory, with the addition of filesystem persistence. However, as the operating system has to synchronize the file contents with the memory contents, memory-mapped files are not as fast as shared memory. Therefore, I don’t see any market value in this knowledge.

posix sharedMem: key points { boost

http://www.boost.org/doc/libs/1_65_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.sharedmemory.shared_memory_steps is excellent summary

* We (the app developer) need to pick a unique name for the shared memory region, managed by the kernel.

* we can use create_only, open_only or open_or_create

* When we link (or “attach” in sysV lingo) App1’s memory space to the shared memory region, the operating system looks for a big enough memory address range in App1’s address space and marks that address range as an special range. Changes in that address range are automatically seen by App2 that also has mapped the same shared memory object.

* As shared memory has kernel or filesystem persistence, we must explicitly destroy it.

Above is the posix mode. The sysV mode is somewhat different.

##boost modules USED ]finance n asked]IV

#1) shared_ptr (+ intrusive_ptr) — I feel more than half  (70%?) of the “finance” boost usage is here. I feel every architect who chooses boost will use shared_ptr
#2) boost thread
#3) hash tables

Morgan uses Function, Tuple, Any, TypeTraits, mpl

Gregory (RTS) said bbg used shared_ptr, hash tables and function binders from boost

Overall, I feel these topics are mostly QQ types, frequently asked in IV (esp. those in bold) but not central to the language. I would put only smart pointers in my Tier 1 or Tier 2.

—— other modules used in my systems
* tuple
* regex
* noncopyable — for singletons
** private derivation
** prevents synthesized {op= and copier}, since base class is noncopyable.

* polymorphic_cast
* numeric_cast — mostly int types, also float types
* bind?
* scoped_ptr — as non-copyable [1] stackVar,
[1] different from auto_ptr

sample code showing boost scoped_lock# !! c++14

#include <iostream>
#include <boost/thread.hpp>
#include <string>
#include <iostream>
#include <sstream>
using namespace std;
boost::posix_time::milliseconds sleepTime(1);
 
template<typename T>
class MyQueue {
 public:
 void enqueue(const T& x) {
  cout << "\t\t\t > enqueuing ... " << x << "\n";
  boost::mutex::scoped_lock    myScopedLock(this->mutex_);
  cout << "\t\t\t >> just got lock ... " << x << "\n";
  list_.push_back(x);
  // A scoped_lock is destroyed (and thus unlocked) when it goes out of scope
 }
 T dequeue() {
  boost::mutex::scoped_lock lock(this->mutex_);
  if (list_.empty()) {
   throw 0; // unlock
  }
  T tmp = list_.front();
  list_.pop_front();
  cout << "< dequeu " << tmp << "\n";
  return (tmp);
 }
 private:
 std::list<T> list_;
 boost::mutex mutex_;
};
MyQueue<std::string> queueOfStrings;
int reps = 5;
void sendSomething() {
 for (int i = 0; i < reps; ++i) {
  stringstream st;
  st << i;
  queueOfStrings.enqueue("item_" + st.str());
  boost::this_thread::sleep(sleepTime);
 }
}
void recvSomething() {
 for (int i = 0; i < reps*3; ++i) {
  try {
   queueOfStrings.dequeue();
  } catch (int ex) {
   cout << "<- - (    ) after releasing lock, catching " << ex <<"\n";
   boost::this_thread::sleep(sleepTime*2);
  }
 }
}
int main() {
 boost::thread thr1(sendSomething);
 boost::thread thr2(recvSomething);
 thr1.join();
 thr2.join();
}

boost intrusive smart ptr phrase book #no IV

* MI — P35 [[beyond c++ standard lib]] shows your pointee CLASS can be written to multiple-inherit from a general-purpose ref_count holder class. This is a good use case of multiple inheritance, perhaps in Barclays QA codebase.

* real-estate — The 32-bit ref counter lives physically on the real estate of the pointee. Pointee type can’t be a builtin like “float”. In contrast, a club of shared_ptr instances share a single “club-count” that’s allocated outside any shared_ptr instance.

* legacy — many legacy smart pointer classes were written with a ref count in the pointee CLASS like QA YieldCurve. As a replacement for the legacy smart pointer, intrusive_ptr is more natural than shared_ptr.

* builtin — pointee class should not be a builtin type like float or int. They don’t embed ref count on their real estate; They can’t inherit; …

* TR1 — not in TR1 (http://en.cppreference.com/w/cpp/memory), but popular

* ref-count — provided by the raw pointee Instance, not the smart pointer Instance. Note the Raw pointER Instance is always 32 bit (assuming 32-bit bus) and can never host the reference count.

* same-size — as a raw ptr

* expose — The pointee class must expose mutator methods on the ref count field

Boost Any: cheatsheet

* bite-sized introduction — best is P164 [[beyond c++ standard lib]]. Best 2nd intro is P165.
* Most essential operations on Any are

1) ctors — desposit into safebox
2) any_cast — check out using the “key”

* void pointer — Any is better than void pointers. (It’s good to know that void pointers meet the same basic requirement.)
* shared_ptr — to store pointers in an Any instance, use shared_ptr.
* STL — Any can be used with or without STL containers. First get a firm grip on one of them.

— P164 – without containers
myAny=std::string(“….”);
myAny=3.281; // a double
//As shown you can put any object into the variable. To retrieve it, you specify the expected type.
any_cast (myAny);
any_cast (myAny); // would throw exception.

Q: big 3?
Q: can subclass?
Q: name a real use case
Q: this sounds powerful, convenient, too good to be true. What’s the catch? Why not widely used?

STL/boost in quant lib

To a quant developer, I feel STL containers are more useful than STL algorithms. However, if you use STL containers, then some STL algorithms will be handy.

1-D and 2-D arrays are the bread-and-butter of quant lib. STL vector and vector-of-vector are good enough.

Sets and maps are widely used too in a few banks.

Daniel Duffy said STL isn’t enough for quant, but according to 2 friends (GS and an FX house) STL is the most important library for quant codebase; boost is clearly the 2nd (for quant library).
* The most important boost component is smart pointers.
* boost serialization

C-str ^ std::string (+boost) – industry adoption

I believe the boost features aren’t needed for coding test. C++ coding IV doesn’t care std::string or cStr. Perhaps std::string is easier.

C++ threading, c++ trading, c++ pricing, c++ connectivity… all use C string functions like a workhorse. STL string class is still largely displaced by C string.

I think many teams in need of custom optimization create their homemade string classes.

Boost offers a bunch of standalone functions as string utilities + the tokenizer class. See http://www.boost.org/doc/libs/1_36_0/doc/html/string_algo.html, also covered in my boost book.

wait() locking/unlocking #boost

For both boost-thread, java 1.4 wait() and java Condition variables, here’s the sequence of events
————————————–
1) before wait(), application programmer must lock explicitly.
——– going into wait() ——-
2) unlock implicitly

3) lock implicitly
——– return from wait() ——

Q: look at (3) — will an abnormal “exceptional” return from wait() still grab the lock?

A: yes for boost. This is a choke point — before the thread emerges from “under water” and awakes from “the avatar dream”, it must grab the lock — grabbing indefinitely if it’s not available.

A: yes for java 1.4. i guess the wait() method never throws any checked or unchecked exception except those 2 listed in API. The standard wait() call is wrapped in synchronized block, so it’s obvious that exceptional return from wait() must grab the lock.

A: ditto for Condition.java. See example in javadoc

thread detach — boost ^ java

In most (if not all) systems, a thread object is a “poor handle” on a real world thread.

in java, if the thread object becomes unreachable (and garbage collected) then you can't join or interrupt the thread, or test if it holds a particular lock. You have to wait for it to finish or pull the plug with System.exit().

Same scenario exists in boost thread. Additionally, boost lets you explicitly call myThread.detach(), to decouple the myThread object and the real world thread.

Q: after detach(), what's the thread object?
A: it represents not-a-thread

cyclic barrier – boost thread ^ java

A quick and memorable analogy of a boost thread barrier object (of the barrier CLASS). Think of such an object as a thread-safe collection (say vector) of 3 items. 3 threads all have a pointer to the same collection.

Thread A calls myBarrier.wait() inside some function1, to remove one item, and blocks
Thread B does the same
Thread C does the same in some function2. Now the collection is empty, all 3 threads join the Eligible club (as described in the post on Eligible^Running)

java offers CyclicBarrier.

boost thread to run a method of my object (full source

#include
#include
class Worker{
public:
    Worker()    {
        // the thread is not-a-thread ie a dummy thread object, until we assign a real thread object
    }
    void start(int N)    {
        // pass “this” as first arg to processQueue(), which runs not as a method but as a free func in its own thread
        m_Thread = boost::thread(&Worker::processQueue, this, N);
    }
    void join(){m_Thread.join();}
    void processQueue(unsigned N)    {
        float ms = N * 1e3;
        boost::posix_time::milliseconds workTime(ms);
        std::cout << "Worker: started, will work for "
                  << ms << "ms"
                  << std::endl;
        boost::this_thread::sleep(workTime);
        std::cout << "Worker: completed" << std::endl;
    }
private:
    boost::thread m_Thread;
};
int main(int argc, char* argv[]){
    std::cout << "main: startup" << std::endl;
    Worker worker;
    worker.start(3);
    std::cout << "main: waiting for thread" << std::endl;
    worker.join();
    std::cout << "main: done" << std::endl;
    return 0;
}

RAII lock auto-release with boost SCOPED_lock

#include
#include
#include
#include
#include
using namespace std;
boost::posix_time::milliseconds sleepTime(1);

template
class MyQueue {
public:
    void enqueue(const T& x) {
        cout < enqueuing … ” << x << "n";
        boost::mutex::scoped_lock myScopedLock(mutex_);
        cout <> just got lock … ” << x << "n";
        list_.push_back(x);
        // A scoped_lock is destroyed (and thus unlocked) when it goes out of scope
    }
    T dequeue() {
        boost::mutex::scoped_lock lock(mutex_);
        if (list_.empty()) {
            throw “empty”; // unlock
        }
        T tmp = list_.front();
        list_.pop_front();
        cout << "< dequeu " << tmp << "n";
        return (tmp);
    }
private:
    std::list list_;
    boost::mutex mutex_;
};
MyQueue queueOfStrings;
int reps = 5;
void sendSomething() {
    std::string s;
    for (int i = 0; i < reps; ++i) {
        stringstream st;
        st << i;
        s = “item_” + st.str();
        queueOfStrings.enqueue(s);
        boost::this_thread::sleep(sleepTime);
    }
}
void recvSomething() {
    std::string s;
    for (int i = 0; i < reps*3; ++i) {
        try {
            s = queueOfStrings.dequeue();
        } catch (…) {
            cout << "<- – (    ) after releasing lock n";
            boost::this_thread::sleep(sleepTime*2);
        }
    }
}
int main() {
    boost::thread thr1(sendSomething);
    boost::thread thr2(recvSomething);
    thr1.join();
    thr2.join();
}

simple boost thread program

#include
#include
#include

void workerFunc() {
    boost::posix_time::seconds workTime(3);
    std::cout << "Worker: starting up then going to sleep" << std::endl;
    boost::this_thread::sleep(workTime); // Thread.sleep
    std::cout << "Worker: finished" << std::endl;
}
int main(int argc, char* argv[]) {
    std::cout << "main: startup" << std::endl;
    boost::thread workerThread(workerFunc); // pass a func ptr to thread ctor
    boost::posix_time::seconds workTime(1);
    std::cout << "main: sleeping" << std::endl;
    boost::this_thread::sleep(workTime); // Thread.sleep
    std::cout << "main: waking up and waiting for thread" << std::endl;
    workerThread.join();
    std::cout << "main: done" << std::endl;
    return 0;
}

boost thread — recursive, try-lock, reader/writer, timeout …

These lock features are implemented at different layers in Boost.

In java, Reentrance is the only option — all locks are recursive. Reader/writer lock is a feature built on top of the basic reentrant lock. tryLock and lock timeout are basic features of the Lock interface.

In Boost, those CONCEPTS, lock types, free functions, typedef are noises to a beginner;) Specifically, people say mutex objects are accessed not directly but through wrapper, but the wrappers add noise. Actually, core classes[1] are mutex and unique_lock. Both of them support try-locking. However, to understand our big 4 features, it’s cleaner to focus on the mutex classes —

* Try-lock — supported by all mutex classes
* Timed locking — supported by a subset of the mutex classes, namely timed_mutex and recursive_timed_mutex.
* Reader/writer — supported by exactly one mutex class, namely shared_mutex.
* Reentrance — supported by a subset of the mutex classes, namely recursive_mutex and recursive_timed_mutex. Note Reentrance is implemented by these 2 mutex classes only, and not in the Lockable concepts or those complicated lock types. Just scan the boost documentation.

Once we are clear on these features in the mutex classes, we can understand the structure among the Lockable CONCEPTS —

+ Try-lock — supported by all Lockable concepts.
+ Timed locking — TimedLockable and derivatives
+ Reader/writer — SharedLockable only.
(Reentrance — not in these concepts)

[1]How about the workhorse —> scoped_lock? Confusing to some novices, scoped_lock is a typedef of unique_lock

msvs – integrate boost into msvs

My worst initial fears in c++ dev are A) IDEs like visual studio, B)external libraries [1] and integration of A and B.

http://www.boost.org/doc/libs/1_52_0/more/getting_started/windows.html presents comprehensive tutorials showing
– How to add boost header files into include path
– How to add boost precompiled binary files into the link path

I feel in most projects, the compiler only need these 2 “locations” in order to use an External library.

Just a side note, compiler/linker needs to reference 2 types of external library content
– source code – by preprocessor copy-paste
– compiled code – by linker

[1] external meaning not built into the IDE, like boost library

boost thread described by its creator

Excerpts from http://www.drdobbs.com/cpp/184401518. There's also a very short program featuring boost mutex, and a program featuring boost bind, …

Many C++ experts provided input to the design of Boost.Threads. The interface is not just a simple wrapper around any C threading API. Many features of C++ (such as the existence of constructors/destructors, function objects, and templates) were fully utilized to make the interface more flexible. The current implementation works for POSIX, Win32

Currently (2002), not a lot can be done with a thread object created in Boost.Threads. In fact only two operations can be performed. Thread objects can easily be compared for equality or inequality using the == and != operators to verify if they refer to the same thread of execution, and you can wait for a thread to complete by calling boost::thread::join. Other threading libraries allow you to perform other operations with a thread (for example, set its priority or even cancel it). However, because these operations don’t easily map into portable interfaces….

Boost.Threads is designed to make deadlock very difficult. No direct access to operations for locking and unlocking any of the mutex types is available. Instead, mutex classes define nested typedefs for types that implement the RAII (Resource Acquisition in Initialization) idiom for locking and unlocking a mutex. This is known as the Scoped Lock pattern. To construct one of these types, pass in a reference to a mutex. The constructor locks the mutex and the destructor unlocks it. C++ language rules ensure the destructor will always be called, so even when an exception is thrown, the mutex will always be unlocked properly.

up and down casting nonref/ref/ptr

Primary usage of dynamic_cast is down-cast
* base/derived class must have vptr, or you get compile-time error
* original and target types must be ptr/ref, or you get compile-time error [1]
* there’s just one object involved, not 2
** That one object must be a complete and full[2] Derived object, otherwise you get runtime (not compile time) failure, indicated by 1) null ptr or 2) exception (during reference down-cast)
*** boost polymorphic_cast adds uniformity by throwing exception for both

[1] static_cast can cast nonref.
[2] static_cast doesn’t incur the overhead to check that

Q: up-casting a nonref??
A: no casting operator required, but you get sliced. qq/ myB = myD; /
A: by the way, no such thing in java, since java doesn’t use “nonref”

Q: down-casting a nonref??
A: impossible in dynamic_cast. “dynamic_cast can be used only with pointers and references to objects”

Q: down-casting a ptr (to a polymorphic object only)?
A: dynamic_cast. May return NULL. java has a similar feature.
A: see also boost polymophic_cast

Q: down-casting a ref (to a polymorphic object only)?
A: dynamic_cast. Never returns NULL. .. down cast a reference

Q: down-cast a base ptr (or ref) to a derived object but no vtbl/vptr no virt func?
A: impossible. dynamic_cast won’t compile.

Q: up-cast a ptr?
A: common in c++ and java. everyday virtual function scenario. no operator required.

Q: up-cast a ref?
A: legit but less common than ptr. See post on virtual^redefining