cloud: more relevant to system- than software-architects

Software architects worry about libraries, OO designs, software components, concurrency etc.

They also worry about build and release.

How about app integration? I think this can be job for system architect or software architect. If the apps run in the same machine, then probably software architect.

— 10 considerations to develop for cloud

https://www.developer.com/cloud/ten-considerations-for-realizing-the-potential-of-the-cloud.html was a 2015 short article written by a java developer. I like the breakdown into 10 items.

Some of the 10 items are more devops than developer considerations. The other items are more for system architect than software architect.

However, hiring managers expect a senior candidates like me to demonstrate these QQ knowledge. By definition, QQ is not needed on the job.

 

rvalue objects before/after c++11

Those rvalue objects i.e. unnamed temp objects have been around for years. So how is rvr needed to handle rvalue-objects?

  • C++11 added language features (move,forward,rvr..) only to support resource stealing where resource is almost always some heapy thingy.
  • Before c++11, rvalue objects don’t need a special notation and don’t need a special handle (i.e. rvr). They are treated just like a special type of object. You were able to steal resources, but error-prone and unsafe.

pointer as class member #key points

Pointer as a field of a class is uninitialized by default, as explained in uninitialized is either pointer or primitive. I think Ashish’s test shows it

However, such a field creates an opportunity for mv-ctor. The referent object is usually on heap on in static memory

If you replace the raw ptr field with a smart pointer, then your copier, op=, dtor etc can all become simpler, but this technique is somehow not so popular.

Note “pointer”  has a well-defined meaning as explained in 3 meanings of POINTER + tip on q[delete this)]

enumerate()iterate py list/str with idx+val

The built-in enumerate() is a nice optional feature. If you don’t want to remember this simple simple syntax, then yes you can just iterate over xrange(len(the_sequence))

https://www.afternerd.com/blog/python-enumerate/#enumerate-list is illustrated with examples.

— to enumerate backward,

Since enumerate() returns a generator and generators can’t be reversed, you need to convert it to a list first.

for i, v in reversed(list(enumerate(vec)))

Y avoid us`boxed Integer as mutex

https://stackoverflow.com/questions/34049186/why-not-lock-on-a-value-based-class section on “UPDATE – 2019/05/18” has a great illustration

Auto-boxing of “33” usually produces distinct objects each time, but could also produce the same object repeatedly. Compiler has the leeway to optimize, just as in c++.

Remember: locking is based on object identity.

git | reword historical commit msg

Warning — may not work if there’s a recent merge-in on your branch

Find the target commit and its immediate parent commit.

git rebase -i the.parent.commit

First commit in the list would be your target commit. Use ‘r’ for the target commit and don’t change other commits. You will land in vim to edit the original bad commit msg. Once you save and quit vim, the rebase will complete, usually without error.

Now you can reword subsequent commit messages.

c++low-^high-end job market prospect

As of 2019, c++ low-end jobs are becoming scarce but high-end jobs continue to show robust demand. I think you can see those jobs across many web2.0 companies.

Therefore, it appears that only high-end developers are needed. The way they select a candidates is … QQ. I have just accumulated the minimum critical mass for self-sustained renewal.

In contrast, I continue to hold my position in high-end coreJava QQ interviews.

y smart ptr popular in c++IV

I think even if a hiring team doesn’t use smart ptr (for that matter, also TMP, threading,,), they would drill into this topic as it is a good probe on big4 and a lot of other core language features.

See some of the essential requirements on a smart ptr in drop-in replacement class for a raw ptr. Some of these items are part of an interview discussion on smart ptr.

reliable multicast for replicated-cache update

https://www.usenix.org/conference/usits-99/scalable-web-caching-frequently-updated-objects-using-reliable-multicast is a 1999 research paper. I hope by now multicast has grown more mature more proven. Not sure where this is used, perhaps within certain network boundaries such as a private network of data servers.

This paper examines reliable multicast for invalidation and delivery of popular, frequently updated objects to web cache proxies.

## defaultdict tricks #matrix

  • defaultdict(list) — is one of the most used. I think “list” is a function pointer.

https://github.com/tiger40490/repo1/blob/py1/py/algo_tree/commonAncestor_Indeed.py demos a few time-saving tricks with defaultdict

  • mydict[key] # without printing or using the return value … would trigger the dictionary to construct a default value for this key IFF missing
  • from key/value pairs, build dict {key -> valueList}

https://github.com/tiger40490/repo1/blob/py1/py/algo_combo_perm/staircase_CSY.py shows

  • defaultdict(int) as a frequency counter.
    • mydict[key] += 1 #works even when key was previously missing in mydict
  • defaultdict(lambda: ‘_’).
    • mydict[(row,col)] can implement a matrix. negative (or other invalid) indices results in default value of string ‘_’. I believe dict key can be tuple but not list.
    • In contrast, the 2D array interprets negative indices as wrap-around !
  • defaultdict(lambda: [[],[]]) creates a list of 2 small lists for any “invalid” key. In contrast defaultdict(list) will create a monolithic list which hits runtime error when you access element of the nested list i.e. dict[someKey][0][anyIndex]

context switch: 1-10 microsec

See also #CPU-cycles]1 time slice ~ millions #AshS 

(A thread context switch is faster….) could be 10 millisec, in theory. See P 351 [[linux kernel]]

In a MS interview, Deepak was asked about (typical) duration of a TaskSwitch or context switch. I think 20 millisec is way too long.

https://stackoverflow.com/questions/21887797/what-is-the-overhead-of-a-context-switch and other sites suggest “below 10 microsecond”

— One of the most interesting and popular focus points is the difference between thread switching vs process switching

The L1-cache (private to a cpu core) reload happens if a core switches to another Process…?

 

demanding mental power: localSys imt QQ

I told grandpa that I have noticed signs that my mental power is declining, mostly in localSys and local codebase absorbency. However, localSys has always been a weakness in me. The decline, if any, is very gradual.

QQ learning is still in good progress. Experience, accumulation, deepening.. thick->thin.

Note A.Brooks’ article talks about creativity, innovation, analytical … My issue is mostly memory capacity and speed. Recall the Indeed interview.

BST: finding next greater node is O(1) #Kyle

https://stackoverflow.com/questions/11779859/whats-the-time-complexity-of-iterating-through-a-stdset-stdmap shows that in a std::map, the time complexity of in-order iterating every node is O(N) from lowest to highest.

Therefore, each step is O(1) amortized. In other words, finding next higher node in such a BST is a constant-time operation.

https://en.cppreference.com/w/cpp/iterator/next confirms that std::next() is O(1) if the steps to advance is a single step. Even better, across all STL containers, iterator movement by N steps is O(N) except for random-access iterators, which are even faster, at O(1).

Kyle’s simple trie ] python

https://github.com/tiger40490/repo1/blob/py1/py/algo_tree/trie_Kyle.py is Usable for some rare coding IV. By Kyle Stewart.

https://leetcode.com/problems/implement-trie-prefix-tree/ has a slightly more complete requirement.

I think I can modify (not write from scratch) Kyle’s code to add the word search.

Looks similar to the 10-billion phone number problem

void list.reverse() ^ reversed()→iterator #sorted(),items()

These three constructs do three different things, without overlap

  • mylist[::-1] returns a reversed clone
    • myStr[::-1] works too
  • mylist.reverse() is a void in-place mutation, same as mylist.sort()
  • reversed(mylist) returns an iterator, not printable but can be used as argument to some other functions
    • I feel this can be convenient in some contexts.
    • reversed(myStr) works too but you can’t print it directly

Similarly,

  • Unlike items(),dict.iteritems()return iterator

Different case:

  • sorted(any_iterable) returns a new vector (i.e. “list”). Returning an iterator would be technically stupid.

generate Fib(N) in O(1)time and space: 2 techniques

Like all Constant-recursive sequences, the Fib sequence has a closed-form formula Fib(N) that returns an integer but in floating point form. By rounding, we can generate Fib(N) in constant time.

https://www.math.hmc.edu/funfacts/ffiles/10002.4-5.shtml has the formula we can easily program.

[ Phin – ( -Phi)-n ] /√5  # Notice the three minus signs

where the constant Phi := (1+√5) / 2.

Similarly, if we are given 89 as a Fib number, we can use logarithm to compute the rank of this Fib number.

https://github.com/tiger40490/repo1/blob/cpp1/cpp/lang_33template/FibDeepak.cpp shows two O(1) solutions

max salary: simple game-plan

The strategy — “Whether I ask for a big base or modest base, there’s a chance I may have problem with manager expectation. So let’s just go for the max salary and forget about learning/tsn

    • algo trading? tend to pay a premium, but I wonder how they assess my trec.
    • java/c++ combo role? will Not pay lower
    • Some quant skill? tend to pay a premium
    • If a HFT shop makes a real offer at S$150k base I will decline — no real upside for me. Similarly, if a quant dev job pays $170k base I will decline — the promised accu (across jobs) is a big mirage. Accu can happen within a single job, but so is the technical accu on a single job.

Max-salary game plan must not ignore :

  • correlation between salary and expectation — as observed in some past jobs but not in every lucrative role. My Barclays and 95G roles were great.
  • the stigma, damagedGoods and high expectations in Stirt and Macq…. Ashish’s view — just earn the money for 6 months and leave if not happy.
  • commute
  • reputation risk at the major banks.

Am i still a survivor? I would say YES in OC and GS, and yes in Macq based on the internal transfer offer.

Mithun suggested — Are we traumatized/scarred and fixated on the stigma? I said the same to Deepak CM.

pass generator output to next generator

I think this technique can be extremely time-saving in coding tests.

https://github.com/tiger40490/repo1/blob/py1/py/algo_combo_perm/1fromEachSet.py my code demos:

for myset in pool:     output = list(gen(output, myset))

The gen() function uses yield. For the first call to gen(), we exhaust all of its items and save into a list named “output”.

Then we pass this list into the second gen(), this time with a different myset

luckily free of negativity@CIV #CSY#CShi

I feel lucky about freedom from negativity towards coding drill.

Major competitive advantage! Most of my peers are bogged down by this negativity.

I think most of the younger guys (like Cheng Shi) would NOT sustain continuous coding drill as described in [18] Continuous coding drill #Shi . I guess there’s negativity in most of these individuals.

checked STL^checked java Collections

jdk checkedList, checkedMap etc are designed to check type errors — checking any newly added item has the correct type for the collection. See P 246 [[java generics]]

STL checked container checks very different coding errors. See http://www.informit.com/articles/article.aspx?p=373341, which is extracted from my book [[c++codingStd]]

Docker containers

Container is widely used in cloud computing; Docker is a “popular new kid” in jxee, probably not in coreJava.

https://jaxenter.com/nobody-puts-java-container-139373.html is not too shallow, and not too deep. Free of marketing propaganda.

  • containers are standard Linux processes running a shared kernel, not isolated kernels

https://www.redhat.com/en/topics/containers/what-is-docker addresses several confusions and then contrasts raw linux containers with docker containers.

— docker and container

The word “Docker” refers to several things, including

  • tools from an open source project;
  • additional tools from Docker Inc., the company that primarily supports that project

Here’s a brief explainer:

  • The software “Docker” (community or commercial) is containerization technology that enables the creation and use of Linux® containers.
  • The company, Docker Inc., builds on the work of the Docker community, makes it more secure, and shares those advancements back to the greater community. It then supports the improved and hardened technologies for enterprise customers.

I feel docker is a facade/wrapper over raw linux container features.

The tools built on top of Linux containers—what makes Docker user-friendly and unique—gives users unprecedented access to apps, the ability to rapidly deploy, and control over versions and version distribution.

git | commit – -am # risky

Consider the harmless-looking command git commit –am

Most of the time, I rely on this command as a safe way to amend commit message without touching the original files in the commit.

However, if there’s further changes on file1 in staging, then this command would silently introduce this change into the commit. Once you finalize the commit message and save it, this change inadvertently becomes part of the commit !

techShop-plan: rare opp2try tech start-ups

The Sg2019 presents a rare opportunity to work for such a firm and gain some insight how they select candidates.

Salary might be slightly lower like 150k vs 180k@SCB, but i think it’s completely fine, until we (irrationally) compare to some peers.

The inherent risk of stigma (damagedGood, PIP..) are lower since I’m older and newer, bringing down the expectation. I would feel like Michael Jordan joining baseball.

new c++features=mostly4lib developers

In C++0x, Half of the major new language features are designed for the standard library developers.

  • The unspoken assumption — these features are equally useful to other library developers.
  • The follow-up assumption — app developers also need to use some (and understand all) of the nitty-gritty details.

In reality, these nitty-gritty details are Not relevant to GTD for app developers.

missing q[ return ] in non-void function #Josh

I told Josh about this real Scenario: my non-void function has 3 paths to exit the function, one of them without a return statement.

g++ usually give a warning under “-Wall”. Flowing off the end of a function is equivalent to a return with no value — undefined behavior in non-void function.

If you print the function return value and the 3rd path executes, you get undefined behavior.

https://stackoverflow.com/questions/3402178/omitting-return-statement-in-c shows -Wreturn-type

I believe it’s a compilation error in other languages.

avoid lopsided BST

Self-balanced is not a theoretical nicety but an essential BST feature. Without it a BST could easily become lopsided and all operations will slow down.

If for any reason (I don’t know any) we can’t use a AVL or RBT, then we could randomize the input and insert them (without deletion) and get a fairly balanced BST.

competitiveness: Pre-teen sprinter + HFT selectivity

I was a decent sprinter at age 6 through 13, then I realized I was not good enough to compete at Beijing municipal level.

There are quite a number of HFT shops in Singapore. I think they have a higher bar than ibanks. At ibank interviews I felt again like that pre-teen sprinter, but HFT interview is generally too hard for me

prevent c++compiler reordering statements

http://www.cplusplus.com/reference/atomic/atomic_signal_fence/ says this low-level c++11 function is something like a

directive to the compiler inhibiting it from making optimizations that involve moving writing operations beyond a releasing fence or read operations before an acquire fence.

I doubt there’s any need for this in any application.

1)num-theory 2)combinatorics: most-quizzed math

  • For regular programmers, most quizzed non-trivial subdomains of math are 1) number theory and 2) combinatorics.
  • For Quant developer interview is a different game, but these subdomains are still highly relevant.

These math sub-domains would have remained obscure if programming as (a profession) had not become this wide spread, mainstream, lucrative (油水) and economically important. Since the 2000’s, more and more IV focus is shifted to math, largely in these two sub-domains.

This trend is bad news for the GTD types like Ashish/CSY, the domain experts and the managers like Shuo etc

I think some strong STEM students don’t bother with coding skill since it is less emphasized, less quizzed in school.

I suspect that in some colleges, math is regarded as more refined, more classic than coding skill in any language such as python or javascript. Coding is considered blue-collar.

java setting -Xmx Twice

https://stackoverflow.com/questions/2740725/duplicated-java-runtime-options-what-is-the-order-of-preference

$ java -XX:+PrintFlagsFinal -Xmx2G -Xmx1G 2>&1 |grep MaxHeap #shows the 2nd setting would overwrite the first setting

uintx MaxHeapSize                              := 1073741824                          {product}

export JAVA_OPTIONS=-Xmx…. # would append a -Xmx. The last -Xmx value overrides previous

 

 

restatement in FIX: triggers

http://fixwiki.org/fixwiki/ExecutionReport — represents an ExecutionRpt sent by the sellside communicating a change in the order (or a restatement of the order”s parameters) WITHOUT an electronic [1] request from the buyside.

“unsolicited order replace”

[1] this exec report can be triggered by a non-electronic request from buyside

A less common trigger — repricing of orders by the sellside such as making Sell Short orders compliant with uptick / downtick rules

UDP: send/receive buffers are configurable #CSY

It’s wrong to say UDP uses a small receive buffer but doesn’t use send buffer.

Receive — https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html shows how to increase UDP receive buffer to 25MB

Send — https://stackoverflow.com/questions/2031109/understanding-set-getsockopt-so-sndbuf) shows you CAN configure send buffer on a UDP socket.

Send — https://www.ibm.com/support/knowledgecenter/en/SSB23S_1.1.0.15/gtpc2/cpp_sendto.html also confirms SO_SNDBUF send-buffer size applies to UDP sockets

In addition, Application is free to use its own custom buffer in the form of a vector for example.

##lockfree queue implementations c++J

I think a linked queue with 2 pointers (head / tail) could be relatively easy to implement by myself.

%%approaches to classic generator problemS

* p2u in next_perm

* insert at each position of each array, in a growing collection to create new arrays for the collection. Optionally, all arrays are immutable.

Actually, only two approaches so far, though each of them can be adapted in many ways.

Note many (simple) classic generator algos are required in coding tests or (practical) algo problems

compare-up too much; too little compare-out

Comparison with “peers” is really hard to rationalize away.

Now my habit is 80% compare-up; and only 20% of the time we feel fortunate comparing to the wider population.

It’s better to become 20% compare-up.

Reason – Those up-comparisons are usually impractical, irrational and with wrong groups i.e. managers.

c++ lockfree data types: mostly bool/int/pointer

  • In generally, only atomic bool/int/pointer can be lock-free. Bigger types need special implementation techniques and I have read that lock-free is usually not possible.
  • Atomic flags are lock-free (this is the only type guaranteed to be lock-free on all library implementations)

const vector{string} won’t let you modify strings #ChengShi

Point #1: a const vector only gives out const references. In this case, reference to const string objects.

My friend ChengShi said he has never seen const vector<const string>, because const vector<string> is good.

Point #2: In fact, my experiment of q[vector<const string>] can’t compile. Why?

https://stackoverflow.com/questions/17313062/vector-of-const-objects-giving-compile-error explains that in vector<T>, T must be assignable!

git | list all(and only)commits from branching point to a descendant commit

Background — Suppose you made 3 commits on your feature branch name “parix”, but meanwhile, someone added 4 commits in master branch. Therefore there is now a divergence in the commit history graph.

Often times you need to visualize the divergence. You need to exactly what 3 commits are on your branch after the common ancestor.

git log master..pairx # listing the 3 additional commits in pairx branch, right after the common ancestor i.e. the branching point

git log pairx..master # to show those 4 commits.