commit involves two transitions between three “trees”:
- from local tree -> staging
- from staging -> index
git-reset has three strengths:
- soft: reset the index tree only
- mixed: reset the staging tree too
- hard: reset local tree and all other trees
commit involves two transitions between three “trees”:
git-reset has three strengths:
— optimistic reading using a StampedLock object
A thread can hold this object in optRead mode. This mode can be thought of as an extremely weak version of a readLock, that can be broken by a writer at any time.
Your thread would first tryOptimisticRead() to get a temporary stamp. Then validate(theStamp) would pass iFF still “free”.
After first successful validation, you should quickly read just a few data items, and validate() again to confirm availability.
Remember that, with zero notice a writeLock can be created from this object, invalidating your optRead lock.
OptRead lock is designed for quick small reads, like stealing
— relation to the (interview-wise) hotter AtomicStampedReference
— PendingCancel report
— cancel completed i.e. Out report
In 2017 I wrote to Gerald Robinson that spare time depletion would become one of the key challenges/pains of my forthcoming/upcoming adjustments.
For many (probably most) of my peers, if they don’t feel the drive to keep learning in their spare time, the resource depletion doesn’t feel so harmful, hazardous, stressful.. Beside this “learning”, I also have a need for quiet time for reflection.
— commute — Bayonne commute is tolerable only when I have plenty of spare time as “slack resource”.
— Parenting — (esp. my son) became a huge commitment and strain on my slack resources.
— Additional paid annual leave is supposed to be a slack resource but I think it doesn’t reduce the pressure to perform
… but I did spend these hours (weekends + evenings) in office to learn localSys, provided I’m engaged, in the mood.
Need to think harder about how to turn these
large quantities of high-quality spare resource into stress-relief. These resources feel like untapped oil reserve.
Interviewers often focus on x86 instruction sets, but I could point out competing instruction sets.
NYFIX is a business, traded under the confusing ticker “NYFX”.
https://www.itiviti.com/managed-fix-services/nyfix-order-routing-network has a nice description.
I think NYFIX is a technology and “managed service” provider. I feel it might be similar to Bloomberg trading platforms.
What’s in common between a luxury car and a brank like “Director”?
My contractor job is like a mid-range reliable Japanese car.
jolt: FSM dividend has never delivered the de-stressor as /envisioned/. In contrast, my GRR has produced /substantial/ nonwork income, but still insufficient to counter the threat of PIP!
In my experience, Job-loss stressor is indeed alleviated by this income or the promise thereof 🙂
Excluding the unlucky (broken, sick,,) families, I feel
most “ordinary” people’s stress primarily come from burn rate i.e. making ends meet, including job-loss fear. I feel the middle class families around me could survive at a much lower theoretical burn rate of SGD 3.5-4.5k (or USD 6k perhaps… no 1st-hand experience) but they choose the rat race of keeping up with the Jones’s (FOMO). Therefore, their burn rate becomes 10k. See also SG: bare-bones ffree=realistic #WBank^wife^Che^3k.
For some, FOMO takes them to the next level — bench-marking against the high-flyers.
— jolt: PIP^job-loss fear
For the middle class, any burn rate exceeding 3k is a real (albeit subconscious) stressor because the working adults now need to keep a job and build up a job-loss contingency reserve. Remember Davis Wei….3-month is tough for him? How about 30 years?
With a full time job, salaried workers experience a full spectrum of stressors including PIP. PIP would be impotent/toothless if the job is like a hobby. I would say very few people have such a job.
Aha .. Contract career is free of PIP.
For me, job loss is a lighter stressor than PIP fear. In fact, I don’t worry about end of contract and bench time. I worry more about humiliating bonus.
Most people assume that my PIP fear was fundamentally related to cashflow worry, but I am confident about job hunting. So my PIP fear is all about ego and unrelated to cashflow.
— https://www.channelnewsasia.com/news/commentary/career-mobility-new-normal-career-stability-job-hopping-11469760 is a Singapore perspective:
Long gone is the notion of the career ladder, where the ideal CV looks like a narrow, vertical progression. Today’s gold-standard CV looks like a career matrix, with horizontal and vertical moves signifying depth and breadth of experience, skills and exposure to different cultures.
Employers have gone from being cynical about hiring job-hoppers to becoming accustomed to seeing diverse CVs from top talent who are in frequent demand.
I often say that the young, bright candidates basically ignore contract jobs — no benefit, no leave, no promotion prospect.
“many jobseekers turn away such contracts, choosing to hold out for something long-term that may or may not come”
RTS uses no STL containers.
In my first programming job, I used C without malloc, because I used only array + struct. If my collection were to grow at run time, then I would have needed DAM.
I feel low latency generally avoid mid-day DAM. If DAM required, then pre-allocation is preferred.
Design a car park class for concurrent access given
Without using locks, Provide enter() and leave() API methods that can be safely invoked on multiple threads.
— If at any time there’s only one thread calling enter (and leave), then volatile-based solution might work. I think this assumption is reasonable. Given there’s only one physical entry point, it’s not physically possible for two threads to run enter() simultaneously.
volatile int sensorA, sensorB;
count = this.senserA – this.sensorB;
if (count > CAPACITY) throw IllegalStateException;
if (count == CAPACITY) return 0;
this.senserA ++ ;
//concurrent access disallowed for this R/W operation
this.sensorB ++ ;
— If multiple threads are allowed to increment this.senserB, then we need (either a lock or) atomicInteger. ABA problem is not a concern here given the integer never decrements.
features/advantages of sorted collection
Q: insert is typically O(log N), but can we improve on it?
— A: if there’s only one insert, we can hold it in a hidden field of the collection, to be checked at the end of each operation above.
— A: if there are unlimited inserts but all 64-bit integers, then we can treat the incoming integer as a short radix array (8-elements for example). Take each element of the array as a lookup key, look up in an array to find a subtree.
Basically the entire sorted collection is an 8-level tree with fan_out=256
You may say that there are 8 lookups required so O(k) where K=8, but here we assume the size of the radix array is a constant equal to eight. Note the collection size is unlimited and far more than 2^64 if duplicates are permitted.
I have a real self-esteem problem as I tend to belittle my theoretical and low-level technical strength. CHENG, Shi was the first to point out “你就是比别人强”.
Nowadays I routinely trivialize my academic strength/trec relative to my sister’s professional success. To be fair, I should say my success was more admirable if measured against an objective standard.
Q: do you feel any IQ-measured intelligence is overvalued?
Q: do you feel anything intellectual (including everything theoretical) is overvalued?
Q: do you feel entire engineering education is too theoretical and overvalued? This system has evolved for a century in all successful nations.
The merit-based immigration process focus on expertise. Teaching positions require expertise. When laymen know you are a professional they expect you to have expertise. What kind of knowledge? Not GTD but published body of jargon and “bookish” knowledge based on verifiable facts.
!! how is this related to the CSY staircase problem?
Q: given K dollars and N distinct mutual funds (including a cash fund), how many unique allocation strategies are there? It’s permissible to allocate $0 to any fund, but total amount in a strategy must add up to exactly K dollars
Note K can be higher or lower than N, both positive integers. Allocation amount must be whole dollars.
Without loss of generality, let’s say K is $5 and N is 3.
I think this is more a mathematical than an algorithm question. Each strategy can be represented by an N-element array. [0,5,0] is one strategy, allocation $5 to 2nd fund and $0 to the other funds. If we are required to enumerate all the strategies, we can print out the arrays.
My solution starts by defining a mathematical function f(k, n) := count of allocation strategies given k dollars and n funds. Based on this definition,
Without loss of generality, let’s say K is 33 and N is 22. Using bottom-up DP, I assert that when we get a “new” fund, the 22nd fund, we can use the earlier answers to find f(33,22):
f($33,22) = f($33,21)f($0,1) + f($32,21)f($1,1) + f($31,21)f($2,1) +… +f($1,21)f($32,1)+f($0,21)f($33,1)
Let me explain the 3rd term on the right-hand-side. This term means the count for “$31 into the 21 existing funds” multiplied by the count for “$2 into the new fund”. This describes the scenario of first allocating $2 to the new fund, and then allocating the remaining $31 to the existing funds. The above formula simplifies to
f($33,22) = f($33,21) + f($32,21) + f($31,21) +… + f($1,21) +1
This formula for f(33,22) uses earlier results for f(smaller or equal amounts, one fund fewer). A bottom-up DP algorithm is straightforward. Generalization of this example gives
f(K,n+1) = f(K,n) + f(K-1,n) + f(K-2,n) + f(K-3,n)+..+ f(2,n) + f(1,n) + f(0,n)
Many zbs learning experiences (me or DQH) were arguably irrelevant if never quizzed.
–iv questions on malloc, tmalloc and the facebook malloc
In latency benchmarks, c++ should simply 1) avoid DMA or 2) pre-allocate. Therefore, if the iv question touches on benchmark, then the malloc question become moot.
–iv questions on lockfree in c++
messier and more low-level than java lockfree. I would say lockfree is seldom written by a c++ app developer. Therefore not a GTD topic at all. Purely QQ halo topic.
–iv question on volatile
Again, many interviewers use volatile as concurrency feature in c++ but it’s non-standard ! Even if it has a side effect under gcc, the practice is “not something I would recommend”, though I won’t criticize hiring team’s current practice.
Not a GTD skill, not zbs either.
–iv question on throwing dtor
c++ standard says UDB but still some interviewers ask about it. So this topic is not zbs not GTD but really QQ
–iv question on debugger breakpoint
Not GTD, not even zbs
–iv question (GS-HK) on CAS no slower than uncontended lock acquisition
I feel I don’t need to know how much faster. My knowledge is good enough for GTD
–iv question on how many clock cycles per instruction
no GTD but zbs for a latency enthusiast
Some interviewers really really go into UDB, so to impress them it is very very useful to understand UnSpecifiedBehavior (USB):
Compliant compilers are not required to document the behavior but the behavior should be consistent and valid such as crash. For a given program containing a USB, where possible the Standard provides two or more possibilities (i.e. permissible outcomes), such that an executable from a compliant compiler should produce one of these outcomes.
Compared to USB, UDB gives compilers more freedom. The actual result could be anything including invalid results in one instance and in another instance turning a blind eye.
No one has quizzed me but am curious how is the cloning done on a large container.
Reality warning — every time I try the SG job market, recruiters would tell me there are so many “new” employers or new markets, but invariably, i need to focus on the old guards .. mostly ibanks, as the new market is not open to me. This is similar to Deepak, Shanyou trying the wall St c++ job market.
I must stop /romanticizing/ about the “improvement” in SG job market.
Basically no change in the landscape since 2015. The jobs available to me are mostly ibanks. Cherish the MLP job but beware attachment. If this job goes sour, I would have to consider WallSt, rather than another perm job in SG.
As explained in 3rd effect@volatile introduced@java5 ,
http://tutorials.jenkov.com/java-concurrency/volatile.html has more details.
https://stackoverflow.com/questions/9169232/java-volatile-and-side-effects also addresses “other writes“.
I think java could deliver similar latency numbers to c/c++, but the essential techniques are probably unnatural to java:
My friend Qihao commented —
There are more management barriers than technical barriers towards low latency java. One common example is with “suppressing gc is unnatural”.
I feel some people are good at (or trained for) juggling big responsibilities that affect other people as “They depend on my system as a service provider”. I’m not so good at it so I feel these responsibilities are very heavy, and a type of cancer-stress.
Someone like Josh, Larry … need to be on the ball and keep track of the large number of (big or small ) changes affecting “his baby”.
In contrast, as a foot soldier I only have obligation (professional responsibility) to keep an eye on my project or my module. Some foot soldiers are eager to learn, but usually her scope of responsibility is much smaller.
Stay on top of “everything” — as a parent I have to be on top of everything about my kids and my house.
LocalSys — If I don’t understand one of many things in and outside my team’s applications, I don’t need to struggle to find out and clarify my understanding.
 it’s not always straightforward to determine if a change is big or small. The app owner need to quickly estimate the impacts of each change. Sometimes the estimate is inaccurate. There’s a bit of risk.
I feel a reusable technique is
With these three shadow arrays, many problems can be solved visually and intuitively, in linear time.
How about the classic max profit problem?
How about the water container problem?
Q: What’s the time complexity of sorting J integer scores which are all [1, N], possibly non-unique?
This is classic counting sort. My book [[algo in a nutshell]] incorrectly says O(J). Incorrect when N is large.
My analysis below uses a total of three dimensions N, J and K, where N and J can be vastly different, and K <=min(N, J), but K could be much smaller.
Textbook counting sort is listed at the bottom. If N is smaller than J, then O(N+J) is dominated by O(J). Now suppose N is huge (like the revenue figure of a company).
Suppose there are K distinct scores among the J scores. I would want a constant-time translation from each distinct score to a distinct counter. I wish there’s a perfect hashing from the K scores to K buckets but I think it’s impossible since the K distinct values are not known in advance. Even with imperfect hashing, I can get O(1) translation from each score value to a distinct counter. I will iterate over the J scores. For each,
O(K logK) : Now pop the min-heap K times, each time to get a distinct score in ascending order. Lookup the score in hash table to get the counter. If counter for score 55 the count is 3, then output 55 three times. This output would be a sorted sequence of the original J scores.
— comparing with alternatives: Mine is O(J + K logK). If K*logK exceeds N, then I would fall back to standard counting sort.
The comparison-based sort would be O(J logJ), inferior to mine if K is much smaller than J.
The textbook counting sort would be O(J + N) , using N independent counters.
https://www.hackerrank.com/challenges/crush/problem Q: Starting with a 1-indexed array of zeros and a list of operations, for each operation add a “bump-up” value to each of the array element between two given indices, inclusive. Once all operations have been performed, return the maximum in your array.
For example, given array 10 of zeros . Your list of 3 operations is as follows:
a b k 1 5 3 4 8 7 6 9 1
Add the values of k between the indices a and b inclusive:
index-> 1 2 3 4 5 6 7 8 9 10 [0,0,0, 0, 0,0,0,0,0, 0] [3,3,3, 3, 3,0,0,0,0, 0] [3,3,3,10,10,7,7,7,0, 0] [3,3,3,10,10,8,8,8,1, 0]
The largest value is 10 after all operations are performed.
This (contrived) problem is similar to the skyline problem.
— Solution 1 O(minOf[N+J, J*logJ ] )
In the example, after sorting, we get 1 4 5 6 8 9.
We all hear (and believe) that the 40-50 period is “supposed” to be the peak period in the life of a professional man. This expectation is created on the mass media (and social media such as Linkedin) brainwash that presents middle-aged managers as the norm. If not a “manager”, then a technical architect or a doctor.
[[Preparing for Adolescence]] illustrates the peer pressure (+self-esteem stress) felt by the adolescent. I feel a Deja-vu. The notion of “normal” and “acceptable” is skewed by the peer pressure.
Q: Out of 100 middle-aged (professional or otherwise) guys, how many actually reach the peak of their career in their 40’s?
A: Probably below 10%.
In my circle of 40-somethings, the norm is plateau or slow decline, not peak. The best we could do is keep up our effort and slow down the decline, be it wellness, burn rate, learning capacity, income,,,
It’s therefore hallucinatory to feel left behind on the slow track.
Q: at what age did I peak in my career?
A: I don’t want to overthink about this question. Perhaps towards the end of my first US era, in my late 30s.
I think middle-aged professional guys should read [[reconciliations]] by Theodore Rubin. The false expectation creates immense burden.
a native FIX protocol is a customized FIX protocol
a binary FIX protocol encodes FIX messages in binary codec, such as SBE.
The well-known Rule 1 — a const data member must be initialized exactly once, no more no less.
The lesser-known Rule 2 — for class-type data member, there’s an implicit default-initialization feature that can kick in without us knowing. This default-init interacts with ctor initializer in a strange manner.
The well-known Rule 3 — the proper way to explicitly initialize a const field is the ctor initializer, not inside ctor body.
The lesser-known Rule 4 — at run-time, once control passes into the ctor body, you can only modify/edit an already-initialized field. Illegal for a const field.
To understand these rules, I created an experiment in https://github.com/tiger40490/repo1/blob/cpp1/cpp/lang_misc/constFieldInit.cpp
— for primitive fields like int, Rule 2 doesn’t apply, so we must follow Rule 1 and Rule 3.
— for a class-type field like “Component”,
In either case, there’s only one initialization per const field (Rule 1)
[[effModernC++]] P 252 explains why in c++ joinable std::thread objects must not get destroyed. Such a destruction would trigger std::terminate(), therefore, programmers must make their std::thread objects non-joinable before destruction.
The key is a basic understanding of “joinable”. Informally, I would say a joinable std::thread has a real thread attached to it, even if that real thread has finished running. https://en.cppreference.com/w/cpp/thread/thread/joinable says “A thread that has finished executing code, but has not yet been joined is still considered an active thread of execution and is therefore joinable.”
An active std::thread object becomes unjoinable
The primary mechanism to transition from joinable to unjoinable is via join().
For a thread to actually become eligible, a Java thread needs start(), but c++ std::thread becomes eligible immediately after initialization i.e. after it is initialized with its target function.
For this reason, [[effModernC++]] dictates that between an int field and a std::thread field in a given class Runner, the std::thread field should be the last initialized in constructor. The int field needs to be already initialized if it is needed in the new thread.
Q1: Can you initialize the std::thread field in the constructor body?
A: yes unless the std::thread field is a declared const field
Now let’s say there’s no const field.
Q2: can the Runner copy ctor initialize the std::thread field in the ctor body, via move()?
A: yes provided the ctor parameter is non-const reference to Runner.
A: no if the parameter is a const reference to Runner. move(theConstRunner) would evaluate to a l-value reference, not a rvr. std::thread ctor and op= only accept rvr, because std::thread is move-only
See https://github.com/tiger40490/repo1/tree/cpp1/cpp/sys_thr for my experiments.
[[effModernC++]] says brace-initialization is usable everywhere, so it simplifies my life
I feel the simplest justification to use ‘noexcept’ is performance. This is also the first pointer in Item 14 of [[effModernC++]]
I ought to admire my peers’  efforts and knowledge (not their STATUS) on :
Even though some of my peers are not the most /accomplished/ , they make a commendable effort. That attitude is admirable.
 Many people crossing my path are … not really my peers, esp. those managers in China. Critical thinking required.
I don’t have a more descriptive title..
— https://www.informatica.com/downloads/1568_high_perf_messaging_wp/Topics-in-High-Performance-Messaging.htm is a 2011 white paper by some experts. I have saved the html in my google drive. Here are some QQ + zbs knowledge pearls. Each sentence in the article can expand to a blogpost .. thin->thick.
Using words and numbers, am trying to “capture” my perceptions (intuitions + observations+ a bit of insights) of the c++/java job market trends, past and future. There’s some reality out there but each person including the expert observer has only a limited view of that reality, based on limited data.
Those numbers look impressive, but actually similar to the words — they are mostly personal perceptions dressed up as objective measurements.
If you don’t use words or numbers then you can’t capture any observation of the “reality”. Your impression of that reality  remains hopelessly vague. I now believe vague is the lowest level of comprehension, usually as bad as a biased comprehension. Using words + numbers we have a chance to improve our perception.
 (without words you can’t even refer to that reality)
My perceptions shape my decisions, and my decisions affect my family’s life chances.
My perceptions shape my selective listening. Gradually, actively, my selective listening would modify my “membrane” of selective listening! All great thinkers, writers update their membrane.
Am not analyzing reality. Instead, am basically analyzing my perception of the reality, but that’s the best I could do. I’m good at analyzing myself as an object.
Refusing to plan ahead because of high uncertainty is lazy, is pessimistic, is doomed.
Warning — latency measurement gotchas … is zbs but not GTD or QQ
— My tech bet — Demand for latency QQ will remain higher in c++ than java
 In the same vein, (unlikely UDP) TCP interferes with data transmission rate control, so even if I control both sender and receive, I still have to cede control to TCP, which is a kernel component.
— jvm performance tuning is mainstream and socially meaningful iFF we focus on
* machine saturation
* typical user-experience response time
— In contrast, a narrow niche area is micro-latency as in HFT
After listening to FPGA, off-heap memory latency … I feel the arms race of latency is limited to high-speed trading only. latency technology has limited economic value compared to mobile, cloud, cryptocurrency, or even data science and machine learning.
Q: modified slightly from Leetcode 974: Given an array of signed integers, print all (contiguous, non-empty) subarrays having a sum divisible by
https://github.com/tiger40490/repo1/blob/py1/py/algo_arr/subarrayDivisibleByK.py is my one-pass, linear time solution. I consider this technique an algoQQ. Without prior knowledge, a O(N) solution is inconceivable.
I received this problem in an Altonomy hackerrank. I think Kyle gave me this problem too.
Sliding window? I didn’t find any use.
Key idea — build data structure to remember cumulative sums, and remember all the positions hitting a given “cumsum level”. My homegrown solution kSub3() shows more insight !
The built-in enumerate() is a nice optional feature. If you don’t want to remember this simple simple syntax, then yes you can just iterate over xrange(len(the_sequence))
https://www.afternerd.com/blog/python-enumerate/#enumerate-list is illustrated with examples.
— to enumerate backward,
enumerate() returns a generator and generators can’t be reversed, you need to convert it to a
for i, v in reversed(list(enumerate(vec)))
This code works best for small indices. It shows only the last digit of each index.
print ' '.join(list(ss[le:ri])) print ' '.join(str(i%10) for i in xrange(le, ri))
Is this applicable in java? I think so, but my focus here is c++.
— 温故而知新 is less effective at my level. thick->thin, reflective.
— 知新而后温故 — x-ref, thin->thick->thin learning.
However, the pace of learning new knowledge pearls could appear very slow and disappointing. 5% new learning + 95% refresh. In such a case, the main benefit and goal is the refresh. Patience and Realistic expectation needed.
In some situations, the most effective learning is 1% new and 99% refresh. If you force yourself to 2% new and 98% refresh, learning would be less effective.
This technique is effective with distinct knowledge PEARLS. Each pearl can be based on a sentence in an article but developed into a blogpost.
Unsafe.getObjectVolatile() and setObjectVolatile() should be the only access to the field.
I think for an integer or bool field (very important use cases), we need to use Unsafe.putIntVolatile() and Unsafe.getIntVolatile()
Q: why not use a volatile field?
A: I guess in some designs, a field need not be volatile at most access points, but at one access point it needs to behave like volatile field. Qihao agrees that we want to control when we want to insert a load/store fence.
Non-volatile behavior usually has lower latency.
https://dzone.com/articles/a-look-at-stampedlock is not an expert article but illustrates a real-world issue.
Reality — we are living longer and healthier.
Observation — compared to old men, old women tend to have more of a social life and more involvement with grandchildren.
I suspect that given a choice, half the white-collar guys in my age group actually wish to keep working past 65 (or 70), perhaps at a lower pace. In other words, They will decide to retire not by choice. My reasoning for the suspicion — Beside financial needs, many in this group do not have enough meaningful, “engaging” things to do. Many would suffer.
I think most of the guys in this category do not prepare well in advance and will find themselves unable to find a suitable job. (We won’t put it this way, but) They will be kinda forced into early retirement. The force could be health or in-demand skillset or …
“Most of them in the 40s are already stable and don’t want to quit. Even though the pay may not be so good, they’re willing to work all the way. It’s an easy-going life.”
The observer was comparing across age groups.
 I think “all the way” means all the way to retirement age, no change of direction, not giving in to boredom, sticking to the chosen career despite occasional challenges.
This observation is becoming increasingly descriptive of me… Semi-retired on the back of my passive income streams.
If a (implicitly final) local variable  is captured inside a nested class, where is the variable saved?
https://stackoverflow.com/questions/43414316/where-is-stored-captured-variable-in-java explains that the anonymous or local class instance has an implicit field to hold the captured variable !
 local variable can be an arg passed into the enclosing function. Could a primitive type or a reference type i.e. heapy thingy
The java compiler secretly adds this hidden field. Without this field, a captured primitive would be lost and a captured heapy would be unreachable when the local variable goes out of scope.
A few hours later, when the nested class instance need to access this data, it would have to rely on the hidden field.
A java lambda expression is used very much like an instance of an anonymous class. However, http://tutorials.jenkov.com/java/lambda-expressions.html#lambda-expressions-vs-anonymous-interface-implementations pointed out one interesting difference:
The anonymous instance in the example has a field named. A lambda expression cannot have such fields. A lambda expression is thus said to be stateless.
Q: given a collection of N positive integers, you perform K operations like “half the biggest element and replace it with its own ceiling”. Find the collection sum afterwards.
Note the collection size is always N. Note K(like 5) could exceed N(like 2), but I feel it would be trivial.
This is a somewhat contrived problem.
I think O(N + K log min(N,K)) is pretty good if feasible.
Key question — Q1: which commit would have multiple parents?
— scenario 1a:
In this simple scenario, your merge is a fast-forward merge. The updated master would now show hash1 at the tip, whose only parent is hashJJ.
A1: No commit would have multiple parents. Simple result. This is the default behavior of git-merge.
However, github or bit-bucket pull-request flow don’t support it exactly.
— scenario 1b:
Instead of simple git-merge, what about pull request? A pull-request uses q[ git merge –no-ff brA ] which (I think) unconditionally creates a merge-commit hashMM on maser.
A1: now hashMM has two parents. In fact, git-log shows hashMM as a “Merge” with two parent commits.
Result is unnecessarily complex. Therefore, in such simple scenarios, it’s better to use git-merge rather than pull request.
— Scenario 2: What if ( master’s tip ) hashJJ is Not parent of hash1?
Now maser and brA have diverged. I think you can’t avoid a merge commit hashMM.
— Scenario 3: continue from Scenario 1b or Scenario2.
3. Then you commit on brA again , creating hash2.
Q: What’s the parent node of hash2?
A: I think git actually shows hash1 as the parent, not hashMM !
Q: is hashMM on brA at all?
A: I don’t think so but some graphical tools might show hashMM as a commit on brA.
I think now master branch shows hashMM having two parents (hash1+hashMM), and brA shows hash1 -> hash2.
I guess that if after the 3-way-merge, you immediately re-create (or reset) brA from master, then hash2’s parent would be hashMM.
LMAX disruptor is probably Martin’s implementation at LMAX. After he left LMAX he founded Aeron based on the same ideas.
log4j2 uses disruptor.
Initially written for java. There’s a dotnet version too. c++ version? No idea.
Looking back at the coding tests we did together, I feel it’s comparable to a form of “marathon training” — I seldom run longer than 5km, but once a while I get a chance to push myself way beyond my limits and run far longer.
Extreme and intensive training builds up the body capacity.
On my own, it’s hard to find motivation to run so long or practice coding drill at home, because it requires a lot of self-discipline.
Nobody has unlimited self-discipline. In fact, those who run so much or takes on long-term coding drills all have something beside self-discipline. Self-discipline and brute force willpower is insufficient to overcome the inertia in every one of these individuals. Instead, the invisible force, the wind beneath their wings is some forms of intrinsic motivation. These individuals find joy in the hard drill.
( I think you are one of these individuals — I see you find joy in lengthy sessions of jogging and gym workout. )
Without enough motivation, we need “organized” practice sessions like real coding interviews or hackathons. This Optiver coding test could probably improve my skill level from 7.0 to 7.3, in one session. Therefore, these sessions are valuable.
typically used in “report” messages for Fill, Bust and Correction.
I like the latency-measurement talk by Azul CTO https://www.youtube.com/watch?v=lJ8ydIuPFeU
In the low-latency seminar https://www.youtube.com/watch?v=BD9cRbxWQx8 , Dan Shaya said our thread should hog the cpu by spinning, with interrupts off (to fend off other threads).
I like this unconventional advice.
(This topic is not GTD not zbs, but relevant to some QQ interviewers.)
https://www.youtube.com/watch?v=BD9cRbxWQx8 is a 2018 presentation.
Edge-to-edge latency from AA to BB, if implemented in a given language:
— c/c++ can only aim for 10 micros … no better than java.
The stronghold of c++, the space between java and fpga, is shrinking … “constantly” according to Dan Shaya. I think “constantly” is like the growth of Everest.. perhaps by 2.5 inches a year
I feel c++ is still much easier, more flexible than FPGA.
I feel java programming style would become more unnatural than c++ programming in order to compete with c++ on latency.
— IPC latency
Shared memory beats TCP hands down. For an echo test involving two processes:
Using an Aeron-based messaging application, 50th percentile is 250 ns. I think NIC and possibly kernel (not java or c++) are responsible for this latency.
10’s of mlllisec between London and NY
In Git, a commit message is immutable but github supports editable comments on a commit:)
Context — a buy-side shop (say HRT) uses a DMA connection sponsored by a sell-side like MS (or Baml or Instinet) to access NYSE. MS provides a DMA platform like Speedway.
The HRT FIX gateway would implement the NYSE FIX spec. Speedway also has a FIX spec for HRT to implement. This spec should include minor customization on the NYSE spec.
I have seen the HPR spec. (HPR is like an engine running in Baml or GS or whatever.) HPR spec seems to talks about customization for NYSE, Nsdq etc …re Gary chat.
Therefore, the HRT FIX gateway to NYSE must implement, in a single codebase,
The FIX session would be provided (“sponsored”) by MS or Baml, or Instinet. I think the HRT FIX gateway would connect to some IP address belonging to the sponsor like MS. Speedway would forward the FIX messages to NYSE, after some risk checks.
For a big Buy order,
IS benchmark could be $79
Vwap benchmark could be $76
A vwap algo may deliver $77, under-performing its benchmark.
An IS algo may deliver $78, beating its benchmark, but worse than the vwap algo performance.
I assume most clients want absolute performance, but they are the ones to specify the benchmark.
In the context of a broker algos (i.e. an execution algo offered by a broker), vwap is
The vwap benchmark is simple, but the vwap algo implementation is non-trivial, often a trade secret.
Avichal is observant and sat next to me for months. Therefore I value his judgment. Avichal is the first to point out I was too distracted.
For now, I won’t go into details on his specific remarks. I will simply use this simple pointer to start a new “thread”…
— I think the biggest distraction at that time was my son.
I once (never mind when) told grandpa that I want to devote 70% of my energy to my job (and 20% to my son), but now whenever I wanted to settle down and deep dive into my work, I feel the need and responsibility to adjust my schedule and cater to my son. and try to entice him to study a little bit more.
My effort on my son is like Driving uphill with the hand-brake on.
As a result, I couldn’t have a sustained focus.
$HOME/.gradle holds all the jars from all previous downloads.
 When you turn on debug, you can see the actual download : gradle build –debug.
 Note IDE java editor can use version 123 of a jar for syntax check, but the command line compilation can use version 124 of the jar. This is very common in all IDEs.
When I make a change to a gradle config,
 For a given dependency jar, “refresh” means download a new version as specified in a modified gradle config.
— in console, run
gradle build #there should be a ./build.gradle file
category? same as JIT blogposts
Q: are java and c# interpreted? QQ topic — academic but quite popular in interviews.
https://stackoverflow.com/questions/8837329/is-c-sharp-partially-interpreted-or-really-compiled shows one explanation among many:
The term “Interpreter” referencing a runtime generally means existing code interprets some non-native code. There are two large paradigms — Parsing: reads the raw source code and takes logical actions; bytecode execution : first compiles the code to a non-native binary representation, which requires much fewer CPU cycles to interpret.
Java originally compiled to bytecode, then went through an interpreter; now, the JVM reads the bytecode and just-in-time compiles it to native code. CIL does the same: The CLR uses just-in-time compilation to native code.
C# compiles to CIL, which JIT compiles to native; by contrast, Perl immediately compiles a script to a bytecode, and then runs this bytecode through an interpreter.
— post-commit hook runs after the commit is completed and closed. So I don’t think it can modify or cancel the commit.
— pre-commit hook can’t distinguish a Amend vs a regular Append.
I have a career plan to work as a developer till my 70’s. When I told you, you pointed out bone health, to my surprise.
You said that some older adults suffer a serious bone injury and become immobile. As a result, other body parts suffer, including weight, heart, lung, and many other organs. I now believe loss of mobility is a serious health risk.
These health risks directly affect my plan to work as a developer till my 70’s.
Lastly, loss of mobility also affects our quality of life. My mom told me about this risk 20 years ago. She has since become less vocal about this risk.
Fragile bones become more common when we grow older. In their 70’s, both my parents suffered fractures and went through surgeries.
See ## strengthen our bones, reduce bone injuries #CSY for suggestions.
Time is a quintessential /limiting factor/ — when I try to break through and reach the next level on some endeavor, I often hit a /ceiling/ not in terms of my capacity but in terms of my available time. This is a common experience shared by many, therefore easy to understand. In contrast, a more subtle experience is the limiting factor of “productive mood” .
 This phrase is vague and intangible, so sometimes I speak of “motivation” — not exactly same and still vague. Sometimes I speak of “absorbency” as a more specific proxy.
“Time” is used as per Martin Thompson.
— WITH VOLUME
Trade in proportion to actual market volume, at a specified trade rate.
The participation rate is fixed.
— Relative Step — with a rate following a step-up algo.
This algo dynamically adjusts aggressiveness(participation rate) based on the
relative performance of the stock versus an ETF. The strategy participates at a target percentage of overall
market volume, adjusting aggressiveness when the stock is
significantly underperforming (buy orders) or outperforming (sell orders) the reference security since today’s open.
An order example: “Buy 90,000 shares 6758.T with a limit price of ¥2500.
Work order with a 10% participation rate, scaling up to 30%
whenever the stock is underperforming the Nikkei 225 ETF (1321.OS)
by 75 basis points or more since the open.”
If we notice the reference ETF has a 2.8% return since open and our 6758.T has a 2.05% return, then the engine would assume 6758.T is significantly underperforming its peers (in its sector). The engine would then step up the participation to 30%, buying more aggressively, perhaps using bigger and faster slices.
What if the ETF has dropped 0.1% and 6758.T has dropped 0.85%? This would be unexpected since our order is a large order boosting the stock. Still, the other investors might be dumping this stock. The engine would still perceive the stock as underperforming its peers, and step up the buying speed.
mastered in the open blog — burn=materialistic_ROTI + self_discipline
Based on my speculation, hypothesis, imagination and a tiny bit of observation.
Majority of the effective, efficient, productive tech professionals don’t work long hours because they already earn enough. Some of them can retire if they want to.
Some percentage of them quit a big company or a high position, sometimes to join a startup. One of the reasons — already earned enough. See my notes on envy^ffree
Most of them value work-life balance. Half of them put this value not on lips but on legs.
Many of them still choose to work hard because they love what they do, or want to achieve more, not because no-choice. See my notes on envy^ffree
a fixtag is not an atomic item. Instead, a fixtag usually comprise two parts namely the value and the identifier.
The identifier is usually a fixtag-num. Note the fixtag-name is not always unique ! It’s more like a fixtag-descriptor.
“Payload” is not a good term. “pair” is kinda uncommon.
Most of us are more familiar with runtime actions than compiler actions and concepts. Now, mv-semantics complexity is 90% inside compiler and 10% inside runtime action of mv-ctor.
r-val complexity is 100% in compiler concept.
mv-semantics is more part-of-the-fabric, more so than the big-4.
Q: given a sequence of payload values produced from a by-level traversal of a binary tree, could the tree be a BST?
Ashish gave me this question. We can assume the values are floats.
(Not contrived, not practical)
— idea 1:
Say we have lined up the values found on level 11. We will split the line-up into sections, each split point being a Level-10 value.
Between any two adjacent values on level 10 (previous level), how many current-level values can legally fit in? I would say up to 2.
In other words, each section can have two nodes or fewer. I think this is a necessary (sufficient?) condition.
–idea 2: One-pass algo
Construct a BST as we consume the sequence.
Aha — there’s only one possible BST we can build.
Relevant in coding tests like speed-coding, take-home, onsite. Practical knowledge is power !
.. by-key is cleaner, not complicated by the iterator invalidation complexities.
You can save all the “bad” keys, and later erase one by one, without the invalidation concerns. You can also print the keys.
if you were to save the “bad” iterators, then once you erase one iterator, are the other iterators affected? No, but I don’t want to remember.
STL iterator invalidation rules, succinctly has a succinct summary, but I don’t prefer to deal with these complexities when I have a cleaner alternative solution.
std::pair has no pointer field so I thought it needs no meaningful mvctor, but actually std::pair mvctor is defaulted i.e. field-wise move i.e. each field is moved.
If a pair holds a vector and a string, then the vector would be move-constructed, and so does the string.
Q1: So what kind of simple class would have no meaningful mvctor?
%%A: I would say a class holding no pointer whatsoever. Note it can embed another class instance as a field.
Q2: so why is std::pair not one of them?
A: std::pair is a template, so the /concrete/ field type can be a dynamic container including std::string.
All dynamic containers use pointers internally.
Needed in some online coding questions (never in GTD). Suppose stdin data is line-delimited like
2 #a count
, then you can do cin>>myCnt>>myKey>>myVal
The newline and any white spaces are ignored 🙂
Q1: “Here is 90% of the logic” — when is such a documentation complete? Answered at the end.
When we programmers read source code and document the “business logic” implemented thereby, we are sometimes tempted to write “My write-up captures the bulk of the business logic. I have omitted minor details, but they are edge cases. At this stage we don’t need to worry about them”. I then hope I have not glossed over important details. I hope the omitted details are just fine prints. I was proven wrong time and again.
Sound byte: source code is all details, nothing but details.
Sound byte: Everything in source code is important detail until proven otherwise. The “proof” takes endless effort, so in reality, Everything in source code is important detail.
The “business logic” we are trying to capture actually consists of not only features and functionalities, but functional fragments i.e. the details.
When we examine source code, a visible chunk of code with explicit function names, variable names, or explicit comments are hard to miss. Those are the “easy parts”, but what about those tiny functional fragments … Perhaps a short condition buried in a complicated if/while conditional? Perhaps a seemingly useless catch block among many catches. Perhaps a break/continue statement that seems serve no purpose? Perhaps some corner case error handling module that look completely redundant and forgettable, esp. compared to other error handlers. Perhaps a missing curly brace after a for-loop header?
( How about the equal sign in “>=” … Well, that’s actually a highly visible fragment, because we programmers have trained vision to spot that “=” buried therein. )
Let me stress again. The visibility, code size … of a functional fragment is not indication of its relative importance. The low-visibility, physically small functional fragments can be equally important as a more visible functional fragment.
To the computer, all of these functional fragments are equally significant. Each could have impact on a production request or real user.
Out of 900 such “functional fragments”, which ones deal with purely theoretical scenarios that would never arise (eg extraneous data) .. we really don’t know without analyzing tons of production data. One minor functional fragment might get activated by a real production data item. So the code gets executed unexpectedly, usually with immediate effect, but sometimes invisibly, because its effect is concealed by subsequent code.
I would say there are no fine-prints in executable source code. Conversely, every part of executable source code is fine print, including the most visible if/else. Every executable code has a real impact, unless we use real production data to prove otherwise.
A1: good enough if you have analyzed enough production data to know that every omitted functional fragment is truly unimportant.
intellij (the community version) is much cleaner than eclipse, and no less rich in features.
On a new job, My choice of java ide is based on
1) other developers in the team, as I need their support
2) online community support — as most questions are usually answered there
I think eclipse beats intellij
3) longevity — I hate to learn a java ide and lose the investment when it loses relevance.
I think eclipse beats intellij, due to open-source
)other factors include “clean”
The most popular tools are often vastly inferior for me. Other examples:
* my g++ install in strawberryPerl is better than all the windows g++ installs esp. msvs
* my git-bash + strawberryPerl is a better IDE than all the fancy GUI tools
* wordpress beats blogger.com hands down
* wordpad is a far simpler rich text editor than msword or browsers or mark-down editors
Value-based type is a new concept in java. Optional.java is the only important example I know, so I will use it as illustration.
One of the main ideas about value types is the lack of object identity (or perhaps their identity is detectable only to the underlying implementation i.e. JVM not Java applications). In such a world, how could we tell whether variables aa and bb “really” are the same or different?
Q: why avoid locking on value-based objects?
%%A: locking is based on identity. See why avoid locking on boxed Integers
Side note — compared to java, c++ has a smaller community and collective brain power so discussions are more limited.
https://stackoverflow.com/questions/34049186/why-not-lock-on-a-value-based-class section on “UPDATE – 2019/05/18” has a great illustration
Auto-boxing of “33” usually produces distinct objects each time, but could also produce the same object repeatedly. Compiler has the leeway to optimize, just as in c++.
Remember: locking is based on object identity.
Many China colleagues (YH, CSY, CSDoctor, Jenny as examples) say their classmates earn much more than them in the U.S. These colleagues seem to claim their Typical Chinese counterpart earns higher than in the U.S.
Reality — those classmates are outliers. The average salary in China is much lower than U.S. Just look at statistics.
Some of these guys (like CSY) feel inferior and regret coming to the U.S.
— cost of reasonable lifestyle
Many Chinese friends complain that cost level is higher in China than U.S. and Singapore. A young MLP colleague (Henry) said a RMB 500k/year feels insufficient to a Chinese 20-something.
In reality, per-sqft property price is indeed higher in some cities than in the U.S. Everything else in China, cost level is much lower than in the U.S. Just look at statistics.
https://fiximate.fixtrading.org/ is the best available.
https://empirica.io/strategies-catalog/pov has a the formula to determine the slice sizes.
I think Symbology is a strict naming convention.
RIC is one symbology. NYSE symbology is well known. RTS also has its own symbology .
For both my son and my own tech learning over the long term, “interest” is not necessarily the best word to capture the key factor.
I was not really interested in math (primary-secondary) or physics (secondary). In college, I tried to feel interested in electronics, analog IC design etc but unsuccessful. At that level, extrinsic motivation was the only “interest” and the real motivation in me. Till today I don’t know if I have found a real passion.
Therefore, the strongest period of my life to look at is not college but before college. Going through my pre-U schools, my killer strength was not so much “interest” but more like keenness — sharp, quick and deep, absorbency…
Fast forward to 2019, I continue to reap rewards due to the keenness — in terms of QQ and zbs tech learning. Today I have stronger absorbency than my peers, even though my memory, quick-n-deep, sharpness .. are no longer outstanding.
Throughout my life, Looking at yoga, karaoke, drawing, sprinting, debating, piano, .. if I’m below-average and clueless, then I don’t think I can maintain “interest”.
Q: if an optional is empty, will it remain forever empty?
— An Optional.java variable could but should never be null, as this instance need a state to hold at least the boolean isPresent.
If a method is declared to return Optional<C>, then the author need to ensure she doesn’t return a null Optional ! This is not guaranteed by the language.
https://dzone.com/articles/considerations-when-returning-java-8s-optional-from-a-method illustrates a simple rule — use a local var retVal throughout then, at the very last moment, return Optional.ofNullable(retVal). This way, retVal can be null but the returned reference is never null.
If needed, an Optional variable should be initialized to Optional.empty() rather than null.
https://dzone.com/articles/considerations-when-returning-java-8s-optional-from-a-method shows the pitfalls of returning Optional from a method.
I feel this is like borrowing from a loan shark to avoid accidental credit card interest charge.
If you are careful, it can help you avoid the NPE of the traditional practice, but if you are undisciplined (like most of us), then this new stragey is even worse —
Traditional return type is something like String, but caller has to deal with nulls. New strategy is supposed to remove the risk/doubt of a null return value, but alas, caller still needs to check null first, before checking against empty!
–immutability is tricky
Similarity to String.java — [B/C]
Compare to shared_ptr instance — [A] is true.
— get() can throw exception if not present
— not serializable
— My motivation for learning Optional is 1) QQ 2) simplify my design in a common yet simple scenario
https://www.mkyong.com/java8/java-8-optional-in-depth/ is a demo, featuring … flatMap() !!
L1 cache is usually private to a core. L2 can be private or shared with other cores.
Q: can we control that size?
In these and other OO-modeling decisions, there are many variations of “common practices” in c++ but in java/c# the best practice usually boils down to one or two choices.
No-choice is a Very Good Thing, as proven in practice. Fewer mistakes…
These dynamic languages rely on a single big hammer and make everything look like a nail….
This is another example of “too many variations” in c++.
https://en.wikipedia.org/wiki/Memory_refresh explains how DRAM data is lost without refresh …
My friend Yifei spent 6+ years in ICG (i.e. the investment banking arm) of Citi Singapore.
I feel Citi is rather profitable and not extremely inefficient, just less efficient than other ibanks.
Overall, I have a warm feeling towards Citi and I wish it would survive and thrive. It offers good work-life balance, much better than GS, ML, LB etc
— Scenario 3: If you app uses just one lock, you can still get deadlock. Eg: ThreadM (master) starts ThreadS (slave). S acquires a lock. M calls the now-deprecated suspend() method on S, then tries to acquire the lock, which is held by the suspended S. S is waiting to be resumed by M — deadlock.
Now the 4 conditions of deadlock:
* wait-for cycle? indeed
* mutually exclusive resource? Just one resource
* incremental acquisition? indeed
* non-Preemptive? Indeed. But if there’s a third thread, it can restart ThreadS.
— Scenario 2: a single thread using a single non-reentrant lock (such as StampedLock.java and many c++ lock objects) can deadlock with itself. A common mis-design.
— Scenario 1:
In multithreaded apps, I feel single-lock deadlock is very common. Here’s another example — “S gets a lock, and wait for ConditionA. Only M can create conditionA, but M is waiting for the lock.”
Here, ConditionA can be many things, including a Boolean flag, a JMS message, a file, a count to reach a threshold, a data condition in DB, a UI event where UI depends partly on ThreadM.
P90 [[eff c#]] also shows a single-lock deadlock. Here’s my recollection — ThrA acquires the lock and goes into a while-loop, periodically checking a flag to be set by ThrB. ThrB wants to set the flag but need the lock.
Q: if I declare a huge int array in static memory, will the object file become huge?
This is possibly a QQ nlg pearl, a halo zbs, not GTD
I often need my debugger to step into library source code.
Easy in java:
c++ is harder. I need to find more details.
Overall, I feel debugger support is significantly better in VM-based languages than c++, even though debugger was invented before these new languages.
I guess the VM or the “interpreter” can serve as an “interceptor” between debugger and target application. The interceptor can receive debugger commands and suspend execution of the target application.
Deepak told me that Rahul, Padma etc stayed in RTS for many years and became “complacent” and uninterested in tech topics outside their work. I think Deepak has sharp observations.
I notice many Indian colleagues (compared to East European, Chinese..) uninterested in zbs or QQ topics. I think many of them learn the minimum to pass tech interviews. CSY has this attitude on coding IV but the zbs attitude on socket knowledge
–> That’s a fundamental reason for my QQ strength on the WallSt body-building arena.
If you regularly benchmark yourself externally, often against younger guys, you are probably more aware of your standing, your aging, the pace of tech churn, … You live your days under more stress, both negative and positive stress.
I think these RTS guys may benchmark internally once a while, if ever. If the internal peers are not very strong, then you would get a false sense of strength.
The RTS team may not have any zbs benchmark, since GTD is (99.9%) the focus for the year-end appraisal.
These are some of the reasons Deepak felt 4Y is the max .. Deepak felt H1 guys are always on our toes and therefore more fit for survival.
One Conclusion — my c++ /mileage/ made me a slightly more confident, and slightly more competent programmer, having “been there; done that”, but see the big Question 1 below.
— Historical view
For half my career I avoided enterprise technologies like java/c++/c#/SQL/storedProc/Corba/sockets… and favored light-weight technologies like web apps and scripting languages. I suspect that many young programmers also feel the same way — no need to struggle with the older, harder technologies.
— many guys are unused to large codebases
Look around your office. Many developers have at most a single (rarely two) project involving a large codebase. Large like 50k to 100k lines of code excluding comments.
I feel the devops/RTB/DBA or BA/PM roles within dev teams don’t require the individual to take on those large codebases. Since it’s no fun, time-consuming and possibly impenetrable, few of them would take it on. In other words, most people who try would give up sooner or later. Searching in a large codebase is perhaps their first challenge. Even figuring out a variable’s actual type can be a challenge in a compiled language.
Compiling can be a challenge esp. with C/c++, given the more complex tool chain, as Stroustrup told me.
Tracing code flow is a common complexity across languages but worse in compiled languages.
* Qz? Not a python codebase at all
* pwm comm perl codebase? I would STILL say codebase would be bigger if using a compiled language
Many young male/female coders are not committed to large scale dev as a long-term career, so they probably don’t like this kinda tough, boring task.
— on a new level
I feel web (or batch) app developers often lack the “hardcore” experience described above. They operate at a higher level, cleaner and simpler. Note Java is cleaner than c++. In fact I feel weaker as java programmer compared to a c++ programmer.
Q1: I have successfully mastered a few sizable codebases in C++, java, c#. So how many more successful experiences do I need to feel competent?
Note sometimes I feel my current ffree is so basic it’s not real ffree at all. At other times I feel it is real, albeit basic, ffree. After achieving my basic ffree, here are 3 deeper motivations for working hard for even more money:
–Below are some motivations not so “deep”
— Below are some secondary $-motivations
Most (simple) java profilers are based on jvm safepoint. At a safepoint, they can use JVM API to query the JVM. Safepoint-based profiling is relatively easy to implement.
s-sync (Martin) is not based on safepoint.
Async profiler is written for openJDK, but some features are usable on Zinc JVM. Async profiler is based on process level counters. Can’t really measure micro-latency with any precision.
Perf is an OS-level profiler, probably based on kernel counters.
“Strategic” i.e. long-term planning/t-budgeting needs a re-definition. quant and c# were two wake-up calls that I tragically missed.
For a long time, the No.1 strategic t-expense was quant, then c#/c++QQ, then codingDrill (the current yellowJersey).
Throughout 2019, I considered workout time as inferior to coding drill .. Over weekends or evenings I often feel nothing-done even though I push myself to do “a bit of” yoga, workout, or math-with-boy, exp tracking,
Now I feel yoga and other fitness t-spend is arguably more strategic than tech muscle building. I say this even though fitness improvement may not last.
Fitness has arguably the biggest impact on brain health and career longevity
Warning — may not work if there’s a recent merge-in on your branch
Find the target commit and its immediate parent commit.
git rebase -i the.parent.commit
First commit in the list would be your target commit. Use ‘r’ for the target commit and don’t change other commits. You will land in vim to edit the original bad commit msg. Once you save and quit vim, the rebase will complete, usually without error.
Now you can reword subsequent commit messages.
As of 2019, c++ low-end jobs are becoming scarce but high-end jobs continue to show robust demand. I think you can see those jobs across many web2.0 companies.
Therefore, it appears that only high-end developers are needed. The way they select a candidates is … QQ. I have just accumulated the minimum critical mass for self-sustained renewal.
In contrast, I continue to hold my position in high-end coreJava QQ interviews.
Is there an oth risk? Comparable to MSFM, my perception of the whole experience shapes my outlook and future decision.
Q: I was brought in as a seasoned c++ old hand. Did I live up to that image? Note I never promised to be an expert
A: I think my language knowledge (zbs, beyond QQ) was sound
A: my tool chain GTD knowledge was as limited as other old hands.
Q: was the mvea c++ codebase too big for me?
A: No, given my projects are always localized. and there were a few old hands to help me out.
I had a few proud deliveries where I had some impetus to capture the momentum (camp out). Listed below. I think colleagues were impressed to some extent even though other people probably achieved more. Well, I don’t need to compare with those and feel belittled.
This analysis revealed that Josh is not easily impressed. Perhaps he has high standard as he never praised anyone openly.
Similar to https://bintanvictor.wordpress.com/2017/09/15/more-mcq-questions-on-core-java-skills/, hopefully these questions can be given over phone.
Other topics easy to quiz over phone: smart pointers, std::thread, rvr/move, big4
Can pass variables from local to remote via double-quote command string
Pipe? Better run simple commands remotely, then apply pipe locally.
Note this post is more about churn (phased-out) and less about accumulation (growing depth)
In each case, there are recurring patterns.
Background — automation scripts have higher values in some areas than others
However, for now let’s focus on short-term local GTD. Automation scripts are controversial in this respect. They take up lots of time but offer some measurable payoff
— strength — Automation scripts is my competitive strength, even though I’m not the strongest.
— respect – Automation scripts often earn respect and start a virtuous cycle
— engagement honeymoon — I often experience rare engagement
— Absorbency — sometimes I get absorbed, though the actual value-add is questionable .. we need to keep in mind the 3 levels of value-add listed above.
Advantage 1: faster allocation, as explained in other blogposts
Advantage 2: programmer can "carelessly" create an "local" Object in any method1, pass (by reference) the object into other methods and happily forget about freeing the memory.
In this extremely common set-up, the reference itself is a stack variable in method1, but the heapy thingy is "owned" by the GC.
In contrast, c/c++ requires some "owner" to free the heap memory, otherwise memory would leak. There’s also the risk of double-free. Therefore, we absolutely need clearly documented ownership.
cat some.txt | egrep –color “40490.$rand|“
Note the trailing pipe gets rid of filtering 🙂
— other solutions:
less +/pattern # not ideal as it scrolls
I favor free, standard tools because they are my only choice in some companies
• VIM, available in git-bash !
• Putty + screen
• Notepad++ and ascii files in ‘recoll’, instead of RTF
• Standard diff tools
Q: given a signed int array a, find the best sub-sequence sum. Any Adjacent elements in a should not both show up. O(N) time and O(1) space.
— O(1) idea 2:
Two simple variables needed: m_1 is the m[cur-1] and m_2 is m[cur-2]. So compare a[cur] + m_2 vs m_1.
https://github.com/tiger40490/repo1/blob/py1/py/algo_arr/nonAdjSeqSum.py is self-tested DP solution.
So overall I feel getting much closer to passing. Now I feel One interview is all it takes to enter a new world
* higher salary than HFT
* possibly more green field
* possibly more time to research, not as pressed as in ibanks
Therefore, given a single (possibly long) string, sorting it should Never be O(N logN) !
Whenever an algo problem requires sorting a bunch of English letters, it’s always, always better to use counting sort. O(N) rather O(N logN)
Similarly, in more than one problems, we are given a bunch of integers bound within a limited range. For example, the array index values could be presented to us as a collection and we may want to sort them. Sorting such integers should always , always be counting sort.
A low-latency c++ engineer used this question to very experience with low-latency dev.
Q: how do you compute myInt % 1024 using bitwise operation only?
Q: how about % 1023 and %1022?
Nowadays I feel in-demand on 1) Wall St, 2) with the web2.0 shops. I also feel welcome by 3) the U.S. startups. In Singapore, this feeling of in-demand was sadly missing. Even the bank hiring managers considered me a bit too old.
Singapore banks only has perm jobs for me, which feel unsuitable, unattractive and stressful.
In every Singapore bank I worked, I felt visibly old and left behind. Guys at my age were all managers… painful.
Q: A lot of times we get technical information in forums, online lecture notes, research papers. why do I trust some more than others?
–adapted from online article (Covalent)
Problem: Multithreaded apps create new objects at the same time. During object creation, memory is locked. On a multi CPU machine (threads run concurrently) there can be contention
Solution: Allow each thread to have a private piece of the EDEN space. Thread Local Allocation Buffer
You can also Analyse TLAB usage -XX:+PrintTLAB
Low-latency c++ apps use a similar technique. http://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919 reveals insight of lock contention in malloc()
A facebook interviewer asked me “I agree that python is great for coding interviews. When do you choose c++?” I said
Now I think there are other reasons
I didn’t showcase c++ but I felt very confident and strong about my c++, which probably shined through in front of the last interviewer (from the architect team)
I think my c++/python combo was good combo for the FB onsite, even though only one interviewer noticed my c++ strength.
— Q: design type-ahead i.e. search suggestion. Scalability is key.
— Q: innerProduct2SparseArray (Table aa, Table bb). First you need to design the undefined “Table” class to represent a sparse array.
Then you need to write real code for innerProduct2SparseArray(), assuming the two Tables are already populated according to your definition.
— Q: add2DecimalsSameLength (string decimal1, string decimal2) to return a string version of the sum. Can’t use the python integer to hold the sum, as in java/c++ integers have limited size. You must use string instead.
Aha — carry can only be 0 or 1
I forget to add the last carry as an additional digit beyond the length of the input strings 😦
— Q: add2longBinary(string binaryNum1, string binaryNum2). I told interviewer that my add2BigDecimal solution should work, so we skipped this problem.
— Q: checkRisingFalling(listOfInts) to return 1 for rising, -1 for falling and 0 for neither. Rising means every new num is no lower than previous
— Q: checkDiameter(rootOfBinTree)
For a static STL container, the iterator objects can be safely stored and reused.
The dreaded iterator invalidation is a risk only under structural changes.
Many coding interview questions allow (even require) me to save those iterators and store them in a vector, a hash table …
Afterwards, we retrieve an iterator value, and visit the next/previous of it.
server-push update ^ TTL ^ conditional-GET # write-through is not cache expiration
Few Online articles list these solutions explicitly. Some of these are simple concepts but fundamental to DB tuning and app tuning. https://docs.oracle.com/cd/E15357_01/coh.360/e15723/cache_rtwtwbra.htm#COHDG198 compares write-through ^ write-behind ^ refresh-ahead. I think refresh-ahead is similar to TTL.
B) cache-invalidation — some “events” would trigger an invalidation. Without invalidation, a cache item would live forever with a infinity TTL, like the list of China provinces.
After cache proxies get the invalidation message in a small payload (bandwidth-friendly), the proxies discard the outdated item, and can decide when to request an update. The request may be skipped completely if the item is no longer needed.
B2) cache-update by server push — IFF bandwidth is available, server can send not only a tiny invalidation message, but also the new cache content.
IFF combined with TTL, or with reliability added, then multicast can be used to deliver cache updates, as explained in my other blogposts.
T) TTL — more common. Each “cache item” embeds a time-to-live data field a.k.a expiry timestamp. Http cookie is the prime example.
In Coherence, it’s possible for the cache proxy to pre-emptively request an update on an expired item. This would reduce latency but requires a multi-threaded cache proxy.
G) conditional-GET in HTTP is a proven industrial strength solution described in my 2005 book [[computer networking]]. The cache proxy always sends a GET to the database but with a If-modified-since header. This reduces unnecessary database load and network load.
W) write-behind (asynchronous) or write-through — in some contexts, the cache proxy is not only handling Reads but also Writes. So the Read requests will read or add to cache, and Write requests will update both cache proxy and the master data store. Drawback — In distributed topology, updates from other sources are not visible to “me” the cache proxy, so I still rely one of the other 3 means.
|if frequent query, in-frequent updates||efficient||efficient||frequent but tiny requests between DB and cache proxy|
|if latency important||OK||lowest latency||slower lazy fetch, though efficient|
|if in-frequent query||good||waste DB/proxy/NW resources as “push” is unnecessary||efficient on DB/proxy/NW|
|if frequent update||unsuitable||high load on DB/proxy/NW||efficient conflation|
|if frequent update+query||unsuitable||can be wasteful||perhaps most efficient|
When a production system fails to work, do you contact
Clearly we trust XX more. She knows the localSys and likely has seen something similar.
Exception — what if YY has a power tool like a remote debugger? I think YY may gain fresh insight that XX is lacking.
XX may be poor at explaining the system design. YY may be a great presenter without low-level and hands-on know-how.
If you discuss the language and underlying technologies with XX he may show very limited knowledge… Remember Andrew Yap and Viswa of RTS?
Q: Who would earn the respect of teammates, mgr and external teams?
XX may have a hard time getting a job elsewhere .. I have met many people like XX.
http://15418.courses.cs.cmu.edu/spring2013/article/46 explains ABA in the specific context of a … lockfree stack. To counter ABA problem, It uses CAS2 i.e. CAS on a group of two objects — the original object + a stamp.
Java CAS2? I believe AtomicStampedReference is designed specifically for it.
— http://tutorials.jenkov.com/java-util-concurrent/atomicstampedreference.html#atomicstampedreference-and-the-a-b-a-problem explains the AtomicStampedReference solving the ABA problem but in the last section it doesn’t clearly explain the benefit of get().
Also, the retry need a loop.
====Actually, many ABA illustrations are simplistic. Consider this typically simplistic illustration of a CAS stack:
— With a vector-of-pointer implementation, Thread 0 needs to save integer position within the stack. At Step 5, it should then detect that the A sitting at stack top is now at a higher position than observed earlier, detecting a symptom of ABA.
The rest is simple. Thread 0 should then query the new item (D) below A. Lastly, the CAS would compare current stack top position with the saved position, before committing.
However, in reality I think a vector-of-ptr is non-trivial to implement if we have two shared mutable things to update via CAS: 1) the stackTopPosition integer variable and 2) the (null) ptr in the next slot of the vector.
However, if Thread 1 replaces B with D, keeping the stack size unchanged, then the ABA would be undetectable by position check alone.
— with a linked list implementation, I think we only have node addresses, so at Step 5, Thread 0 can’t tell that D has been inserted between A and B.
You may consider using stack size as a second check, but it would be similar complexity as CAS2 but less reliable.
For HFT + mkt data + .. this is a favorite interview topic, kind of similar to retransmission.
Perhaps look at … https://web.archive.org/web/20141222151051/https://dl.dropboxusercontent.com/u/3001534/engine.c has a brief design doc, referenced by https://quant.stackexchange.com/questions/3783/what-is-an-efficient-data-structure-to-model-order-book
I feel like impostor more on the job than in interviews. I have strong QQ (+ some zbs) knowledge during interviews. I feel it more and more often in my c++ in addition to java interviews.
Impostor’s syndrome is all about benchmarking. In job interviews, I sometimes come up stronger than the interviewers, esp. with QQ topics, so I sometimes feel the interviewer is the impostor !
In my technical discussions with colleagues, I also feel like an expert. So I probably make them feel like impostors.
So far, all of the above “expert exchanges” are devoid of any locaySys. When the context is a localSys, I have basically zero advantage. I often feel the impostor’s syndrome because I probably oversold during interview and set up sky-high expectations.
I think Rahul may have a different view, but I feel some of QQ questions qualify as zbs, but very few of the algo tricks do.
In contrast, data structure techniques are more likely to qualify as zbs.
Obviously, the algorithm/data-structure research innovations qualify as zbs, but they are usually too complex for a 45-min coding interview.
All the lockfree data structures I have seen have 2 writer threads or more.
Q: what if in your design, there can be at most one writer thread but some reader threads. Any need for lockfree?
A: I don’t think so. I think both scenarios below are very common.
Q: why choose multi-threaded design instead of single-threaded processes?
Most publications mention multi-core hardware as an answer. Questionable.
With multi-threading, you can run 30 threads in one process. Or you can run 30 single-threaded processes as in RTS parser and Rebus — industrial strength proven solution. Both designs make use of all CPU cores.
Between these two designs, heap memory efficiency can be different, as the 30 threads are able to share 99GB of objects in the same address space, but the 30 processes would need shared memory.
I feel the lesser known middle-ground design is … 30 threads running in single-threaded-mode, By definition, these 30 threads can only share immutable data only,. Anything mutable is thread-local.
In any multi-threaded design, those 30 threads can share the text segment i.e. memory occupied by code. Text segment tend to be smaller than the heap footprint.
In a boss-worker design, the worker threads may need to share very few mutable object, so these worker threads are not strictly STM. They are quasi-STM
Suppose you write a dumpTree() which takes a “callback” parameter, so you can invoke callback(aTreeNodeOfTheTree).
Sounds like a common requirement?
Additionally, Suppose you want “callback” to have a default value of no-op, basically a nullptr.
Sounds like a common requirement?
I think c/c++ doesn’t have easy support for this. In my tested code https://github.com/tiger40490/repo1/blob/cpp1/cpp/algo_binTree/binTreeUtil.h, I have to define a wrapper function without the callback parameter. Wrapper would call the real function, passing in a nullptr as callback. Note nullptr needs casting.
Nowadays I use the word “clean” language more and more.
Classic example — java is cleaner than c# and c++. I told a TowerResearch interviewer that java is easier to reason with. Fewer surprises and inconsistencies.
The hardware is not “consistent” as wished. Those languages closer to the hardware (and programmers of those languages) must deal with the “dirty” realities. To achieve consistency, many modern languages make heavy use of heap and smart pointers.
For consistency, Everything is treated as a heapy thingy, including functions, methods, metadata about types …
For consistency, member functions are sometimes treated as data members.
https://leetcode.com/problems/symmetric-tree/solution/ iterative solution is elegant adaptation of BFT.
Now I think the real low-latency systems always prefer Single-Threaded-Mode. But is it feasible?
I feel CSDoctor has reasonable job satisfaction. If I were in his role my job satisfaction would be lower.
His financial domain is rather niche. Equity volatility data analysis, not a big infrastructure that needs a team to support. I guess it could be considered a glorified spreadsheet. I think the users are front office traders looking at his data to make strategic business decisions.
I see very high career risk in such a role. What if I get kicked out? (Over 7 years the team did kick out at least 2 guys.. Last-in-first-out.) What if I don’t get promoted beyond VP and want to move out?
I won’t enjoy such a niche system. It limits career mobility. 路越走越窄.
This is one of the major job dis-satisfactions in my view. Other dis-satisfactions:
* long hours — I think he is not forced to work long hours. He decides to put in extra hours, perhaps to get promotion.
* stress — CSDoctor is motivated by the stress. I would not feel such high motivation, if put in his shoes.
* commute — I believe he has 1H+ commute. I feel he has very few choices in terms of jobs closer to home, because his domain is extremely niche.
Stroustrup singled out these 2 constructs in C and said they represent nice, simplified models of memory usage.
Note a struct field can be a pointer.
Now I see that in C, there’s no other basic data structure. What about C++? Same! Java, c# seem to follow suit.
Stroustrup said every resource (usually heapy thingies) need to have an owner, who will eventually return the resource.
By his definition, every resource has an acquire/release protocol. These resources include locks, DB connections and file handles. The owner is the Object responsible for the release.
git push --force-with-lease
This option is a safer version of push-force.
It will not overwrite any work on the remote branch if more commits were added to the remote branch (usually by another team-member). It ensures you do not overwrite someone else’s work by force pushing.
I don’t know how often we encounter this stringent requirement —
Soccer world cup final, or a big news about Amazon … millions of users posts comments on a web page and all comments need to be persisted and shown on some screen.
Rahul and I discussed some simple design. At the center is a single central data store.
Since the comments come in a burst, this data store becomes the bottleneck. Rahul said there are two unrelated responsibilities on the data store updaters. (A cluster of updaters might be possible.)
Each read-server has a huge cache holding all the comments. The server receives the broadcast and updates its cache, and uses this cache to service client requests.
Reusable coding techniques include my classic generators, DP, augmented trees, array-element swap, bigO tricks like hash tables and medium-of-3 pivot partitioning.
More useful than “reusable techniques” are pattern-recognition insight into the structure and constraints in a given coding problem. Without these insights, the problem is /impenetrable/intractable/.
Often we need a worst input to highlight the the hidden constraint. The worst input can sometimes quickly eliminate many wrong pathways through the forest and therefore help us see the right pathway.
However, in some context, a bigO trick could wipe out the competition, so much so that pattern recognition, however smart, doesn’t matter.
In this blogpost I focus on broad, high-level viability factors for my dev-till-70 career plan.
I have grown from a sysAdmin to a dev
I have grown from web and scripting pro into a java pro then c++ pro !
Next, I hope to grow my competence with large codebase.
I feel large codebase is the #1 diffentiator separating the wheat from the chaff — “casual” vs hardcore coders.
With a large codebase, I tend to focus on the parts I don’t understand, regardless that’s 20% or 80% of the code I need to read. I can learn to live with that ambiguity. I guess Rahul was good at that.
In a few cases, within 3M I was able to “take up” a sizable brown field codebase and become somewhat productive. As I told Kyle, I don’t need bottom-up in-depth knowledge to be competent.
In terms of count of successes taking up sizable brown-field codebase, I am a seasoned contractor, so I would score more points than a typical VP or an average old timer in a given system.
GregR convinced me that most of the young developers aren’t interested in Wall St contract jobs. I can’t remember the reasons but something like
In my experience the contractors I see are mostly above 40 or at least 35+. The younger guys (Nikhil..) tend to be part of a contract agency like Infosys.
“Older hires are more stable. They would only move within the industry and within the same region.” I guess a counter offer has a better chance.
Bright young hires are more likely to leave for greener pastures, but easier to train and more willing to take on BAU tasks,