jvm stack size; pthreads usage

— some findings from https://pangin.pro/posts/stack-overflow-handling

Most of the interesting implementation techniques are C/C++ techniques, not usable as java coding techniques. Low level techniques, mostly dealing with memory.

Most of the content is low market value because too obscure for IV, not required for GTD at all, and not even relevant as zbs or expertise.

  • — some worthwhile knowledge pearls:
  • stack size for a single thread .. configurable via -Xss command line switch, and defaults to 1MB for a 64-bit system (8 bytes/reference on a frame).
  • .. “This is usually enough to place several thousand average frames.”
  • jvm codebase includes pthreads function calls. “HotSpot explicitly disables glibc guard for all Java threads by calling pthread_attr_setguardsize …” I think author was talking about the non-Windows JVM.

Note some parts of this arcitle discusses non-hotspot JVM’s. These JVM implementations could be very obscure and low-value, therefore quite a distraction.

G1^ZGC

Warning: high churn

ZGC is relevant to low-latency trading. I think G1 is general-purpose and probably not designed for low-latency trading, but it could be more than adequate for your specific latency requirements. Remember Martin Thompson’s comment on “good enough”.

ZGC was introduced in java11 as an experimental GC.

G1 is the default GC since java9, replacing the Parallel GC (not the ConcurrentMarkAndSweep).

 

[20] hiding^overriding ] java inheritance

Upcast from a subclass instance to a superclass variable ..

  • can call the overshadowed (hidden) static methods of the superclass
  • can’t call the replaced nonstatic methods of the superclass (for example nonstaticMeth()).
  • .. I learned this first from my Verizon colleague from India. If you have a subclass instance, then the JVM probably removes all access to the replaced nonstaticMeth() of superclass

There’s also an obscure mechanism involving private methods

— based on stackoverflow:

All methods that are accessible are inherited by subclasses. –> I believe Private methods are not inherited.

From the Sun Java Tutorials:

A subclass inherits all of the public and protected members of its parent, no matter what package the subclass is in. If the subclass is in the same package as its parent, it also inherits the package-private members of the parent. You can use the inherited members as is, replace them, hide them, or supplement them with new members

The only difference with inherited static (class) methods and inherited non-static (instance) methods is that when you write a new static method with the same signature, the old static method is just hidden, not overridden.

From the page on the difference between overriding and hiding.

The distinction between hiding and overriding has important implications. The version of the overridden method that gets invoked is the one in the subclass. The version of the hidden method that gets invoked depends on whether it is invoked from the superclass or the subclass

subclass add`VIRTUAL@inherited nonVirt func

See also

Suppose superclass has a public nonVirtual say() method , and subclass redefines it as virtual. (See my typeid_WithoutVptr.cpp in github.)

Q: Is this a practical scenario? I feel very rarely.
Q: Is this common in interviews? Less rare.

The vptr/polymorphic behavior is not widely understood.

Q: Is the behavior even standardized? Not sure and I won’t bother to find out.

Below is my empirical understanding based on experiment. Suppose we have B* b = new D() and then call b->say().

  • if B has nonVirtual say() and D has virtual say() as in the opening example, then b.say() hits base class B, bypassing vptr.
  • .. I guess the “B” subobject inside the D object has no vtpr. The B class could be written long ago, so the compiler of its ctor would NOT allocate a vptr hidden field.
  • if B has virtual say(), then D (and all descendant classes) has an implicit virtual keyword. Therefore, b->say() hits subclass D, via vptr in the “b” subobject.
  • … if both B and D declare say() virtual? same as above, because the virtual in D is implicit and redundant.

 

keep2solutions]1py: print funcName+retVal

Here, stack() returns a list of stack frames. First frame is the caller of stack(), i.e. myFunc. The 4th item in this “frame record” is the function name, but this level of detail is never appreciated in interviews.

The technique, though, is useful in many coding drills, and even coding tests.

def myFunc():
  ...
  print inspect.stack()[0][3], "returning", ret
  return ret

— alternatively
Note the second stack frame is used in this case, because first stack frame is ‘fn’.

def myFunc():
  ...
  print fn(), ret
  return ret
def fn():
  return inspect.stack()[1][3] + " returning"

##major non-convertible currencies #Korea

A non-convertible currency is a currency used primarily for domestic transactions and not openly traded on FX markets.

“Blocked currencies” is another word.

I think all Non-deliverable-forward currencies are probably blocked currencies?

I feel these governments tend to be weaker or more prudent to protect their own currencies.

  • CNY and TWD (Taiwan)
  • INR – India
  • BRL – Brazil
  • Korea and Taiwan — competitive export-driven economies
  • MYR
  • PHP
  • IDR – Indonesia

java.util.Object methods #@NotNull

This class consist of static util methods for operating on any java object. Mostly for zbs and GTD. Might be useful in rare interviews, but useful in coding IV.

For each method, the purpose is quite practical, but the practical usability depends on your project and requires field testing.

— deepEquals(nullable1, nullable2) — arguments can be arrays.
— equals(nullable1, nullable2)
— hash(obj1, obj2, obj3…) — see javadoc for usage
— hashCode(nullableObj) is useful when the argument is nullable
— requireNonNull(obj, “custom exception msg”)
https://stackoverflow.com/questions/45632920/why-should-one-use-objects-requirenonnull explains why this fail-fast method can be used. I see it as alternative to assert, as executable documentation.

Typical usage of requireNonNull is validation of some arg1 passed to myMethod. If you use requireNonNull the stack trace would show this method as the site of exception. If you don’t use requireNonNull, then the stack trace may not even show myMethod and you may have a hard time locating where the null comes from.

When in a hurry, we can first document the nonNull requirement using @NotNull. Later we can add the requireNonnNull() validation. See https://stackoverflow.com/questions/34094039/using-notnull-annotation-in-method-argument

 

noSql@cloud #lesson 0.5

I have spent about 5 minutes on this topics so far.

Background — If you app runs in the elastic cloud, then local disk files are quite “dangerous”. I like short sentences like that, even though it’s vague. It means “not best practice” and “dangerous design”. Among the alternatives, noSQL is one of the most popular and best-supported.

https://searchcloudcomputing.techtarget.com/tip/Compare-NoSQL-database-types-in-the-cloud hints that NoSQL and elastic cloud are natural partners.

Q: which of the four types (warning: churn) of the current /crops/ of noSQL solutions are good for cloud?

— churn: Both cloud and noSql are evolving. Granted, there are stable features in both but the best practices are likely to change over 20Y as new stuff replace the old.

One case in point is the meaning of noSql. NoSQL stands for Not Only SQL, but some online articles may treat it as NotSql.

 

getRichQuick: meaningful vocation@@

There are many GRQ (getRichQuick) advertisements such as

  • personal e-commerce project
  • property investment
  • stock picking
  • MLM

All of them emphasize low effort. However, I want some meaning in my vocation. I also want some level of intellectual challenge to fend off boredom.

Many of these “quick” ideas are unsustainable, unable to provide long-term “engagement” for 90% of the participants even if we remove the unsuccessful people from our sample.

Most of them offer no or limited organic growth and personal development.

— How do they stack up against my traditional ideas of semi-retirement occupations like

  • traditional: teach math, coding,,,
  • traditional: dev-till-70
  • traditional: technical writer?

I guess some of these GRQ ideas might become more meaningful, more engaging to me.

refresh@blog beats friend tech discussion

There’s accumulation of insights in this blog for 温故知新.

In my recent experience (as of 2020) this is arguably more fruitful than email exchanges with friends, and more fruitful than reading books.

Email exchange is far more efficient than calling friends, unless there’s something confusing.

Reading books is generally more rewarding than reading online.

  • online — URL and text can be copied into blog
  • reading — easier to refresh

restricted FIX session ^ socket shutdown()

— Based on a discussion with my colleague.

In a case of FIX session issue (any type) we can run the session in a restricted mode which allows us to send cancels (but not new/replace) and receive reports.

If incoming sequence numbers are beyond repair, then we will not see the reports but we still can send cancels.  Not an ideal situation but lets you out of the market.

In many venues we have an agreement of cancel on disconnect, so we would simply disconnect.

— socket shutdown() has a comparable feature
The syscall shutdown() can receive a “direction” argument of 0, 1 or 2.

The 0 and 1 constants specify either incoming or outgoing direction. “2” shuts down both.

 

##eg@ MainSt IT salary: SGD 5k

Multiple blogposts should refer to this blogpost

— 2020 HaiFeng
Years ago HaiFeng’s senior at NCS was around 35 (based on HF’s calculation), earning $5k. Now in 2020 this guy is around my age, earning 6k in some main street (healthcare?) sector.

Therefore, HaiFeng felt that within the broader IT job market, the specific sub-sector is crucial.

— 2020 Raymond’s job search
— 2015 my own job hunting experience: Software engineer salary doesn’t rise faster than inflation.
— hardware engineers —

  • many mid-level engineers earn up to SGD 5k in their 30’s, such as Integrated Circuit designers.
  • Canon — My 2012 OC intern from a polytechnic said his dad, engineering team lead in a Japanese electronics giant (Canon?), only earned $5k.

search-tree algos

This blogpost is based on a “Path finding in AI” chapter in [[Algo in a nutshell]] . The illustration problem is the 8-puzzle. The chapter’s highlight is the famed A*search, but I will focus on the brute-force DFS/BFS and other basic stuff.

Each tree node is a “board-state”, or “snapshot”. Each directed edge between two nodes is a “move”.

(Not every developer candidate has the “tree” concept for a board problem. Recall the BBG odometer problem?)

The tree is recombinant, since two paths can reach the same snapshot. Therefore, the search tree is a directed graph. In practice, this directed graph is more like a tree, so it’s useful to treat it as a tree.

Q: shall we use edge set to represent this directed graph?
A: I doubt it , because the graph is being constructed as we go. The graph is not given to us as static input.

Equivalence between two snapshots is based on a snapshot.key() method. Conceptually, two equivalent snapshots be mirror images or rotated images of each other.

snapshot.storedData() returns arbitrary stored data for a given tree node.

— BFS on this type of search tree

  • would evaluate a tremendous (costly) number of snapshots
  • 🙂 no recursion
  • 🙂 queue is constant-time
  • 🙂 guarantees to find shortest path

— DFS on this type of search trees

  • maintains a relative small stack of “open” nodes and a large set of “closed” nodes.  Closed means fully explored dead-ends.
  • .. I think open means not fully explored, including the white and grey nodes defined on P144.
  • .. what if we come back to a node existing on the current path? It would represent a useless move, not one of the validMoves()
  • DFS (not BFS) features back-tracking
  • 😦 DFS doesn’t know when the current node is steps away to the endgame

— relevance to CIV?
I guess some advanced interviews (Google?) may pose an unusual problem that has elegant solutions based on the essentials of path-finding. The problem might be solvable on-paper in 45-minutes iFF you have those basic concepts, either from reading, or raw intelligence, or “intelligent conversation with the interviewer“. I would say prior reading is a huge advantage.

In reality, most candidates won’t have the knowledge or the raw intelligence, so the conversation is really the only way out. Interviewer knows that you the candidate may have zero prior experience, so she attempts to drop hints to “guide” you, as a test of your communication/problem-solving capacity. I remember Rahul said some candidates (like Deepak) simply couldn’t get the hint. Sadly, I don’t always get the hint. Recall the first Facebook onsite.

 

c++macro-function: concat as string arg

The context: In C++ (as in most languages), we often need to pass into a function an argument like

“some text” + someNumber + someBoolean

In this context, java makes it easy.  Python requires str(someNumber) or the comma as connector. In C++, strict typing is …. quite restrictive. The standard solution uses a stringstream object, but clumsy compared to java. The more flexible concat is

cout<<“some text” << someNumber ….

However, this concat is unusable in our context, which requires a string argument.  The trick, found on P12 [[safe C++]], is a macro function SCPP_ASSERT(). You can invoke it like

SCPP_ASSERT(someBooleanCondition,  “some text”<<someNumber );

 

 

semi-smart ptr: 2 parts@SMART

[[safeC++]] has a 2-page chapter on semi-smart ptr. The title of this chapter is de-referencing null ptr. So this is actually one of the two parts of SMART.

So traditional smart pointers have two essential Smarts

  1. ownership i.e. delete
  2. deref control
  3. … other Smarts …. such as copy control

The semi-smart ptr in this chapter features only one of these Smarts.

This semi-smart ptr class is smart about … deref.

— deref — involves two operators i.e. 1) the arrow and 2) the asterisk. Therefore, the semi-smart ptr in this chapter overloads these two operators, to detect null pointers
— footprint — Is the size of this semi-smart ptr instance same as a raw ptr? I guess so. Can write a test program.
— simplicity — Compared to the traditional smart pointers, this semi-smart ptr is very simple.

  • easy to reason with.
  • Easy to incorporate into my own projects.

FIX server can cope with 2 client IPs sharing 1 compID

In a legit scenarios, ip1 and ip2 are the failover machines running identical software, so both would use the same compId7.

However, due to mis-configuration, two unrelated testers on ip1 and ip2 could be using compId7 unknowingly. The sequence numbers would be inconsistent between the two testers.

FIX server would assume these two FIX sessions are operated in tandem. If sequence numbers of ip1 and ip2 are not out of sync, then the 2 could coexist.

 

fill+bust+correction+amend: 4 exec-reports

In a typical buy-side FIX gateway, people are only interested in about 10 important FIX reports from a venue to the order originator (let’s call it the Portfolio Manager). Beside these 10, there are probably many other reports, but it’s usually harmless to filter them out, thrown down the drain.

Among the 10 report types, fill, bust, correction and amend are in the same category with these common traits:

  • unsolicited — these 4 reports are generated due to venue events, not as a immediate reply to the PM. Some call these 3 reports “unsolicited”. Actually, there’s an irrelevant 5th unsolicited report — the unsolicited cancel of an unfilled order (or a portion thereof).
  • position impact — these 4 reports have immediate impact on the PM’s position and therefore need to go into the PM’s books immediately.
  • execution reports — these 4 reports are true execution reports. I feel the other reports tend to update order status.

I believe a correction report represents a corrected version of an erroneous fill report. I feel this is as rare as the bust.

Note in addition to an exchange, a liquidity venue or a broker/dealer can issue such corrections and busts. Any sell-side can. (I worked on a bust processor at a big equity broker.) Of course, the sell-side need to clean up their own record before sending these amendments.

Note both the bust and the correction target a “trade” meaning a fill, not an order i.e. request-to-trade.

I guess overfill can be a reason for a bust, but I don’t see any online discussion of it, so I guess overfill is often taken as is.

##UChicago t$cost overspent: similar items

Q: are these t-investments below similar to my UChicago t-investment overspend? This is another retrospective, after a few others.

— socket programming?
I feel sockets are similar to quant skillset in churn-resistance.

sockets are slightly closer to my current skillset than quant is. Therefore socket skillset is more likely to be relevant.

— latency (micro-)engineering?
Latency skillset is a QQ halo on WallSt, similar to quant.

Unfortunately, latency is similar to quant as an unlikely domain for my future jobs.

My t-spend on this front is guided by interview experience, not a formal college education.

— bigO analysis, DP+greedy, speed coding (ECT etc)
My strength is stronger in this theoretical domain like bigO and quant.

Could I overspend time here? Am I overrating this skill like quant skills?

My t-spend on this front is guided by interview experience, not a formal college education.

— FIX protocol?

social chat: pantry^cubicle: appearance++

I tend to prefer pantry chat but there are disadvantages

— risk: you can miss an urgent issue, a meeting reminder…. During office hours we are supposed to be available.

This is esp. import when you are on support rather than dev project

— disadvantage: unprofessional appearance.

We all have prejudice about “those Indians” or “Those Chinese workers”, or “Those young developers”

Don’t reinforce their impression of you !