[18]realistic+significant Legacy as a developer..possible@@

See also %% Original contributions as a Wall St techie

Background — I often feel there’s no (social) value in my career.

Q: How about system architect or app-owner in a big firm?

I feel a successful and stable app can survive 5 to 20 years before it gets replaced.

Q: How about documentation for an (open source or otherwise) long-living technology? Documentation can even take the form of StackOverFlow QnA.

This “legacy” may not last very long as better answers could supersede your answer any time. In fact, as the topical technology evolves, your answer is doomed to become outdated gradually. Even a very popular computer book becomes outdated over 20 years.

Also, this doesn’t feel realistic for me.

Export Credit Agency — some basics

Each national government has such an “exin bank”, funded by the ministry of finance. (There are also some multinational exin banks like Asian Dev Bank, World Bank…) Their mandate is to support their own exporters in terms of *default risk*. The ECA guarantees to the supplier that even if the overseas client (importer) defaults, the ECA would cover the supplier. It’s technically a loan to the importer, to be paid back. For those non-commercial risks affecting large deals (up to several billion dollars), the ECA’s have a natural advantage over commercial banks – they are financed by the government and can deal with the political and other risks across the borders.

Political risk is quite high, but the guarantee fee charged by ECA is very low. This paradox disappears if you understand that those big deals support domestic job creation, and tax/revenue generation of the large national exporters, so even if the fee charged by the ECA is arguably insufficient to cover the credit risk they take on, the decision still make sense. I think these ECA’s are using tax-payer’s money to help the home-grown exporters.

However, the ECA won’t blindly give big money to unknown foreign importers. Due diligence required.

The ECA’s are usually profitable on the back of those fees they charge (something like 1% above Libor). I guess the default intensity is statistically lower than feared, perhaps thanks to the risk analysis by the various parties. Risk assessment is the key “due diligence” and also the basis of the pricing. The #1 risk event being assessed is importer default. The exporter (supplier) are invariably blue chip corporations with a track record, and know what they are doing. 80% of the defaults (either by importer, exporter or the lending bank) are due to political risk, rather than commercial risk.

Many entities take part in the risk assessment, bringing with them special expertise and insight. The commercial bank has big teams dealing with ECA; the exporter needs to assess the buyer’s credit; the ECA has huge credit review teams… There are also specialist advisory firms who do not lend money. If any one of them identifies a high risk they can’t quantify and contain, I would say it’s only logical and prudent to hesitate.

The exporter first approach a (or a group of) commercial bank(s). The bank would seek *guarantee* from the national ECA. The guarantee covers 90% to 100% of the bank loan, so the bank has a very small credit exposure. (The ECA themselves have very high credit rating.) In the event of a default, the bank or exporter would be compensated by the ECA.

They mostly cover capital goods export, such as airplanes/trains/ships, power plants, infrastructure equipment, with long term repayment … So the supplier are mostly blue chip manufacturers. These loans are tricky because

· Long term, so event risk is much higher

· The entity to assess is a foreign entity, often in a developing country

· Big amount, so potential financial loss is sometimes too big for a single commercial lender

China, Japan and Korea are some of the biggest exporter nations.

probability density #intuitively

Prob density function is best introduced in 1-dimension. In a 2-dimensional (or higher) context like throwing a dart on a 2D surface, we have “superstructures” like marginal probability and conditional probability … but they are hard to understand fully without an intuitive feel for the density. Density is the foundation of everything.

Here’s my best explanation of pdf:  to be useful, a bivariate density function has to be integrated via a double-integral, and produce a probability *mass*. In a small region where the density is assumed approximately constant, the product of the density and delta-x times delta-y (the 2 “dimensions”) would give a small amount of probability mass. (I will skip the illustrations…)

Note there are 3 factors in this product. If delta-x is zero, i.e. the random variable is held constant at a value like 3.3, then the product becomes zero i.e. zero probability mass.

My 2nd explanation of pdf — always a differential. In the 1D context, it’s dM/dx. dM represents a small amount of probability mass. In the 2D context, density is d(dM/dx)/dy. As the tiny rectangle “dx by dy” shrinks, the mass over it would vanish, but not the differential.

In the context of marginal and conditional probability, which requires “fixing” X = 7.02, it’s always useful to think of a small region around 7.02. Otherwise, the paradox with the zero-width is that the integral would evaluate to 0. This is an uncomfortable situation for many students.

beta ^ rho i.e. correlation coeff #clarified

Update: I don’t have a intuitive feel for the definition of rho. In contrast, beta is intuitive, as the slope of the OLS fit

Defining formulas are similar for  beta and rho:

rho   = cov(A,B)/  (sigma_A . sigma_B)
beta = cov(A,B)/  (sigma_B . sigma_B) ,  when regressing A on B
= cov(A,B)/  variance_B

Suppose a high tech stock TT has high beta like 2.1 but low correlation with SPX (representing market return). If we regress TT monthly returns vs the SPX monthly returns, we see a cloud — poor fit i.e. low correlation coefficient. However, the slope of the fitted line through the cloud is steep i.e. high beta !

Another stock ( perhaps a boring utility stock ) has low beta i.e. almost horizontal (gentle slope) but well-fitted line, as it moves with SPX synchronously i.e. high correlation !

http://stats.stackexchange.com/questions/32464/how-does-the-correlation-coefficient-differ-from-regression-slope explains beta vs correlation. Both rho and beta measure the strength of relationship.

Rho is bounded between -1 and +1 so from the value you can get a feel. But rho doesn’t indicate how much (magnitude) the dependent variable moves in response to an one-unit change in the independent variable.

Beta of 2 means a one-unit change in the SPX would “cause” 2 units of change in the stock. However, rho value could be high (close to 1) or low (close to 0).

tiny team of elite developers

Upshot — value-creation per head and salary level would rival the high-flying manager roles.

Imagine a highly successful trading shop. Even though the trading profit (tens of millions) is comparable to a bank trading desk with hundreds of IT head count, this trading shop’s core dev team *probably* have a few (up to a few dozen) core developers + some support teams [1] such as QA team, data team, operations team. In contrast, the big bank probably have hundreds of “core” developers.

[1] Sometimes the core teams decide to take on such “peripheral” tasks as market data processing/storage, back testing, network/server set-up if these are deemed central to their value-add.

In the extreme case, I guess a trading shop with tens of millions of profit can make do with a handful of developers. They want just a few top geeks. The resultant efficiency is staggering. I can only imagine what personal qualities they want:

* code reading — my weakness
* tools – https://bintanvictor.wordpress.com/2012/11/08/2-kinds-of-essential-developer-tools-on-wall-st-elsewhere/ * manuals — reading tons of tech info (official or community) very quickly, when a “new” system invariably behave strangely
* local system knowledge
* trouble-shooting — and systematic problem-solving. I feel this largely depends on system knowledge.
* design — it right, and able to adjust it as requirements change * architecture?
* tuning?
* algorithms?

(soft skills:)
* clearly communicate design trade-offs in a difficult discussion * drive to get things done under pressure without cutting corners * teamwork — teamwork to back down when needed to implement a team decision

tech mgr^functional mgr

Most of my immediate managers are “technical managers”, aka
“specialist managers”, or “tech leads” or “development leads” — leading by doing.

The other major category of managers are “functional managers” or “generalist managers”. Examples — PM, senior managers. Lim Yanguang pointed out that the PM’s he has seen don’t have/need any specialist or product knowledge. Job duty is cost control including time-lines, resource management.

When I think about (or compare with) the “managers”, I need to distinguish these 2 types.

• Key feature — Tech managers can, if needed, write a module by himself. That means the guy must remain hands-on “forever”.
• Key feature — No one has a better grasp of the technical side of things.
• Key feature — has to read a lot of code.
• Quant team manager are always specialist managers.
• I feel Google’s managers are tech managers.
• German’s career path is a tech mgr.
• Avichal’s career path is a tech mgr.

How about product managers? Could be a specialist manager, though Some minor ones don’t really lead any team.

q[less] line-wrap head-chop bug

I only hit this problem in 2017. My q(less) has line-wrap by default. When I scroll , half the time, a wrapped line would show only last portion.  Somehow the earlier portions are hidden.

I would have to scroll up then come back here.

Another option: turn on line numbers to help me detect the problem.

Still looking for the cause and fix.

factorial is worst O(n!) > O(2^n)

Factorial complexity is (much) worse than exponential complexity. Look at n! and 53^n, without loss of generality. Model them as two snowballs rolling over 1,2,3,..1000… Initially, the exponential (E) snowball is bigger, but after we pass n=53, the factorial(F) snowball grows by a factor of 54, while the snowball E grows by a factor of 53. From then on, F grows faster and faster and will outgrow E sooner or later.

[[Discrete Mathematics]] puts Factorial as the worst algorithm, but both Factorial and Exponential are unacceptable, while Polynomial is acceptable.

mis-perception: wrong peer group

see salary probabilities(distro): mgr^NBA#market depth

A poor perception (Maslow) and the wrong peer comparison group directly hurts my drive, my effort, my self-esteem…and leads to self-hate.

Even at the best of times, it is already hard to keep a consistent focus and keep working on any long-term goal. A invalid comparison is extremely destructive.

pyramid of wage levels, CN^sg^US

See also the similar post on NBA salary
See also realities facing China developers.

Look at my in-law. (I’m not too sure about his situation so I will use tentative statements.) He’s smart, dedicated. Rather long experience as a team lead. He has a MBA from a reputable uni in Shanghai.

However, there are many people with similarly strong track record in China, so he can’t move up the pyramid to, say CNY 1000k. In 2020 he said CNY 400k is achievable, but CNY 600k offer is tough and mostly available from high-end prestigious employers.

In Singapore I’m facing a similar challenge. A S$150k (after tax) tech job is rare and considered elite so you need to be rather strong to get it. In other words, the pyramid has a sharper tip than in the US pyramid, based on a fictitious sample of 5,000 IT jobs.

(I think the Shanghai salary distro is better than most China cities…)

The NBA post brings together other important factors — lifelong income; managerial skill; …

things to read RIGHT-to-left#c++/linq …

python list comprehension/generator expression

Linq is left-to-right, an improvement over SQL query syntax

C++ const ptr/ref type declarations. How about array-of-pointers?

int * arr[9]; # array of 9 pointers to int

The way I would remember how to compose it — “9 pointers to int”, so the asterisk sits between int and [9]

c++delete() using addr@reference variable

Conclusion — you can’t declare a pointer to a reference variable myRef, but you can take the address of myRef!

Q1: if you have nothing but a reference to a heap object, how can you delete it?
AA: yes. q(delete & theReference) works

Q1b: how about valgrind q(g++ -g d.cpp; valgrind ./a.out)?
AA: no leak.

See https://stackoverflow.com/questions/3233987/deleting-a-reference. q(new vector<int>) works but is code smell. Code smells tend to be useful tricks.

https://bintanvictor.wordpress.com/?p=10514&preview=true has details on vector memory allocation.

 // --- d.cpp ---
#include <vector>
#include <iostream>
using namespace std;

typedef vector<int> intArray;

intArray & createArray() {
    intArray *arr = new intArray(3, 0);
    return(*arr);
}

int main(int argc, char *argv[]) {
    intArray& array = createArray();
    for (int i=0; i< array.size(); ++i){
      cout<<array[i]<<' ';
    }
    intArray * ptr2ref = & array;
    delete ptr2ref;
}

c++return by ref #briefly

Background: This is an old topic I studied multiple times, but still unfamiliar. I think most people don’t work at such a low level?

Q1: Can return a malloc object by reference? I tested. You can still get the address by taking the address of the reference.

Q1b: if you only have a reference to a heap object, how can you delete it? See https://bintanvictor.wordpress.com/?p=10505&preview=true

Q: Can return a static local by reference? I think so but not popular

Q: Can return a by-ref argument by reference? No need (but legal) since the caller already has that reference. You could return it in an if/else

Q: can return another object in scope by reference? Yes

Return-by-non-const-ref is often used in operator overloading to support counter[3].increment()

The assignment operator returns a const ref.

 

illegal to assign rvalues into non-const Lval-ref

This is a tough one for the average cpp guy

int getInt(){static int last=3; return last+1);}

int& myint = getInt(); // illegal, since getInt() returns a nonref.

(What if getInt returns a reference??? . I think it’s legal.)

However, adding a const makes it legal:

int const & var1 = 33;

In fact, this is same thing as the (extremely) common idiom of using const ref as function param. With that, you can pass in either an lvalue (like a literal) or rvalue.

I found the above exlanation on
http://eli.thegreenplace.net/2011/12/15/understanding-lvalues-and-rvalues-in-c-and-c . The author (Eli) points out:

• * subtle difference between const lval-ref vs non-const lval-ref (above)
• * lvalue means “locator value”
• * a concise example of mv-semantics
• * An everyday example of a function returning an lvalue is operator[] in

myMap[10] =5.6

how2guage TPS capacity@mkt-data engine

Pump in an artificial feed. Increase the input TPS rate until

1. CPU utilization hits 100%
2. messages get dropped

The input TPS is the the highest acceptable rate i.e. the “capacity” of this one process.

Note each feed has its own business logic complexity level, so the same software may have 600k TPS capacity for a simple Feed A but only 100k TPS for a complex Feed B.

Also in my experience the input interface is the bottle neck compared to the output interface. If System X feeds into System Y, then we want to have System X pumping at 50% of Y’s capacity. In fact, we actually monitor the live TPS rate. The difference between that and the capacity is the “headway”.

python dict bad lookup key: solutions

A common scenario. myDict[‘baaadKey’] throws exception. https://stackoverflow.com/questions/3483520/use-cases-for-the-setdefault-dict-method compared the solutions.

My use case — if any key is not found, return a const default value.

— 1) Solution: defaultdict class. To my surprise, my use case is not easily supported. Need to define a factory method.

— 2) Solution: mydict.setdefault(myKey, myDefault) of the regular dict class. Note this solution is similar to the get() solution below, and it does NOT set a single default for the entire dict.

https://www.programiz.com/python-programming/methods/dictionary/setdefault explains setdefault() with clear examples, but I can’t remember the multiple rules so I won’t use setdefault in coding tests.

— 3) simplest Solution: mydict.get(myKey, myDefault).

• C# supports the same.
• Java lookup method won’t throw. See
  http://stackoverflow.com/questions/3626752/key-existence-check-in-hashmap
• STL map operator[]() can silently insert a pair. The map::at() method throws out_of_range exception, similar to the simple dict lookup.

 

vi: locate a long sentence that may span2lines

Suppose you know where the line break, if any, would be.

/abcd\_s*efgh — Finds abcd followed by any whitespace or newlines then efgh.

python array sum

To my surprise, google search shows many solutions but not the built-in sum() function

numbers = [1,2,3,4,5,1,4,5]

Sum = sum(numbers, 10) #10 is added to the sum