j4 analyzing my perception@reality: c++/bigO..

Using words and numbers, am trying to “capture” my perceptions (intuitions + observations+ a bit of insights) of the c++/java job market trends, past and future. There’s some reality out there but each person including the expert observer has only a limited view of that reality, based on limited data.

Those numbers look impressive, but actually similar to the words — they are mostly personal perceptions dressed up as objective measurements.

If you don’t use words or numbers then you can’t capture any observation of the “reality”. Your impression of that reality [1] remains hopelessly vague. I now believe vague is the lowest level of comprehension, usually as bad as a biased comprehension. Using words + numbers we have a chance to improve our perception. Clarify is a basic improvement.

[1] (without words you can’t even refer to that reality)

My perceptions shape my decisions, and my decisions affect my family’s life chances.

My perceptions shape my selective listening. Gradually, actively, my perception would modify my “membrane” of selective listening! All great thinkers, writers update their membrane.

Am not analyzing reality. Instead, am basically analyzing my perception of the reality, but that’s the best I could do. I’m good at analyzing myself as an object.

Refusing to plan ahead because of high uncertainty is lazy, is pessimistic, is doomed. This applies to retirement planning, what-if scenario planning.

observer@middle-aged SG workers#all-the-way

“Most of them in the 40s are already stable and don’t want to quit. Even though the pay may not be so good, they’re willing to work all the way[1]. It’s an easy-going life.”

The observer was comparing SG (white or blue collar) employees across age groups, and this is the brief observation of the 40-something.

This observation is becoming increasingly descriptive of me… Semi-retired on the back of my passive income streams. Easy life.

[1] I interpret “all the way” as all the way to retirement age, no change of direction, not giving in to boredom, sticking to the chosen career despite occasional challenges (pains, disappointments, setbacks).

mobility thanks_to IV !! localSys

I was able to move between countries and between industries, primarily based on interview skill including coding IV. We can’t rely on GTD, localSys.

Same can be said about Venkat, Henry Wu etc.

body-build`has biggest long term impact

[19] body-build`can hit low visPgress#leverage

[17]#1impactful endeavor(Now)4family

competitiveness: Pre-teen sprinter + HFT selectivity

I was a decent sprinter at age 6 through 13, then I realized I was not good enough to compete at Beijing municipal level.

There are quite a number of HFT shops in Singapore. I think they have a higher bar than ibanks. At ibank interviews I felt again like that pre-teen sprinter, but HFT interview is generally too hard for me

##G3 uncertainties10Y out: #Demand^otherCandd

Let’s be specific — on the competitive landscape over next 10Y, what specific factors are we trying to predict

• G3: predicting my health, energy, motivation, absorbency, “retention rate” (MSFM…), coding drill visible-progress ..
  • I have 3x more insight on this factor than other factors
• G5: predicting strength of competing candd (i.e. candidates), often younger
  • one note — my accumulated theoretical and low-level QQ strength will continue to be rare among the young competitors
• G9: predicting market demand evolution, including churn. Coding IV is the prime example.
  • Note this evolution is not so fast. Read my analysis of MOM and RDBMS

Since 2017, I have tentatively started to shift more focus towards west coast. As a result, the Competition and Demand factors felt too fast-changing, which makes my painstaking analysis look naive and futile.

Critical thinking needed — Suppose I keep my focus on ibanks. In such a case, my analysis and prediction is actually relevant, even valuable. The Competition/Demand factors have not changed drastically.

I’m pretty good at this kind of competitive landscape analysis , but It’s truly tough to predict any of these 3 factors. However, as Warren Buffett (?) observed, those who decide to give up and submit to mindless investing is likely (not guaranteed) to lose out to those who try and analyze/predict.

 

Every useful codebase requires never-ending maintenance #mod_rewrite

Imagine you maintain any mature, stable software product with non-trivial complexity, used by 100+ internal users or 1000+ external users. I would venture to predict that maintenance effort for such a product would never end.

• New feature requirements – The bigger and more diverse your user base, the less likely you would stay stagnant. Stagnant would lead to losing relevance gradually. Consider dinosaurs. If you don’t evolve , then other products would catch up, absorb your best features and make you /irrelevant/.
• New maintainers — have a need to understand and learn your source code, otherwise in 20 years no one will be available to answer user questions or maintain the codebase. New maintainers shall see room for improvements and modernization (like adopting new technologies). After a few changes of guard, they could vote for a rewrite.
• Imperfections — We wish all the bugs get ironed out X years after active development, but sadly, bugs never die off completely.
• New users — come on-board, ask (rarely new) questions that require assistance, which is part of maintenance effort
• Documentation — require periodic update and modernization. Otherwise, new readers would find the documentation old-fashioned.
• testing — testing never ends. There will be better ways to test your product. If lucky, then would uncover hitherto unknown bugs.

Therefore, there’s budget allocated to the system as long as there are important users. Budget creates jobs and employment. Some of these jobs are age-friendly.

I used to think apache mod_rewrite was written once and needs no update. I doubt it now.

[18] G3 Survival capabilities #health;burn rate

Out of the subconscious realm, I hand-picked a couple of personal “qualities” [2] as the (three or five) pillars of my life [1], the most important pillars since 2007. 2007 is the /watershed year/.

These are the pillars for my past, present and my future, immigration plan, family housing, retirement plan, … There’s possibly nothing new here, but this blogpost provides another perspective into the “making” of my career and the foundation of my entire life. As such, this is not a worthless reflection. Still, avoid over-thinking as there are other worthwhile reflections.

I think everyone can consider the same question — “Using words that are as specific as you can find, name 2 or more personal survival capabilities, hopefully the most important ones.”

• A) IV prowess — (since Chartered). Compare CSY’s experience.
  • self-renewal to stay relevant — lifelong learning. Am rather successful so far
  • my reflective blogging is actually part of the my career way-finding, motivation…
  • absorbency — of dry, theoretical, low-level domains. /continuous/ plow-back without exhaustion
  • theoretical complexity — aptitude and absorbency
  • lower-level — overall, i’m stronger than my peers at low-level
• B) my capacity to keep a well-paying dev job (not as “unique” as the other 2 capabilities). Even in the three Singapore cases, I was able to hold it for about 2 years, or receive a transfer or severance.
  • figure-things-out speed?  Not really my strength but my competence
  • camp-out, extra hours
  • attention to details
  • getting the big picture?
• [G3] personal burn-rate — important to my feeling of long term security.
  • See the Davis chat on old-timers.
• — The following factors are less “unique” but I want to give credit to
• [G3] wellness and bread-earning longevity — in the long run this factor would prove increasingly important. It enables me to keep working long past retirement age.
• [G5] my bold investment style [2]
• my education and self-learning capabilities including English skills
• [G3 = a top-3/Group3 factor]

! benchmarking — A is a single-winner competition, whereas B is about staying above minimum standard in the team.

! strengthening — I continue to plow back and build my AA/BB 内力, focusing on localSys + coding drill. A huge amount of energy, absorbency, continuous effort and dedication needed (cf. XR,YH, CSY…), though since 2018 I have noticed the ROTI is not as high as before 2012.

[1] to some extent, my family’s life also rests on these same pillars, but I tend to under-estimate the capabilities of my family members and over-estimate my earning power.
[2] in this post I don’t want to include my financial assets since I want to focus on personal qualities

##legal risks that can tarnish a career

• reputation risks to renters/landlord
• legal risks to accountants — sis
• legal risks to doctors like malpractices
• legal risks to car drivers
• legal risks to security traders
• legal risks to sales people
• legal risks about intellectual property such as stealing code

algo-IV ^ expertise

Background — became expert via external QQ benchmark` !!localSys xx

Q2: how are algo quizzes different from the C++ language expertise tested in job inerviews?
%%A2: I feel this algo expertise is loosely defined — mostly defined by a few hundred “fashionable and common algo interview questions.”

Ironically, the algorithm researchers and inventors may not ace these west-coast algo interviews 😉 Also, the non-trivial algorithms in textbooks are invariably too complicated for a 45-min coding interview.

There’s no deep insight or innovation in this game. This type of “expertise” doesn’t solve tough real-world problems. They can only solve simple real-world problems.

Top performers in this game are possibly students, exam-taking machines. They won’t feel proud of themselves as compared to c++ language expert or algorithm inventor.

Q2b: how are speed coding different?
%%A: in addition to A2, speed coding requires a mastery of a tiny subset of the language features and mastery of many coding idioms based thereon.

##define good life: job,health,life-chances..

I will not list generic and vague items like cash asset or health.

•  job
  • #1 factor — sense of security that I can always get a job. Ashish Singh is the first one to point it out
  • engaging, challenging job, learning something meaningful, not like java/SQL again.
  • some complexity
  • [$$] reasonable commute
• [$$] not so much time spent on home maintenance
• enough time to spend with kids
• enough time for exercise
• [$] yoga classes + other classes
• [$] healthy and convenient diet
• grand parents frequent visits? Very hard
• kids not falling behind in school
• passive income? LG2 but the more the better
• appreciating property?
• parks nearby, not necessarily upscale waterfront
• [$$] clean street
• [$$] car-freedom
• freedom to blog everywhere .. /therapeutic/
• ..
• life chances
• [$ = money can “buy”, to some extent]

What if I hadn’t worked this hard # kids

Over my 20Y career, I showed _some_ talent professionally.

In contrast, I showed significantly more talent in school. My massive study effort increased my abilities [1] to the extent that people don’t notice the difference between my talent vs abilities. Even my IQ score improved due to my intellectual curiosity and absorbency. If these efforts are considered a talent, then yes I have the talent of diligence.

[1] eg — Chinese composition, English grammar/vocabulary, many knowledge-intensive subjects

Q1: what if I had put in just an average amount of effort in school and at work? Some people (mostly guys I really don’t know well) seem to put in sporadic efforts that average out to be “just average” but still got into similar professional level like I did, or higher.
A: For my academic excellence, persistent effort was necessary.

A: I guess sporadic effort could be enough to reach my level of professional “success” for very bright and lucky people. I doubt any professional programmer got higher than mine without consistent effort for many years.

Professional success also depends on opportunity, on positioning and timing, on inter-personal skills and relationship skills, on judgment 判断力. None of these depends directly on consistent effort.

An example of positioning — my break into Wall St java market vs my peers who didn’t.

To analyze this question, I need to single out an under-appreciated talent — absorbency capacity i.e. 耐得住寂寞, where I score in the 97th percentile.

Q2: if my son only puts in sporadic and average efforts, could he end up losing the competitions?

peers’priority^%%priority: G5 #stayVague

In a nut shell, Some of my peers’ priorities I have decidedly given up, and some of my priorities are fairly unique to me.

Actually I only spoke to a small number of (like 10) peers, mostly Chinese and Indian. There are big differences among their views. However, here’s a grossly oversimplified sample of “their” priorities.

• theirs — top schools, top grades
• theirs — move up (long-term career growth) in a good firm
  • build long-term relationships with big bosses
• theirs — early retirement
• theirs — bonus
• theirs — green card
• ———-
• mine — diversify (instead of deepen or stack-up) and avoid stagnation
• mine — stay technical and marketable, till 70
• mine — multiple properties and passive income
• mine — shorter commute
• mine — smaller home price tag

 

job4accu@20Y: which past job was closest2ideal

Update — the domain for accu is less important than a good boss, reasonable expectation … see my spreadsheet on jobSatisfactionPredictor.

Q: do I want to deepen and specialize in one single area, rather than diversify? skill: deepen^diversify^stack up provides a context
A: let’s assume answer is yes deepen.

Q: which area?
A: programmatic data analysis in finance … with two entry barriers in math and coding, with reasonable “moneyness” but market depth is poor.
A: financial enterprise app with multi-threading …?
A: high speed market data engine in TCP/multicast?
A: orderbook replication with retransmission?
A: bond math including IRS?
A: option math? Poor market depth

So which past job is most likely to offer me that professional accu prospect?

• 95G — java threading for trade execution
• Barcap, but market depth is poor for that skill
• citi?
• RTS? but I get NO exposure to the raw sockets and I’m not allowed to ask!

a profession with Enjoyment+income+barrier

Update — Enjoyment is often short-lived and affected by too many factors like

• respect by mgr #bonus
• “strategic” learning #Barclays
• commute
• income cf to peers

Instead of Enjoyment, I often prefer “engagement”.

However, for both enjoyment and engagement, the signal-to-noise ratio is below 0.2 i.e. those “noise factors” above are at least 5 times stronger.

——–

Background — discussion with an older fellow developer about helping our high-school kids select a major.

Example — Look at Aunt Gennifer. She changed her profession around age 40 to find a better paying and less boring job.

Example — grandpa has an ever-green interest in his research field, but most researchers in this field are not well-paid (it’s the least commercial field of research). In terms of income, I think grandpa’s medical and pension benefits are in the the top 1%.

Example — me. I entered trading-engine-dev circa 2007.

• It pays well above the average professional salary across all professions. (I was told U.S. family doctors earn about 150k only.)
• Market depth is excellent — look at my post on NBA salary.
• The constant pressure to upgrade and learn is perfectly acceptable to me. In fact, I seek opportunities to maintain my learning pace
• I don’t find it tiring or boring. For a few years before 2007, I told myself “I don’t want to remain a developer for life” but had a U-turn.

For most people, it’s hard to find a profession that’s not boring, not too tiring, and pays well, a field you could keep plowing till retirement, if you ever retire.

In fact, such a field is likely to be /oversubscribed/ — too many hopeful new entrants but few would get in and fewer would survive 😦

Example — Lianzhong’s daughter Borong loves writing, but she realized it wouldn’t be an easy way to make a living. Similarly, a large number of individuals have a meaningful and rewarding “hobby” but can’t make a decent income from it

• tweaking with computers and gadgets
• visual arts # including photography
• literary arts
• performing arts #including music
• sports, games #including board and electronic
• cooking

[17]#1 impactful endeavor4family: IV^localSys^gym..

See also [20]what protects family livelihood: IV skill^GTD #Ashish

This blogpost is about meaningful endeavor Now: algo^zbs^…

Background — In the quiet hours, inevitably I would get my share of self-doubt about the value of my endeavors. See also what efforts go towards 20Y-career building

At other times, I would feel the impacts (of my effort today) on the people who depend on me — grandparents, kids, wife. There’s a lot I can do to make their lives safer, richer, easier, … For example, the interview-preparation effort looks short-term and less meaningful than zbs, but actually has more impact on family well-being such as education, health-care, housing and life-chances. Re zbs, now I realize zbs accumu is rather theoretical and actually limited. Interviews build my confidence and capacity to provide for them.

Looking at my peers … I feel their personal endeavors are NOT much better than mine:

• move up to leadership positions. I think that’s a good direction if you CAN move up. I gave up long ago. So I see myself almost like a specialist consultant for hire
• personal (property) investments
• kids grades and top schools

		accumu #not a benefit	#3 mental fitness, anti-aging	#2 career after 65 (RnD/teach)	#1 family well-being: impact 1-5		#4 lasting social value?
		good if low churn	good	minimal	4		minimal	IV: QQ/BP #incl. algo ideas
		good	excellent	none	4		none	IV: algo practice
		good	excellent	N.A.	5		none	…cf: yoga, jog
		good if low churn	good	possible	3 #helps PKI !! IV		minimal	zbs #+GTD, instrumentation
		churn	ask HuKun	too volatile	0		none	data science, machine learning
		some	some esp. quant	none	1 {– 2		none	portable dnlg(+quant)
		none	some	none	4 #if stay`long. Can help move-up but low correlation		none	xx local sys
		some	minimal	none	1 #questionable		can help kids; can’t teach	per investment analysis #unlike XR
		NA	minimal	none	2 {–1		none	… cf: family expense management
		some	some	possible	0		some	En/Ch learning
		churn	good	minimal	0 # coding practice		churn! Get Real no bandwidth!	contribute to OSS??

“spread%%nlg” as a column? trivial.

2H life-changing xp#Pimco#income,home location,industry…

Here’s a real story in 2010 — I was completely hopeless and stuck in despair after my Goldman Sachs internal transfer was blocked in the last stage. I considered moving my whole family back to Singapore without any offer, and start my job search there. I was seriously considering a S$100k job in a back office batch programming job. Absolutely the lowest point in my entire career. After licking the would for 2 months, tentatively I started looking for jobs outside Goldman and slowly found my foothold. Then in early 2010, I passed a phone screening and attended a Citigroup “superday”. I spent half an hour each with 3 interviewers. By end of the day, recruiter said I was the #1 pick. I took the offer, at a 80% increment. In the next 12 months, I built up my track record + knowledge in

1. real time trading engine components, esp. real time pricing engine
2. fixed income math,
3. c++ (knowledge rebuild)

I have never looked back since. Fair to say that my family won’t be where we are today, without this Citigroup experience. With this track record I was able to take on relatively high-end programming jobs in U.S. and Singapore. I was able to live in a convenient location, and buy properties and send my kids to mid-range preschools (too pricey in hind sight). Obviously I wanted this kind of job even in 2009. That dream became reality when I passed the superday interview. That interview was one of the turning points in my career.

Fast forward to Apr 2017 — I had a 20-minute phone interview with the world’s biggest asset management firm (Let’s call it PP), then I had a 2-hour skype interview. They made an offer. I discussed with my recruiter their proposal —

• I would relocate to California
• I would get paid around 200k pretax and possibly with an increment in 6 months. PP usually increase billing rate after 12 months if contractor does well.
• recruitment agency CEO said he would transfer my visa and sponsor green card.

If I were to take this offer, my life would be transformed. (I would also have a better chance to break into the  high tech industry in nearby silicon valley, because I would have local friends in that domain.) Such a big change in my life is now possible because … I did well [1] in the interview.

Stripped to the core, that’s the reality in our world of contract programmers.  Project delivery, debugging, and relationship with boss can get you promoted, but those on-the-job efforts have much lower impact than your performance during an interview. Like an NBA playoff match. A short few hour under the spot light can change your life forever.

This is not a rare experience. There are higher-paying contract job offers that could “change your life”, and you only need to do well in the interviews to make it happen.

I feel this is typical of U.S. market and perhaps London. In Singapore. contract roles can’t pay this much. A permanent role has a few subtle implications so I feel it’s a different game.

[1] The 7 interviewers felt I was strong in c++ (not really), java and sql, and competent in fixed income math (I only worked with it for a year). Unlike other high-end interviews, there are not many tough tech questions like threading, algorithms, or coding tests. I feel they liked my interview mostly because of the combination of c++/java/fixed income math — not a common combination.

programmer^mathematician^basketballer #market depth

Programmer is one of few skill with a good “market depth”:

* a mediocre talent can find jobs to pay a decent blue-color knowledge worker wage
* a top talent can always find challenging and well-paying roles. There’s over-demand for the next 100 years. * Any talent level in between can easily find jobs with a suitable salary. * You can even teach this skill at various schools

I guess healthcare as a skill is similar.

Basketballers (any professional athletes) – very few get paid a good salary.

Mathematicians – I only know the teaching and quant jobs.

[18]realistic+significant Legacy as a developer..possible@@

See also %% Original contributions as a Wall St techie

Background — I often feel there’s no (social) value in my career.

Q: How about system architect or app-owner in a big firm?

I feel a successful and stable app can survive 5 to 20 years before it gets replaced.

Q: How about documentation for an (open source or otherwise) long-living technology? Documentation can even take the form of StackOverFlow QnA.

This “legacy” may not last very long as better answers could supersede your answer any time. In fact, as the topical technology evolves, your answer is doomed to become outdated gradually. Even a very popular computer book becomes outdated over 20 years.

Also, this doesn’t feel realistic for me.

top geeks – 5 categories

Let’s try to be critical, incisive… It’s also good but not critical to widen the perspective.
–5) instrumentation for GTD. This is the #1 most valuable skill for GTD, not necessarily to move up to leadership.
–7) Some (not “top top”) geeks can quickly pick up enough details of a new “tool” and use it to get things done (often not perfectly). I had this advantage in my youth. I might have it still.
eg: git, IDE, tibrv,
eg: Venkat, Avichal
–8) Some (not top top) geeks have rare knowledge of advanced/obscure topics like concurrency, memory mgmt, templates, elegant SQL joins, arcane language rules…, but the difference here is — the knowledge is needed only for interviews.
eg: IV questions at many HFT
eg: most threading questions
eg: Venkat
eg: [[effC++]]

–9) Some successful geeks have good local system knowledge. See post on Helicopter view.
Prime eg: Yang.

–1) Some top geeks (+ open source heroes) can quickly create a complete non-trivial software, often single-handedly.
* probably the hardest achievement. These are the doers not just thinkers and dreamers.
* productivity is often 100X a regular developer
–2) [G] Some top geeks are experts on google-style algorithm challenges.
* This domain takes at least a few months (possibly years) of practice. Without this mileage, you won’t be there.
* similar to mathematical talent — somewhat innate
–3) [G] Some top geeks can optimize performance in a demanding system. A specialist knowledge.
eg: the Russian Sybase guru in PWM CT
eg: facebook memory allocator?
I feel this skill is overrated. Not much business value and impact in most cases.
–4) Some top geeks are experts at system (including compiler) internals i.e. under the hood + debugging, but not obscure low level details
* can uncover the “why” no one else can. Widely valuable.
** the very deep “why” is sometimes not that critical. In many real projects we don’t need to uncover the why if we can circumvent (find a workaround), including a rewrite
** in rare cases, we have no choice but find the “why”, but if the team doesn’t have expertise, they still would find some inferior solution or just live with the problem.
—-
[G=needed only at largest websites like Google, but many other employers pose these questions too.]

[13]choose income^easy life #damagedGood

Update — if I avoid the coworker-benchmarking, then I don’t mind sacrificing some income.

Q: Do you prefer a job (under a boss) you are comfortable with, at a typical IT salary? Or do you prefer a job with above-average income, but demanding, suppressive(like Sourabh) boss? In both cases, let’s assume long term job security is average (far from ideal), workload is reasonable, commute is average.

A [2013]: I’d favor the income if it’s more than $1k/m diff, but not sure if I can cope with the unfriendly environmental factors.

A: in early 2018 I chose Mvea team. No regret so far

Health + employability(security) are my real, long term priorities, not “premium” salary. My parents keep reminding me the damage of long-term workplace stress…

Recall the GS/Macquarie experience – workload, job nature, security…. but all are mild compared to performance warnings.

A[2018]: now I feel $2k/m will bribe me.

 

talent + attitude = great developer

In your gym, we discussed that some developers could be very sharp, knowledgeable, analytical, fast-thinking, fast-learning, but most of these developers don’t achieve the greatness like the achievement of your “Hibernate” colleague. I can think of a few other “great” achievers.

– A Citi developer who built the pricing engine almost single-handed, with so much logic thrown in. Perhaps design wasn’t great, but this engine works — no mean achievement given the tons of functionality. There’s so much logic that it took initially 2 developers and later 3/4 full time guys to support it. (One of the guys said this implementation is overcomplicated.)

– A ML manager who knows the entire trading engine (execution, offer/bid, RFQ, PnL, pricing, ECN..) inside out. The other 4 full time developers under him can’t support the app as well as he can by himself. When he is on leave for 2 weeks, we worry about if production issues will stay unresolved.

It takes more than talent + diligence to achieve greatness. For one thing, the guy must spend a few years dedicated to researching and experimenting with the core parts of the system, without cold feat. In my GS team, everyone did this. Outside of GS, i don’t see many doing this sort of hardcore inside-out system hacking, which is mandatory to mastery. Typically it takes 3 (up to 5) years to become a master, but the great developers take 25% of the time. They focus, they work (much) harder, they go deep — fearless and relentless. They seem to (quietly) believe that they “can do it” in 25% of the time, and they stick to the timeline and finish it. Determination, confidence, commitment and drive — common qualities of a few of my team leads.

They also must know when to give up on something hopeless, fruitless, worthless, or something with poor cost/benefit ratio. Without this critical Judgment, one would invariably waste time in all kinds of situations, and lose momentum, lose opportunities, lose management support, lose confidence. I believe the few great achievements I know are all achieved with a tight deadline. Wasting time is deadly. Great developers must know how to identify and avoid deathly traps.

Almost always, they also need to work with many others to finish a big task quickly, so some people skill is required, to varying degrees. I’m not good at analyzing this so let me skip.

Needless to say, none of these qualities are testable in an interview. Everything can be faked up. Therefore I believe it’s impossible to identify powerful achievers by job interview.

##[09] (career)safety net for %%family

See also [[#halo]], [[# towards an app architect]], [[http://bigblog.tanbin.com/2006/11/u-cant-be-developer-for-ever.html ]]

For a techie like me, what long term factors help to create a cushion or safety net?

#1) citizenship
#2) Zofia’s professional credentials
#) brank — titles in well-known companies
#) health and insurance
* home ownership + rental income
* English language skill for wife and me
* trec DBA
* trec presales
* trec PM
* post graduate degree -> teach