fear@large codebase #web/script coders

One Conclusion — my c++ /mileage/ made me a slightly more confident, and slightly more competent programmer, having “been there; done that”, but see the big Question 1 below.

— Historical view

For half my career I avoided enterprise technologies like java/c++/c#/SQL/storedProc/Corba/sockets… and favored light-weight technologies like web apps and scripting languages. I suspect that many young programmers also feel the same way — no need to struggle with the older, harder technologies.

Until GS, I was scared of the technical jargon, complexities, low-level API’s debuggers/linkers/IDE, compiler errors and opaque failures in java/SQL … (even more scared of C and Windows). Scared of the larger, more verbose codebases in these languages (cf the small php/perl/javascript programs)… so scared that I had no appetite to study these languages.

— many guys are unused to large codebases

Look around your office. Many developers have at most a single (rarely two) project involving a large codebase. Large like 50k to 100k lines of code excluding comments.

I feel the devops/RTB/DBA or BA/PM roles within dev teams don’t require the individual to take on those large codebases. Since it’s no fun, time-consuming and possibly impenetrable, few of them would take it on. In other words, most people who try would give up sooner or later. Searching in a large codebase is perhaps their first challenge. Even figuring out a variable’s actual type can be a challenge in a compiled language.

Compiling can be a challenge esp. with C/c++, given the more complex tool chain, as Stroustrup told me.

Tracing code flow is a common complexity across languages but worse in compiled languages.

In my experience, perl,php,py,javascript codebases are usually small like pets. When they grow to big creatures they are daunting and formidable just like compiled language projects. Some personal experiences —
* Qz? Not a python codebase at all
* pwm comm perl codebase? I would STILL say codebase would be bigger if using a compiled language

Many young male/female coders are not committed to large scale dev as a long-term career, so they probably don’t like this kinda tough, boring task.

— on a new level

  • Analogy — if you have not run marathons you would be afraid of it.
  • Analogy — if you have not coached a child on big exams you would be afraid of it.

I feel web (or batch) app developers often lack the “hardcore” experience described above. They operate at a higher level, cleaner and simpler. Note Java is cleaner than c++. In fact I feel weaker as java programmer compared to a c++ programmer.

Q1: I have successfully mastered a few sizable codebases in C++, java, c#. So how many more successful experiences do I need to feel competent?
A: ….?

Virtually every codebase feels too big at some time during the first 1-2 years, often when I am in a low mood, despite the fact that in my experience, I was competent with many of these large codebases.
I think Ashish, Andrew Yap etc were able to operate well with limited understanding.
I now see the whole experience as a grueling marathon. Tough for every runner, but I tend to start the race assuming I’m the weakest — impostor syndrome.
Everyone has to rely on log and primitive code browsing tools. Any special tools are usually marginal value. With java, live debugger is the most promising tool but still limited pain-relief. Virtually all of my fellow developers face exactly the same challenges so we all have to guess. I mean Yang, Piroz, Sundip, Shubin, … virtually all of them, even the original authors of the codebase. Even after spending 10Y with a codebase, we could face opaque issues. However, these peers are more confident against ambiguity.
Advertisements

impostor’s syndrome: IV^on-the-job

I feel like impostor more on the job than in interviews. I have strong QQ (+ some zbs) knowledge during interviews. I feel it more and more often in my c++ in addition to java interviews.

Impostor’s syndrome is all about benchmarking. In job interviews, I sometimes come up stronger than the interviewers, esp. with QQ topics, so I sometimes feel the interviewer is the impostor !

In my technical discussions with colleagues, I also feel like an expert. So I probably make them feel like impostors.

So far, all of the above “expert exchanges” are devoid of any locaySys. When the context is a localSys, I have basically zero advantage.  I often feel the impostor’s syndrome because I probably oversold during interview and set up sky-high expectations.

QQ study=burning_joy

See also the pleasure^chore framework

Unbelievably, QQ study feels like “burning” pleasure. After I complete some chore like coding drill, tax, I long for some QQ study !

There are very few burning pleasures:

  • tech study including zbs, halo…
  • localSys hacking — camp_out
  • weekend coding assignments
  • jogging in stadium
  • learning Oracle or unix in my younger days — camp_out

##IV Q:Name some big-impact projects U contributed to

Kyle pointed out that your change could be a 10-minute job with big impact, but only because there was already an infrastructure. You leverage the infrastructure to create the impact. For a story to be compelling, it has to show complexity and personal contributions

  1. NYSE integrated feed. Most of the financial data websites get nyse data from us
  2. Baml muni trade execution — rewrite from scratch
  3. billing/commission calc at PWM.
    1. eg: mortgage commissions from scratch
    2. eg: special investment (hedge funds) commission rewrite
  4. Citi muni re-offer engine. Biggest muni house in the U.S. Lots of retail investors buy muni bonds through their investment advisors. My changes are actually incremental.

good@pushing myself

am good at pushing myself to the limit, sometimes beyond my limit, even if i may not be “good at” them in a competitive sense.

  • yoga
  • jogging
  • coding drill
  • camp out

coding drill is arguably the most interesting case study. Other people (including students) may complete more problems, but i probably spend more time

  • Ranked by number of problems, i’m just average
  • ranked by hours spent, I’m much higher
  • ranked by effort, I’m even higher, as I push myself harder than others

## stable base;fluid superstructure in G5 tech skills

My perspective in this post is mostly tech interview — accumulation, retention of knowledge, prevention of loss due to churn. As I age, I’m more selective what new technology to invest into.

In contrast to the interview perspective, the GTD perspective is dominated by localSys ! So churn doesn’t matter.

Many of these technologies are past their peak, though none is losing relevance. However, I tend to perceive these tech skills as robust and resilient, time-honored. Among them, I see a partial pattern — Many of them exhibit a stable base; some of them show a fluid superstructure of skillset.

  1. essential algorithms/data_structures and bigO — stable, possibly growing base + fluid superstructure
  2. java skillset — has a stable base i.e. coreJava + fluid ecosystem including jGC, jxee
  3. C/C++ — has a stable base skillset including TMP, STL… The superstructure is fluid mainly due to c++0x
  4. SQL and socket — each has a stable base
  5. pthread, unix internals ..– ditto
  6. unix (no QQ topics, but many GTD skills) — stable superstructure skillset including instrumentation, utils and scripting. Base? tiny.
  7. http stack — stable but tiny base including session/cookies + fluid superstructure

## IV skills: disaster-rescue 逃生

I have relied on this parachute over and over again. Therefore, I have real motivation to strengthen this parachute and keep it in good condition.

  • OC — parachute saved me from transfer to BAU
  • 2007 after first entry to U.S., parachute saved me from sitting on bench for months as happened to … Florence
  • Vz — after my contract was cut unexpectedly, parachute saved me from sitting on bench. This proves to be the first of many rescues including 95G, Barclays,
  • [B] Stirt — after the layoff, parachute helped me avoid sitting on bench for months.
  • [B] Macq — after the PIP, parachute saved me from another slow job search on SG job market
  • [B] deMunk — parachute gave me confidence that I had the technical capabilities to escape — My “parachute” was my marketable skillset
  • [B=example of boxer cornered n beaten]

Other people’s examples:

  • [B] Youwei — his java IV skills rescued him from the MS layoff
  • Venkat of OC — it rescued him from a terrible boss
  • Shanyou and Deepak — need better parachutes
  • Jack Zhang
  • Davis Wei

old-timer paid below grads; unable2quit #Davis

My friend Davis revealed to me that many non-VPs earn below the $115k salary offered to a fresh Master’s grad.

  • Davis said employer won’t give you more than 3% annual increment, so quite often it can’t reach $115k.
  • Anthony also said the big hike happens only when you change job [1].
  • Jack Zhang said over 10 years the total increment could add up to below 20k.

Q3: what’s the market rate for your skill as an old timer?
A: probably higher than the grads.

Q3b: so why the old-timers don’t get a better job elsewhere?
A: they don’t feel they can.
A (Davis): these old timers have value-add only because of their localSys knowledge. In a sense, some new-age employers have a prejudice against people of loyalty. They probably associate Loyalty with stagnation and obsoleteness.

Therefore, one-long-job resume can be bad when you change career. I always felt my job-hopper resume is a liability, but some west coast shops don’t care.

I feel these old-timers are afraid of failure [1] at the new job, perhaps after a long, stressful adjustment. I think people do notice that adjustment to a new environment can be very tough and often unsuccessful.

Contractors keep adjusting.. stronger, more adaptable, more confident than those loyal old-timers.

Q6: why is employer willing to pay grads so much more?
A: Employers don’t want to but that’s the market rate set by supply-demand. Ibanks want to bring in fresh talents and must pay the market rate.

[1] A.Gambino discussion .

GS CIV:highest point]%%visible python proficiency

One of the few real pieces of evidence of progress. Average 1-2 times each year.

I should really recognize (not dismiss) the fact that, the 3 questions together represent the toughest python speed coding test I have ever taken.

GS is selective and python is relatively unfamiliar to me. Before 2019, I would not have imagined I could pass this tough coding interview.

  • extreme time pressure, comparable to Indeed and FB
  • unfamiliar territory — linear probing, defaultdict etc
  • completed without error
  • many issues encountered and resolved on the spot
  • bigO analysis performed. No performance bug
  • added additional tests. My implementation is basically flawless.
  • high energy, high concentration
  • highest GTD proficiency ever

G5 unexpected big wins]%%life #absorbency

should create pointer blogposts in ‘ffree’ and ‘diet’ blogs

20 uphill breakthroughs #wellness/xx is a similar list.

My unexpected success at weight improvement and diet is rather rare … the latest of top 5 achievements of my entire life. Other significant (often unexpected) successes:

  • outstanding project performance in 95G, Barcap and RTS. In other teams, I was able to do a reasonable job among bright young professionals.
  • QQ technical wins on high-end c++ positions (10+ times each), a golden ticket to high salaries on Wall St and then SG
  • [r] high absorbency for self-learning till this age.
    • MSFM
    • 😦 otherwise, the 4.5 years in SG was low.
  • [r] ffree first as bachelor and again in 2018.
  • [r=really rare]

How about my Cambodia investments? No really a personal effort per se.

##fully engaged+!ROTI still beats boredom

Every learning direction has naysayers, and virtually all learning efforts generates low ROTI. ROTI and strategic investment are holy grails, mirages or white elephants.

We can’t dismiss $ROTI but I think we really need to look beyond $ROTI — too elusive. As Mithun put it, java multithreading alone can fetch you highest salary.

Fully engaged for a few short months but without ROTI is actually not bad. It’s like satisfactory sex without simultaneous climax. I enjoyed the journey, without worrying about the destination.

I’m able to fully engaged for a few months, while my peers can’t!

I have blogposts on spare time usage. Tech learning (even the non-strategic topics) are among the most productive. There’s really nothing more worth learning.

  • — most of these are failed technology bets, but 10x better than not engaged (i.e. boredom)
  • 2012-2013 c# study
  • HFT-specific QQ topics
  • MSFM
  • 2011-2012 self-learning volatility
  • self-learning swing
  • MOM study
  • gemfire study
  • — less disappointing ROTI in terms of self-confidence, thick=>thin/zbs, mobility, broad-base
  • bond math, socket, FIX, data structure, algo, threading,
  • coding drill, including c++ language experiments
  • specific SDI and pure algo challenges

##[19] am competent at..as a professional thanks2tsn

Background — When I listen to a professional musician or comedian from some unfamiliar country, I wonder if they are actually good. Similarly, when I consult a doctor or dentist, I wonder if they are qualified.

“Self-respecting programmer” — Yi Hai’s motivation.

I have been tested and proven on the big stage i.e. U.S. tech interviews + GTD

  • [p t w] java/c++/c#
  • [w] algo
  • [w] coding test
  • [t] SQL
  • [t] socket
  • [t] unix power user
  • swing
  • [p] web app developer across php, javascript, java
  • py, perl (and shell/javascript?) — are these professional games?
  • [t = long tradition over 20Y]
  • [w = worldwide contest]
  • [p = a well-known standard profession]

##[15] I do professionally qualify as …

See also ##[19] am competent at..as a professional

See also blogpost on “N years’ experience — unimportant” and #1 career safety enhancer@past5years

On Wall St there are “combo” jobs to my advantage, but unheard of in SG.

I can qualify as :

became expert via external QQ benchmark` !!localSys xx

Suppose I invest heavily and become very productive on a local c++system. Similarly, a java, or SQL, or socket, pyhton .. system, but the point is — Any local system uses only a small portion of the language features.. at most 5% of the typical high-end interview topics on that language.

Therefore the project experience won’t give me confidence to be an expert on the language, but acing multiple QQ interviews does, as these interviews compare dozens of competitive, motivated and qualified candidates across a wide field.

I grew very confident of java through this type of external benchmarking, in contrast to internal benchmarking within the local team.

low-level^high-level expertise

See also my blogpost on wrapper^low-level API

Compare to laymen on the street, I have accumulated fairly deep, specific and demonstrable non-local expertise in two lucrative field i.e. software dev + finance

  • concurrency details (theory++) in real languages
  • dStruct choices and effective combinations
  • dStruct implementation details in java/c++/pyhton
  • SQL joins and tuning
  • sockets? no deep insight but my expertise beats most peers
  • Most standard algo problems are familiar to me
  • memory models, memory layout.. beneath c++ and java
  • drv pricing math, VaR statistics, bond math. My expertise goes beyond the degree

These trophies are won based on TSN, interviews and personal time investment … not automatically

— Let me focus on low-level vs high-level
Most of the dev expertise domains are low-level, consequently very specific — you either know it or don’t. I can even teach in these subjects. The more low-level, the rarer is the expertise. The laymen developers have a vague idea of the low-level details, for many reasons.

High-level understanding is often more useful (than low-level, theoretical QQ knowledge) in projects, but in job interviews, low-level knowledge is differentiator.

Practically all (98%) high-end java/c++ interviews use low-level questions as differentiator. I excluded the start-ups as I’m unfamiliar with them… Eg: Quoine.

In my dev (not system architect) interviews, they rarely asked high-level stuff like spring rationale, design patterns…
I tend to believe these questions can’t /separate the crop from the chaff/

The finance dnlg  also serves as differentiator among developers. The most specific sub-domains are quant math, the “architecture” and some of the jargon.

C++ is more low-level than java; java is more low-level than python…

Q: Paradoxically, C and assembly are even more low-level but not more valuable?
%%A: Some C knowledge is highly valued. For example, kernel knowledge is high-value and mostly at the level of C, compiler, assembly, and hardware
%%A: assembly expertise in device drivers or embedded is not really relevant to HFT, and too far from the money

MOM is not low-level and seldom asked.

fellow experts siz`up each other≈ QQ IV

  • Scenario — Female engineer at an electronics exhibition booth, in the 70’s.
  • Scenario — a passenger falls sick on a flight and two doctors (different countries) came to the rescue. They size up each other to decide who to take the lead.
  • Scenario — two musicians across genres play together impromptu for the first time. Musicians usually are cooperative not competitive though.
  • Scenario — a comp science student visits another campus and logs in on the local network and start exploring (not “hacking”) and encounters local students on the network.
  • Scenario — provincial basketball player coming to a new town and plays a first one-on-one game with a top local player.
  • Scenario — an interesting algo challenge is openly discussed, among programmers from different countries and across industries. A problem to be solved in any language without external tools.

These scenarios are similar to QQ interviews. (Algo interviews are subtly different but mostly similar.) Venkat of OC impressed me with his c++ and c# QQ knowledge.

When fellow experts size up each other … what acid tests can they use? Algo, QQ topics or ZZ topics (I tend to read in my spare time)?

To really size up each other, we need to discuss common topics, not Your or My localsys.

.. but many people only talk about, in vague or high-level terms, how some localSystems were supposed to work, often based on hearsay. They leave out the important details. It’s impossible to verify any of those claims.

When we watch a movie from Europe, India, Korea … we can tell whether it has any international appeal or only limited local audience.  Similarly localSys knowledge has value only within a single company. The expertise I demonstrate in job interviews are relevant across many sites.

If I were a 5Y VP (or ED) I would have self-doubt — am I just a localSys expert or an accredited expert as proven on benchmarks/interviews? Look at Shubin/Steve, the Sprite expert in London, Richard of Quoine ..
Their expertise and value-add can be marginalized in the context of the current mainstream technologies.

##dry$valuable topics 4 high(absorbency)period #flight

This ranking was originally compiled for “in-flight edutainment”. Warning — If high churn, low accu, low traction, or short shelf life then the precious absorbency effort is wasted

See also the pastTechBet.xlsx listing 20+ tech topics and comparing their mkt-depth, demand-trend, churn-resistance … My conclusion from that analysis is — any non-trivial effort is either on a niche or a non-growing tech skill, with notable exceptions of coreJava+threading. All mainstream and churn-resistant skills need only trivial effort.

  1. coding drill — esp. the hot picks. Look at tag “top50”. Practice solving them again quickly.
    • 😦 low accu?
  2. java concurrency book by Lea. More valuable than c++ concurrency because java threading is a industry standard reference implementation and widely used.
  3. java tuning, effJava,
  4. c++ TMP?
    • 😦 seldom asked but might be asked more in high-end interviews. TMP is heavily used in real world libraries.
    • 😦 low traction as seldom used
  5. effModernC++
  6. linux kernel as halo?
    • 🙂 often needed in high-end interviews
    • 😦 low traction since I’m a few layers removed from the kernel internals
    • 😦 no orgro
  7. c++11 concurrency features?
    • 😦 low traction since seldom asked in-depth

c++QQ critical-mass[def2] has started growing!

If I compare myself with young c++ developers or older guys (like CSY, Paul..) I can see clear patterns in QQ and GTD.

  1. QQ and (to a lesser extent) zbs — I’m clearly pulling ahead, sometimes heads and shoulders above them. Some of these guys have wider QQ knowledge but lacks depth. I wrote about “experts” sizing up each other..
  2. zbs: instrumentation — one of the key areas of improvement for me! However, I can see many of the older guys at RTS aren’t more knowledgeable.
  3. GTD: paradoxically, the younger guys are more productive than me or older guys.

— Now let’s retrace the gradual breakthrough

In May 2019, I felt I have achieved enough critical-mass on c++ QQ topics. Critical mass is defined by The two acid test questions.

Q1: without a full-time c++ job, but with enough interviews, will my c++ QQ insight/understanding show resilience against churn and memory fading, as in coreJava?
Q2: thick->thin achieved? Not yet, but cross-reference graph is now built up as a defense against fading memory

This java career review provides a valuable context.

What visible progress gave me this level confidence? Recent technical wins show my improved ranking among c++candidates.

  1. SCB-FM
  2. CVA
  3. SIG
  4. TradeWeb core team

Note I have invested more effort on c++ QQ than java… [18]t-investment: c++now surpassing java

  • — now a sample of critical-mass topics, roughly ranked by importance on high-end interviews, mostly at HFT and ibanks
  • coding tests
  • mv-semantics
  • containers
  • smart ptr
  • heap memory mgmt including new..
  • polymorphism including MI #44 posts in the category
  • TMP — important at high-end but not HFT
  • [e] pthreads + c++11 threads
  • [e] sockets
  • runtime costs of virtual and heap, as Stroustrup explained
  • [e] cache efficiency, compiler optimizations,
  • [e] linux
  • [e] new innovation directions
  • [e] benchmarks involving c++ as Stroustrup explained
  • build tools
  • [e=ecosystem topics]

both DEPTH+breadth expected of%%age: selective dig

Given my age, many interviewers expect me to demonstrate insight into many essential (not-obscure) topics such as lockfree (Ilya),

Interviewers expect a tough combination of breadth + some depth in a subset of those topics.

To my advantage I’m a low-level digger, and also a broad-based avid reader. The cross-reference in blog is esp. valuable.

Challenge for me — identify which subtopic to dig deeper, among the breadth of topics, given my limited absorbency and the distractions.

[19] body-build`does hit low visPgress#leverage,disengaged

In Singapore (but very few NY 🙂 jobs, I noticed a pattern — the longer I stayed on a stable job, the lower I dropped in motivation, incentives and positive feedback/reinforcement for IV body-building including QQ, coding..

Every time I pass a non-trivial tech screening, I feel a real boost … Reinforcement of absorbency and reinforcement of a wellspring of energy lasting a few days to a few months … sometimes years thanks to my retrospective blogging. My Singapore experience is missing this crucial IV element. Without it, my absorbency of dry technical learning is hard to sustain. This also explains why my absorbency of localSys is hard to sustain.

(My son has never experienced such positive reinforcement.)

To gain perspective I find it necessary to compare with other endeavors. My conclusion — body-building has the highest leverage. See also meaningful endeavor(Now)4family: IV^zbs^gym..

Whenever I feel guilty/ashamed of my fixation on IV, and try to learn zbs, localSys, GTD etc,  eventually (often quickly) I hit a broken reinforcement loop and a weak or depleted “energy supply” and invariably give up, very rationally.

Q: is there any /endeavor/ with higher visPgress than IV body-building?

chores that require absorbency visPgress #immediate $ROTI leverage over long-term, on family well-being
body-building Yes in the form of blog+github… not $$ [1] reasonably high, given huge $gain and huge t-investment 😦 higher leverage than everything else combined [2], despite the churn
… cf localSys highly visible respect, not $$ ->necessary but insufficient condition for move-up
non-prop investment easily visible but small low given the small profit no leverage so far
yoga (+ fitness) some but hard to keep $zero high leverage, well-known
diet for BMI highest but hard to keep $zero 😦 low since it’s hard to keep up

[1] I think many of my peers can’t keep up the body-building effort precisely because there’s no $ROTI… personal projects: any ROI ] salary@@
[2] direct fruits of the endeavor:

  • made my nice TPY home possible
  • enabled me to move between countries
  • made GC possible
  • gave wife the SPR then Singapore citizenship
  • gave me decent salary
  • supported my property investments

criticalMass[def]against churn ] tech IV

See also body-building impact{c++jobs#^self-xx

IV knowledge Critical mass (eg: in core java self-study) is one of the most effective strategies against technology churn in tech interviews. Once I accumulate the critical mass, I don’t need a full time job to sustain it.

I have reached critical mass with core java IV, core c++ IV, swing IV (no churn) and probably c# IV.

The acid test is job interviews over a number of years.

Q: how strongly is it (i.e. critical mass) related to accumulation?
A: not much AFTER you accumulate the critical mass. With core java I did it through enough interviews and reading.

Q: how strongly is it related to leverage?
A: not much though Critical mass enhances leverage.

Q: why some domains offer no critical mass?
A: some (jxee) interviews topics have limited depth
A: some (TMP, py) interview topics have No pattern I could identify from interview questions.

 

%%engaging[def2]deep dive shines through QQ interviews

Opening example — I had 3M of fixation on tibrv^JMS. Just look at my blogposts. I go beyond interview topics. This fixation hopefully shines through on interviews.

This post is more about “engaging” than focused effort — I couldn’t really force myself to focus on speed coding or c++ TMP, or high-churning javascript.

Ken Li (Citi-muni) was probably a sharp observer of my personality and once said “If you are interested in software development then you will have a long career till retirement … no need to worry. A lot of developers lack that interest.”

Very few candidates had more than 3M personal spare time engaged in any tech topic.. Something like a deep dive. (Even fewer can sustain 12M focus.) Consider XR’s interest in machine learning. Consider grandpa.

Apparently, many interviewers are personally interested in these same details (the whys/hows/whats) so they seek out like-minded enthusiasts. This is one hypothetical explanation why those crazy theoretical QQ questions come up so often.

Other examples of personal fixation that (hopefully) shine through

  • sockets — blocking operations …; unplugged
  • clustered index
  • timer hardware interrupts
  • userland vs kernel mode
  • nested classes across languages
  • compiler rules on overriding

Now, many of the weeks spent has low ROTI but I was not fixated on ROTI since my earliest engaging studies in 1998 — first deep dive in unix + C programming + perl. As it turned out

  • unix and linux became dominant on server side and enhanced my career
  • perl became m entry point into programming career
  • C domain was hopeless, but with c++ I was able to capitalize on it

Still, many of my engaging studies had low ROTI, but what if I didn’t capture my interest? I think I would end up wasting my spare energy!

Striking balance — too much “engaging” –> regretful ROTI; too little “engaging” –> bored and unfulfilled.

denigrate%%intellectual strength #ChengShi

I have a real self-esteem problem as I tend to belittle my theoretical and low-level technical strength. CHENG, Shi was the first to point out “你就是比别人强”.

  • eg: my grasp of middle-school physics was #1 strongest across my entire school (a top Beijing middle school) but I often told myself that math was more valuable and more important
  • eg: my core-java and c++ knowledge (QQ++) is stronger than most candidates (largely due to absorbency++) but i often say that project GTD is more relevant. Actually, to a technical expert, knowledge is more important than GTD.
  • eg: I gave my dad an illustration — medical professor vs GP. The Professor has more knowledge but GP is more productive at treating “common” cases. Who is a more trusted expert?
  • How about pure algo? I’m rated “A-” stronger than most, but pure algo has far lower practical value than low-level or theoretical knowledge. Well, this skill is highly sought-after by many world-leading employers.
    • Q: Do you dismiss pure algo expertise as worthless?
  • How about quant expertise? Most of the math has limited and questionable practical value, though the quants are smart individuals.

Nowadays I routinely trivialize my academic strength/trec relative to my sister’s professional success. To be fair, I should say my success was more admirable if measured against an objective standard.

Q: do you feel any IQ-measured intelligence is overvalued?

Q: do you feel anything intellectual (including everything theoretical) is overvalued?

Q: do you feel entire engineering education is too theoretical and overvalued? This system has evolved for a century in all successful nations.

The merit-based immigration process focus on expertise. Teaching positions require expertise. When laymen know you are a professional they expect you to have expertise. What kind of knowledge? Not GTD but published body of jargon and “bookish” knowledge based on verifiable facts.

##%%comfortable position among peers

##[19] living%%dream life, here-n-now is more useful because I need to progressively shift my focus away from peer-comparison. and focus more on comparison to my past.

We instinctively know, to varying degrees, where we stand among our chosen peer group. On some games I feel I am unlikely to improve my relative standing, so I need to reconcile:
* kids academics
* brank — I didn’t say “income”
* double income, white-collar wife

On other “fronts”, I feel confident to maintain my comfortable position relative to peers:

  1. (  roughly ranked by surprise ! )
  2. sg citizenship — low-cost health care, car-free, property tax
  3. work-till-70, life-long learning, access to age-friendly job markets
  4. stable marriage, valuable support from grandparents
  5. lower fear of job loss, lower work stress .. thanks to contract
  6. wellness
  7. mobility across multiple job markets
  8. home ownership — low maintenance
  9. top college — relaxed attitude towards and some insight
  10. coding IV — benchmarked against my age group on Wall St
  11. QQ IV beyond coreJava — c++, c# ..
  12. cashflow — burn rate, retirement planning, ffree

## nothing to be proud of ] %% life@@

See also [2] ##4 qualities I admire in%%peers #!!status and
[1] ##envy Them constantly; !!celebrat`family,achievements..

Once a while in a young person’s life she(he) find herself (rather than her life) despicable in every way —

  • studies
  • dating
  • body image
  • fitness
  • team sports, not part of any “gang”
  • ….When we grow older we have some of these same “whiplashes” + some new whiplashes, which we use to beat ourselves up:
  • profession
  • home location, home size, school district
  • personal investments including properties
  • monthly saving rate
  • plenty of life-chances available to me throughout my life

I think in such a situation we desperately need concrete, tangible evidence of personal achievement (not necessarily visible to other people). Here are some evidence to congratulate myself and defend myself against the whipping:

  • wife and kids — see [1]
  • SGP citizenship — see [1]
  • education credentials — see [1]
  • personal finance — see [1]
  • profession — see the first list in [1]
  • knowledge + experience about diet and wellness — see [2]
  • introspective or expressive writing and blogging — as a versatile coping, self-improvement, self-discovery device
    • English proficiency — I struggled for years and now I’m able to use English for expressive writing — something unimaginable in my teenage years.

engagement+spare time utilization: %%strength

Very few peers are so conscious of burn^rot. Higher utilization of spare time is a key strength during my US peak + my dotcom peak + also my high school. We could analyze what’s common and what’s different between these peaks…

Outside those peaks, I also used this strength to complete my UChicago program, but the tangible benefit is smaller.

(This is different from efficiency on the job. Many efficient colleagues spend less time in office but get more done.  My style involves sacrificing personal spare time and family time.)

Looking forward, I guess this strength could be strategic for research-related domains, including any job involving some elements of research and accumulation.

A repeated manager praise for me is “broad-based”, related to this strength.

many IV failures before 1st success #c++/HFT/CIV

  • i had so many failures at c++ interviews before I started passing. Now I look like a c++ rock star to some
  • I had so many failures at HFT interviews before I started passing at DRW, SIG, Tower
  • I had so many failures at remote speed coding before I started passing.

With west-coast type of companies including nsdq, I can see my rise in ranking. If I try 10 more times the chance of further progress is more than 70%.

I /may never/ become a rock star in these west coast CIVs but I see my potential as a dark horse in 1) white-board CIV, 2) pure-algo

“May never” means “may not happen” … Scott Meyers

[18] CONTINUOUS coding drill #Shi

My friend CSY said some students like his son could conceivably focus “all their time” on one skill (coding drill) for fours years in college, so they will “surely” outperform.

I pointed out that I am often seen as such an individual, but my speed coding interview performance is hardly improving.

I pointed at the number of Leetcode problems I solved with all tests passed. It grew by up to 10 each year, but a student can solve 10 leetcode problems in half a day.

I gave an analogy of my Macq manager’s weekly slow-jogging, for health not for performance. Consistent jogging like that is great for health. Health is more important than athletic performance. I said jogging and my coding drill are life-style hobbies and recreations.

For years I practiced continuous self-learning on

  • java, c++, SQL, MOM, swing — IV and GTD growth
  • Unix, python — mostly GTD
  • quant

I consider my continuous self-learning a key competitive advantage, and an important part of my absorbency capacity.

I asked a bright young Chinese grad ChengShi. He practiced real coding for months in college. I said “fast and accurate” is the goal and he agreed. A few years into his full time job, he stopped practicing the same. Result? He said for the same coding problem he was able to make it work in 10 min then, but now probably 20 minutes. I think this is typical of the student candidates.

I asked “Do you know anyone who keeps up the coding drill?” He didn’t tell me any individual but gave a few points

  • he believed the only reason for coding drill is job hunting
  • when I said continuous practice all year round will make a big difference compared to practice only when job hunting, he agreed wholeheartedly and reiterated the same observation/belief
  • but he disagrees that continuous coding drill would lead to significant professional growth as a programmer, so he would probably channel his spare energy elsewhere.
  • I think he has other ideas of significant growth. At my age, I don’t foresee any “significant growth”.

## 9 c++realized $ROTI

I wanted c++ ROTI. After so many years of trial and error, I got two

  • G3 [18] The CVA $122/hr offer
  • G3 [18] SCB-FM S$210k offer, unthinkable in my Singapore job search.
    • In terms of base, This one is about $$190k. My “reasonable” target was S$150k and my “high” target was $170k.
  • G5 [18] SIG technical win
  • G5 [12] BNP forex prop trading contract offer
  • [19] MLP-sg java connectivity team actually has a small c++ requirement.
  • G9 overcame fear@large codebase]c++/j and emerged above most developers.

[19]%%competitive strengths as professional techie

In my 30’s I would look at the managers sitting in their private offices, and tell myself “I’m not a manager type. I’m the technical type”. However, I have always failed to rise above the rest and reach tech leadership, like senior architect or tech fellow.

Eventually I concluded my strength was limited to theoretical interview topics in java and math.

After spending 2 years self-learning and 3 years in UChicago on quant, I still feel stronger in math than my fellow developers, but math is becoming less and less important to most developer jobs. I can’t afford to invest more time in math (like CSDoctor does), because jobs are scarce.

Now finally I can add “c++” after “java” in my profile.

Pure algorithm (without real-coding test) was my traditional strength, but now I know the west coast and HFT standard. I think it’s still my strength by ibank standard. With more mileage and 打通脉络 I can grow stronger.

I was seldom very strong in GetThingsDone, but ironically the GTD guys seldom rise up. GTD productivity is a necessary but insufficient condition for promotion.

##controversial decisions #home,imm,retire..#YJL

Hi Junli,

You don’t need to reply. This is my periodic review of “everything in my life”.

I have recently implemented a few controversial decisions about my career, investment, family..

(As an example, the biggest is moving back to U.S. alone and starting the green card process.)

I make major decisions carefully and slowly (unless decisiveness needed), but an observer may say I’m not a good decision maker and point out my track record. Actually I don’t remember anyone pointed them out, not even my family members. The person who point a finger at my “unwise” decisions is the “judge” in my head…

Here are some of those controversial decisions

  • I will not give up Singapore citizenship, and I will retire in Singapore, relying on the Singapore government for my retirement. Singapore system is much more caring and efficient than China or U.S. systems.
  • I plan to work till 70 or older, perhaps for a token salary. I will keep up my interview skills.
  • I have stayed away from most of the new technologies — javascript, mobile apps, big data, social media, noSQL, block-chain … Instead, I have embraced the shrinking domain of c++
  • I feel my relationship and communication skills are not my strengths so through a series of trials-and-errors I have decided to stick to a technical career.
  • I’m staying in Bayonne, planning to buy my first home here. The schools are just above average.
  • I have always preferred home locations that doesn’t need a car.
  • At age 44 I decided to leave my family in Singapore and come to the U.S. to start the GC process

Name some Original contributions as a wall St techie@@

Background — expert test-takers and Math Olympians don’t create value. Grandpa is more than an expert in his domain — he created ground-breaking original research.

See also realistic+significant legacy of me as a developer@@

Generally, developers make more lasting impact than other roles. Developers are seen as value-creators. Here are my original contributions as a Wall St techie:

  1. 95G — I created the stored proc as the basis of concurrency control
  2. 95G — I created the wait/notify framework in B2B
  3. Barc — emulating the functional programming spreadsheet in java
  4. —-Below are less “original”
  5. RTS — nyse integrated feed XDP parser. Huge impact to billions of downstream users.

theoretical complexity!!guarantee4 relative-adv #STIRT

  • case: Stirt curve building, fixed income risk..
  • case: Qz technical complexities? But not theoretical, so I can’t read a book and then reason about it.
    • given more time, I will develop some in-depth understanding of Qz
  • case: code generator for python and excel add-on

I’m not so fast penetrating the cognitive barriers (built-up of local brain power), but theoretical complexity is still my relative advantage.

Somehow, all of these cases happened in Singapore, in perm roles! I guess as contractor I’m given relatively isolated, carved out, well-defined projects.

GS tech challenge !! beyond me

My GS technical challenge was tough but i conquered it convincingly! There were some technical challenges beyond me (such as the initial rule-engine rewrite project), but most technical challenges were within my grasp.

Somehow, over the years a negative self-image has overshadowed the positive evidence.

  • Everyone said I was technically good enough.
  • My mortgage solution code review were well-received. Mark then organized a “seminar” where I presented my mortgage solution to more than 10 guys.
  • Yang was tough but he liked my designs in errorMemos.
  • I grasped errorMemos code-base quickly, cleaned it up and were able to add new features quickly
  • I convinced many that I was a local technical expert in my team. Perhaps Chad was even stronger but that doesn’t change the above fact — consider Roger Federa.

22surprises{U.S.reentry #traction#stigma

There are many valuable observations below, but let’s not spend too much time polishing…

  1. self-esteem regained — in tech IV/GTD, after 5Y bleeding self-confidence #stigma
  2. coding tests — continues to spread. I improved progressively, gained traction — I even find it enjoyable.
    • dnlg — all 3 domain knowledge categories are losing weight in interviews. Note half my recent interviews are outside ibanks.
  3. my c++ competence (esp.sockets) finally gained traction, thanks to the interviews.
  4. rise of west coast salary level. U.S. tech job market didn’t lose steam. U.S. geek economy continues to grow
  5. Tristate housing — school-district housing is more expensive than I thought, but Edison/Bayonne can be quite affordable
  6. ! java remains robust and dominant in ibanks. c++ is robust too. There are still many c++ roles in U.S.
  7. [c] concentration window — proved to be extremely important to my learning, career planning and reflections. Parenting and household chores are real drags.
  8. [c] my peers didn’t “leave me in the slow track”. Most of them are still regular developers. I guess they can’t move up unless they were already in a lead role by age 35
  9. “strategic technology bet” — is thoroughly discredited, through repeated introspection

–Next 20

  1. [c] ibanks interviews — (including coding IV) continue to play to my advantage, after 5 years
  2. [c] Java QQ continues to feel natural to me… I didn’t lose most of my java QQ strength…
  3. [c] aging developers — I see good examples in Shubin, Paul, Shanyou, Alan, Daniel, Kam, Pinsky, John etc
  4. U.S. high-end contract rate — has grown from $90 to $110
  5. start-ups — There are many interesting start-ups both in U.S. and Singapore, able to pay.
  6. retire — I have decided to retire in Singapore not U.S. I see my Singapore citizenship as a huge strategic advantage over my Chinese/Indian peers.
  7. [c] wife was competent at her job and continues to keep the kids in the current condition without deterioration
  8. [c] kids — my daughter didn’t become alienated; my son didn’t get out of control.
  9. [c] I continue to take unpaid leaves to learn from interviews
  10. mkt data — enjoys growing demand and I gained traction more than I gained a new defensible territory.
  11. quant career and math aptitude — broken dream. Disillusioned. deep pain
  12. U.S. investment yield is typically 6%, higher than what I observe in Singapore.
  13. [c] ibanks didn’t reduce IT budget or go offshore as some predicted
  14. [c] HFT — is robust
  15. c++ demographics — mostly older
  16. apps and coding jobs … (in the global economy) are becoming more important, more wide-spread than I anticipated.
  17. [c = continuation, but unexpected]

##[15] 20 low-level IV topics : c++imt Java

This is collected after i stopped active interviewing in Aug 2012.

Low-level domains are my strength, proven again in 2017-2018 interviews. Beside the obvious domains — threading,  data structures, c++/java/c#..

  • exception in c++/java
  • concurrent singleton
  • GC
  • big-O analysis in west coast interviews
  • linux/compiler/C++ interviews are even more low-level than java. Despite serious “weak-joints”[1], I generally excel at the lower-level
    • shared mem — pointer therein!
    • sockets .. like kernel bypass
    • unix signals
    • cpu cache – instruction and data
    • inline
    • rvr, the most important c++0x feature, is very low-level
    • big-4
    • pbclone^pbref^pbptr, slicing
    • undefined behavior – usually are low level
    • internals of deque, vector/hashtable reallocation…
    • smart pointer internals
    • reinterpet_cast vs move()
    • pass a functor to a thread
    • [1] See items of the same color
  • —-On the other hand, interviewers don’t bother to go low level on …
  • thread pool? somehow never low-level
  • Unix commands, python? never low level
  • (noSQL and) SQL? no
  • FIX? not so low-level
  • SOA, MOM
  • asynch designs
  • design patterns

%%lead ] theoretical+lowLevel IV topics

See also the marketable_xp spreadsheet… I have consistently demonstrated strength in

1) TT: theoretical complexity: %%strength
2) LL: lowLevel IV topics (seldom needed in GTD) — threads, dStruct, vptr, language rules…

So how could TT/LL influence my 10Y career direction?

  • research domain? benefits from TT
  • mkt risk? TT
  • mkt data? LL
  • algo trading? Too competitive and poor market depth
  • quant dev? my TT advantage is not enough .. Too competitive and poor market depth
  • network optimization? My LL advantage is not enough
  • app owner? No. Not benefiting from my strengths and I tend to lose interest quickly

c++QQ/zbs Expertise: I got some

As stated repeatedly, c++ is the most complicated and biggest language used in industry, at least in terms of syntax (tooManyVariations) and QQ topics. Well, I have impressed many expert interviewers on my core-c++ language insight.

That means I must have some expertise in c++ QQ topics. For my c++ zbs growth, see separate blog posts.

Note socket, shared mem … are c++ ecosystem, like OS libraries.

Deepak, Shanyou, Dilip .. are not necessarily stronger. They know some c++ sub-domains better, and I know some c++ sub-domains better, in both QQ and zbs.

–Now some of the topics to motivate myself to study

  • malloc and relatives … internals
  • enable_if
  • email discussion with CSY on temp obj
  • UDP functions

how I achieved%%ComfortableEconomicProfile by44

(I want to keep this blog in recrec, not tanbinvest. I want to be brief yet incisive.)

See 3 ffree scenarios: cashflow figures. What capabilities enabled me to achieved my current Comfortable Economic profile?

  • — top 3
  • by earning SGP citizenships
  • by developing my own investment strategies, via trial-n-error
  • by staying healthy
  • — the obvious
  • by high saving rate thanks to salary + low burn rate — efficiency inspired by SG gov
  • by consistent body-building with in-demand skills -> job security. I think rather few of my peers have this level of job security. Most of them work in one company for years. They may be lucky when they need a new job, but they don’t have my inner confidence and level of control on that “luck”. Look at Y.W.Chen. He only developed that confidence/control after he changed job a few times.

When I say “Comfortable” I don’t mean “above-peers”, and not complete financial freedom, but rather … easily affordable lifestyle without the modern-day pressure to work hard and make a living. In my life there are still too many pressures to cope with, but I don’t need to work so damn hard trying to earn enough to make ends meet.

A higher salary or promotion is “extremely desirable” but not needed. I’m satisfied with what I have now.

I can basically retire comfortably.

##4+2 games I Aced,visibly #MSFM

I /aced / killed / thrived at /many games. Most visible and most profitable game in this list is tech IV, including

  • 1a) branching out to c# and c++
  • 1b) quant self-study to impress many technology interviewers

The above topic already has many many posts in this blog. Below are other games I excelled in:

  1. excellent grades up to college Year 1
  2. paid off multiple rental properties + my own home, by age 43. All in good locations with reliable rental demand.
  3. Earned MSFM with flying colors at age 42 — sustained focus, self mastery

Some domains are not really competitive “games” but still I excelled visibly:

  1. no belly (as Nick pointed out); weight loss in late 2018, along with pull-up.
  2. keeping burn rate very low, and achieving some form of ffree around age 30 and again at 43

It’s instructive to recognize the pattern.

  • I think in each game, I had some talent, and a lot of consistent effort.
  • External positive feedback is far from powerful , immediate or frequent, so internal motivation is crucial.
  • All individual games, not team games. Note promotion is not my game and I don’t need to kill this game to be comfortable and satisfied.

##past easy jobs : my80% did exceed benchmark

Ideally, I want to get a job role slightly lower than the highest salary, where my 80% effort can start to exceed the expectation of THE appraiser.

Grandpa said “At 100% if you don’t hit their requirement, then you don’t need to put in 120% and sacrifice family. It’s their hiring mistake. Their problem. if they don’t pay a compensation package then just leave.” Am I afraid of job change? See separate blogpost.

Looking at past jobs, the numbers below are very imprecise and subjective.

[G/g=Greenfield]
[B/b=brownfield]

  • [b] Citi muni — my 70<-80% might be enough, or might earn me no bonus like many Citi guys.
  • [B/g] GS — after the steep learning curve, my 100% was indeed enough to meet the high bar, but I feel 95% would not be
  • [G] 95G — after I proved myself, my 80<-90% was exceeding.
    • Big factor — colleagues were weaker
    • Big factor — my designs were favored by the boss
  • [G] Barc — after I proved myself, my 70<-90% was exceeding.
    • Big factor — I built a high-value, high visibility part of the system
    • Big factor — no one else was qualified to work on that
  • [g] Macq role was very senior — my 100% was NOT enough even on the basic devops part of the job. I once felt my 80% was fine in the 1st year, but actually expectation is much higher than I thought.
  • [B] Stirt — my 100% was barely good enough but not enough to earn a bonus. The project was a few months old when I joined but I struggled with the Qz platform:(. I was unfairly bench-marked against 3Y veterans! I would look decent if bench-marked against freshers.
  • [b] OC — my 70% would be enough but too relaxed
  • [b] RTS — After initial 9 months (NYSE+Aquis), my 40<-50% would be enough

 

## threading: a better specialization than algo,quant…

  • In summary — appears daunting and opaque but actually simple in practice
  • theoretical — like [[dougLea]].. my strength. Few coding tests and usually easy for me
  • fairly low-level — my strength. Not as low level as debugger, or template hacks …
  • GTD — no GTD challenges, since most projects use only simple threading designs. The code tracing, trouble-shooting expectation on me is relatively low.
  • opaque — for my peers. Even the basic condVar…
  • churn — low churn at the low level, but high churn at high level
  • ever-green — favorite topic of interviewers, esp. java
  • thin->thick->thin — I have achieved this for a long time.

Many of my halos are in this domain — ##halo across domains #specific=better

tech strength: Depth [def]

Some top geeks I know are fast at reading code + logs. Few rely on documentation. I’m OK not the fastest.

Some top geeks in “sister” teams of my team are experts with instrumentation tools and techniques. I guess other top geeks seldom need a lot of instrumentation. I feel they lack the experience but make up for it in other skills.

Some top geeks keep gaining depth if they focus on one complex system. I might have potential here. “Deep” is slightly different from “complex”. Deep means you need to slow down, /quiet down/, absorb, go through thick->thin cycles, get a laser focus, look again, and think deeper.

Perhaps related to my Depth capacity, I’m not that fast with timed online coding tests.

  • — some well-defined, often named technical subjects with depth, often opaque to the uninitiated
  • rvr, RVO
  • RAII, swap
  • [b] TMP including SFINAE, CRTP
  • [b] java generics
  • [b] JNI, python extension module — complexities lurking by the /dimly lit path/ due to low-level interfacing
  • java/c# reflection techniques in practice
  • ? python introspection
  • [B] concurrency in general and java threading in particular
  • [B] algorithms and data structures
  • clever SQL single join to solve tricky problems.
  • [B] query/sproc tuning — depth created by the large number of tables/indices and variations in queries
  • ? serialization over network
  • ? ADL techniques
  • any math and statistics subject
  • [b/B=books have been written on this topic]

addiction2low-level hacking:keep doing; no shame

Update: low-level hacking is generally easier in c++ than java.

When I become interested in a tech topic, I often throw cold water over my head — “This is such a /juvenile/, albeit productive and wholesome, hobby. Look at ex-classmates/colleagues so and so, with their business wing. They deal with business strategies. My tech stuff is so low-level and boring compared to what they deal with.”

Damaging, harmful, irrational, demoralizing SMS! Get Real, Man! Let’s assess our own situation

  • A) On one hand, I need to avoid spending too much time becoming expert in some low-leverage or high-churn technology (php? XML? ASP?).
  • B) On the other hand, the enthusiasm and keen interest is hard to get and extremely valuable. They could be the catalyst that grow my zbs and transform me into a veteran over a short few years. Even with this enthusiasm and depth of interest, such a quick ascent is not easy and not likely. Without them, it’s simply impossible.

Case: grandpa. His research domain(s) is considered unglamorous 冷门 but he is dedicated and passionate about it. He knows that in the same Academy of social sciences, economics, geopolitics and some other fields are more important. He often feels outside the spotlight (kind of sidelined but for valid reasons). That is a fact which had a huge impact on my own choice of specialization. But once he decided to dig in and invest his whole life, he needed to deal with that fact and not let it affect his motivation and self-image. As a senior leader of these unglamorous research communities, he has to motivate the younger researchers.

Case: Greg Racioppo, my recruiter, treats his work as his own business. The successful recruiters are often in the same business for many years and make a long term living and even create an impact for their employees (and people like me). They could easily feel “boring” compared to the clients or the candidates, but they don’t have to.

Case: PWM wealth advisors. They could feel “boring” compared to the filthy rich clients they deal with, but in reality, these advisors are more successful than 99% of the population.

Case: The ratio of support staff to traders is about 50:1, but I don’t feel “boring” because of them.

Case: Look at all the staff in a show, movie, supporting the stars.

##respect,$,familyTime,spareTime..all benefit from强项

If I take a 强项 job, I kind of sacrifice my muscle building and TrySomethingNew, so I better get good money …, but now I feel I don’t have to.

A typical 强项 job would use (the most complete list)

  • math, analytics
  • high volume data processing
  • unix wizardry
  • heavy text processing
  • heavy SQL
  • heavy scripting
  • some http programming
  • data analysis, perhaps using SQL, scripting etc
  • some combo of java/c++/c#/swing
  • threading

tech strength^weak, as@Nov2016

My strengths – quick but more than superficial learning of (in random order):

  • finmath, financial jargon,
  • Comp Science practical topics:
    • OO fundamentals
    • algo, data structures,
    • threading
    • basic tuning of DB + others
  • language fundamentals (more respected in the java, c++, c#, SQL than python communities)
    • versatile in many important languages
  • relative familiarity with scripting, unix, SQL, but not IDE, version control etc
  • relatively quick whiteboard (not compiler) coding

This strength turned out to be extremely valuable on Wall St consulting market.

My key weakness identified in recent jobs (not counting the IV defeats):

  • learning local system
  • large existing codebase – AutoReo, GS
  • superfast GTD

theoretical complexity: a G5 personal strength

Some peers (such as Lu) are good at climbing the corporate ladder. Someone must play the leadership roles in those MNC Asia offices, so if Lu has a strong track record he would get it. In Asia, I feel left behind on the slow track – painful, disappointing. In the US, immigrant techies are less likely to climb up (except some Indians). I often feel an inner competitive strength due to those competences that my peers are usually lacking, such as
– Threading,
– algos and data structures
– SQL hacks
– Perl/linux hacks
– math
– (non-trivial) jargon

These are among my top 3 competitive strengths. My sweet spot domains are typically

  • Complex
  • theoretically, and academically demanding, at least for the uninitiated
  • entry barrier
  • High value and high impact on the project
  • portable, unlike Quartz knowhow

In the US I have such opportunities, though nothing guaranteed.

I can qualify as tech lead on___(system)

Q: what kinda systems can you be an architect or lead developer? See also ##a few projects technically too tough4me

A: In an optimistic mood …

domain firm tech used cfd personal advantage
trade execution b2b sql, tibrv  4
blotter + backend #essential to any citi sql  3
mkt data parser ICE socket  3
bond repricing/risk engine reo sql, async  4 bond math, curve-based pricer
billing, commission PWM SQL, batch  5
STP none sql, batch, MOM  3 SQL
java web … PWM sql, batch  5 many
web … Catcha, NIE lamp, javascript  5 many

I can get these systems to work, although

  • I’m not the fastest
  • i may not use latest, most efficient technology
  • reliability and stability ?
    • Look at GMDS .. MMF is an unstable base technology
    • LOok at Gauardian ssh? not a mature , well adopted base technology

tech specialization: no opp2play%%strength ]sg

In the US I can slowly steer my tech specialist career into my chosen strategic domains of specialization like
– threading
– cpp
– financial analytics
– big data
– statistical analysis
– risk analytics

In many of these domains, most of my peers struggle while I am comfortable…. more confidence, more leisure, more free time, less stress ..

In Singapore, I don’t have the luxury because there are Simply too few jobs at my salary level. Those specialized jobs there are, are simply too selective too demanding

— OC: wanted to go deep into drv pricing..

–stirt: wanted to go into curve building and real time risk but ..

–Mac: wanted to go into quant dev …

killer talent2compensate4many shortcomings

The scope is professional life and financial security.

update — to keep the killer sharp, you need sustained focus and continuous improvement.

eg: my dad is a top researcher in his fields. His research papers are heavy weight and top quality. Someone like him may have many limited skills on many other aspects (tech) but that one killer talent compensates for everything. Crucially, his research output has a financial reward. Otherwise it’s hard to sustain the effort.

eg: sales dragons are simply good at hitting sales numbers. A sales dragon may not understand the products, or remain loyal to the firm, or take care of the clients… Still this is a killer talent to keep her successful, that (kind of) compensates for everything.

eg: I know many IT professionals who are sub-standard technically, but can survive or even thrive in a job for a long time because they keep the boss happy — Killer talent… However, compared to other killer talents this one isn’t as weather-proof as other killer talents. The next boss may not be so easy to please. The company may go under…

eg: at least in my first 10 years of my career, I always could get my jobs done. I was always technically up to the job. This talent did compensates for everything.

eg: On Wall St, my job hunting skill is a killer talent. This is one of the most effective kill talents known to me.

eg: Some people manage to amass a small fortune and invest successfully, to get a decent return (like $5k/m). I guess Anthony Lin, Lian Zhong, Chun Tih .. all had multiple properties.

Within finance, technology outlives most job functions

Look at these job functions —

* Many analysts in finance need to learn data analytics software ….
* Risk managers depend on large risk systems…
* Quants need non-trivial coding skill…
* Everyone in finance needs Excel, databases, and … financial data.
…. while the IT department faces no threat, except outsourcing. Why?

Surely … Financial data volume is growing
Surely … Automation reduces human error, enforces control — operational risk…
Computer capabilities are improving
Financial data quality is improving
Computers are good at data processing, esp. repetitive, multi-step…
Financial info tech is important and valuable (no need to explain), not simple, requires talent, training, experience and a big team. Not really blue-collar.

Many techies point out the organizational inefficiencies and suggest there’s no need for so many techies, but comparatively is there a need for so many analysts, or so many risk managers, or so many accountants or so many traders? Every role is dispensable! Global population is growing and getting better educated, so educated workforce must work.

#1@what I enjoy: body-build`

(Avichal asked me …)

#1 of what I enjoy most – body-building. There are various elements that make it challenging, engaging, sustainable

* some value-add for the tech community

* relevant to immediate IV -and- value-add-for-employer, which translates to stable and hopefully rising income…
Example – high performance c++ on linux

* unlocking additional job markets
Example – c++, c#, swing, socket
Example – FX domain knowledge
Example – quant stuff

* Real insight gained only by in-depth study, overcoming a non-trivial entry barrier, and building a lead over competitors
Example – quant trading strategy
Example – threading, STL,
Example – insights into c#
Example – stat risk content, even though not really relevant to my interviews.

* strategic value to my competitive position
counter-example: secDB

tech zbs( !!GTD) outweigh other(soft)job skills #%%belief

label: big20

In many teams (including …), I feel technical zbs/GTD capabilities

are still the #1 determinant of job security, comfort level,

competence level, work-life balance. Secondary yet important factors

include

– relative caliber within the team

– boss relationship

– criticality of the role within the team, within the firm

– reliable funding of the project, team or role

– closeness-to-money

– long work hours

– fire-n-hire culture

So zbs is key to keeping a job, but for finding a job, something else needed.

tech jobs | new territories acquired

If we are too honest, we could portray ourselves as a junior java developer — only 2009 – 2012. In fact many java veterans stop learning after the first 3 years. Domain knowledge? Only bond pricing/trade capture + vol fitter – mostly non-mainstream domains in the Asia context, since bond and option markets are really mature and developed only in the US.

That would be the situation if we were too honest. However, right now I feel confident to claim many other territories thanks to my “ECA” learning.

* FX – I described several FX projects and I feel confident talking about them in details
* eq
* vol – I studied most of these things on my own before the Barclays interview
* threading — obvious
* swing – I took up some swing projects in Citi and did some personal projects while in BofA. Then I learnt key tricks from Guillaume and practiced over a few days. I know my swing CV and swing interviews work.
* FIX?
* C# — I blogged about many topics. I passed many first-round interviews. I know my c# CV (and c# interviews) works!
* c++ — similar and even more interviews passed.

This is one of the most powerful, deep and indelible personal experiences to shape my perceptions, priorities, motivations,

network server perf improved, real illustration#Guardian

Here’s the infrastructure. Exactly one microagent is installed on each machine to be monitored. An “environment” is defined by a base name on a machine. Strictly 1:M between micro agents and environments. Suppose we have 2 micro agents (on 2 machines) and 3 environments under each, so 6 distinct environments in total. A command like “dir” or “ipconfig” can execute in one of the 6 environments such as Environment #1. We can also run the same command on Environment #2, Environment #3, #4, #5, or all 6 environments. Another command “path” can also hit any environments.

If we single out one microagent, one enviroment under it, and run one command against it, the command output would be the status of one “service”. So a service is identified by a tuple of 3 things – a particular microagent, a particular environment, and a particular command. If we have 2 microagents, 3 environments under each, and 4 commands, then we could have up to 24 services. I use many different terms to refer to a service.

Sometimes I call it a query. You keep firing the same query to get updates from the microagent.
Sometimes I call it a chat room. All clients registered for that CR would get all updates.
Sometimes I call it a message generator.

For each service, GUI clients would continuously monitor its status. Client connects to server to subscribe to updates. Server maintains about 100 “services” like chat rooms, and each one generates messages once a few seconds. Server would push them to the registered clients, using WCF.

In terms of topology, just one server instance in the network, at least 3 microagent-enabled app server machine, and many, many client machines.

—-
Trick: Server doesn’t know when one of the registered clients is offline so I often notice it sending updates to 13 client when only 1 or 0 client is alive. I created a dictionary of connections using their IP address as key, so we won’t have 2 duplicate clients to update, since one of them must be dead.

Trick: many msg generators (“services” or “chat rooms”) share the standard update interval of 60 seconds. Each is driven by a private timer. The timers start at server start time, but I decided to use different initial delays. Therefore one generator fires on the 1st sec of every minute, another generator would fire on the 2nd sec every minute. Spread out the load on all parties.

Trick: when a microagent is offline, the central server would keep hitting it as per schedule (like every 5 sec) driven by the timer. Expensive because the thread must block until timeout. I decided to reduce the timer frequency when a microagent is seen offline. Restored after microagent becomes reachable again.

Trick: some queries on the microagent take a long time (20 sec). Before first query completes, 2nd query in the series could hit the same microagent, overloading both sides. I decided to set a busy flag on each query, so next time a thread from the thread pool “wants” to fire the query, it would see the flag and simply return, without blocking.

am fast at GTD, as Wall St developer

* I grew fast enough for GS. Compared to other banks, GS has 50% of the headcount for any given workload.
* I was fast enough for the Lab project lead. Their standard is even higher than GS. The Lab project lead didn’t complain about my delivery speed though he pointed out Piroz was more experienced and faster.
* Citi senior mgr didn’t complain about my delivery speed at all. Not even once. I was working at a comfortable pace.
* I worked alongside Lab49 consultants who are battle tested fast-coders over many years.
* in every job i managed to steal time for self-study – swing, python, secDB, rv, options, c++, non-blocking I/O, c++ debugger, bond math…
* many trading desk dev teams are elite teams with tiny headcounts. I survived some.
* I generally pay more attention to details than other developers. I could, if I want, switch off this attention.

I guess there are super-developers who are really faster in some specific contexts, but overall it’s hard to be a fair judge considering quality, prod support capability, error checking, knowledge-transfer, automated-testing, help to other team members, client-orientation, maintenance cost, system flexibility, readability, architecture soundness…

%%broad GTD valueAdd cf3Y-younger java programmer

 (Written in 2009)
– i managed difficult teams and difficult customers for years
– real AR experience with perl
– 5 years of Unix system admin experience
– i took part in AR decisions of many projects. 1st-hand AR experience
– I dealt with more diverse challenges of trouble-shooting and problem-solving
– In small projects, I was usually the customer-facing single point of
contact, not working behind the interface person. i was the only one
to negotiate with customer.
– Over the same 3 years, i dealt with many small customers rather than
1 or 2 big customers
– since i took part in more prj over the same 3 years, i witnessed
more failed prj
– i witnessed 1st-hand the “decline” of many technologies — rare xp

learning multiple comp languages — interference@@ slowing down@@

XR,

You once mentioned that some developers feel frustrated that learning too many languages together can be slower than learning one at a time, because the language differences are confusing, and can mess up the mental structure[1] of a single language, in our head. When we start, we may know 200 features of C++. By the end, we have a shaken confidence in 30 features because we no longer remember if Feature A is for C++ or C#, and we may mistakenly assume Feature B (belonging to C#) is also in C++.

I seldom have such a problem. Every time I discover a language difference about a bunch of features, those features become quite hard to forget.

For me, the key is to discover the difference by myself and then blog about it, like I did in http://bigblog.tanbin.com/search/label/c%2B%2B_vs_%20java_vs_c%23.

If a language difference is presented to me in someone's write-up, i often have difficulty internalizing it. I often reach a deeper or higher level of understanding, then write about my own findings. Then, this difference is no longer spoon-fed to me, but metamorphose into my own discovery.

Such a process takes effort each time, so I don't do this every time. I feel learning multiple languages inevitably takes effort, if we want to compete with others in the “knowledge” contest. No pain, no gain.

[1] Speaking of the =:structure:=, all well-designed languages have a ton of nitty-gritty features (200++ pages) organized into a clean structure built on a small number of key principles. Without such a structure, a language is hard to learn and won't catch on. In fact, this is a criticism on perl. I know perl the language has structure and strong principles, but perl appears less structured than most languages.

%%tech strength/weak among Sr trad’dev candd ]NY

^ threading?
^ java overall?
^ c++ overall?
^ generic algorithms and data structures
^ generic OO
^ SQL?
STL
^ Unix?
Boost
JMS?
servlet/JSP
^ DB tuning?
Spring?
Hibernate?
^ scripting?
RV?
XML
design patterns?

Imagine you are an interviewer comparing candidates “feature-by-feature”. Some areas outweigh others. For example, many interviewers feel fundamental knowledge outweigh experience using 3rd-party tools.

%%tech strengths for fin IT jobs

— ranked
java + c# + c++
perl, unix — still widely used in financial but not many people know in-depth
SQL — outer join, self-join, sub-query and other complex joins, case expressions, group-by. See the t-sql book.

# eor
index design and utilization — banks heavy usage
milestoning pitfalls, limitations — banks heavy usage
refactor
complex data analysis using SQL — banks use it all the time.
query plan

y ibanks insist minimum X years finance job xp

… because they want someone 1) used to the pace and work culture, 2) knowledgeable in the domain. As an (unintended) consequence of this hiring criteria, South Asians (India…) candidates outnumber East Asians (Chinese, Korean…).

I notice adjustment problems in some (perhaps 1 out of 3) newcomers into financial IT. Some of them simply quit financial and joined telecom, google, Time.com, universities, hospitals…

It took me a few years — and counting — to get used to the culture.  Quick and dirty — occasionally even code cloning; eliminate purely theoretical scenarios and focus on realistic scenarios; tolerate minor bugs and ship faster; no documentation…. ultimately, focus on business value, that we are paid to deliver.

Financial Knowledge helps too. Last month I sent you a list of some 30 financial terms — take months to grasp.  But even if you know none of the jargon, ibanks still consider you if they think this candidate is battle tested on wall street.

As a concrete illustration of domain knowledge, remember the question you and I discussed repeatedly 2 years ago — how a brokerage house matches a buy order and a sell order and executes it profitably? I now understand it. Besides the jargon, this is example of another type of domain knowledge. Does it help a finance app developer? Valuable but not a must-have knowledge. However for an architect, a Business Analyst, or any manager (project manager, team lead, lead developer), such knowledge could be make-or-break.

me as a good team player

see also [[ rare qualities ] developers ]]
see also [[ strengths.xls ]]

Background: wall street values team players.

#1) fairness and an objective mind is kind of rare but extremely important
* helping new comers
* helping initially slow learners
* helping older colleagues
* feeling and showing gratitude
* documentation designed for teammates
* taking initiative for knowledge sharing. my emails
* cross-cultural
* open-communications with due respect to confidentiality. not “selective sharing”
* defending the vulnerable and victims
* owning up to mistakes, in public
* no air of superiority. Approachable
* no favoritism
* no personal attack

real stories illustrat`%%debugging expertise — focus on java

see other stories in j,ora.xls

DNS reload
freebsd NIC driver
custom classloader breaking EJB casting
weblogic upgrade max connection exceeded — ulimit for the shell,
max_tcp_connections, file descriptors

jdbc failures — snoop

weblogic jms XA conn factory; disable catch clause to trigger rollback

2 (custom) class loaders loading the same class — cast exception

nbc firefox and ie gets different dates from the same url. we reduced
the code to the simplest. then i managed to get a
Runtime.getRuntime().exec( “ls /tmp” );

log4j appenders in assignment are from 2 classloaders, but “murderer”
(classloader bug) looks like an accomplice — log4j errors are usually
harmless

feel good despite insensitivity ]office

) Not a moral issue, even when you hurt someone’s feelings.
) xp: less of a problem in a cross-cultural team. Look at 2006 and later
) Hopefully Americans are more direct
) much less insensitive in writing.
) saving grace — pure heart, fairness, equality, integrity
) saving grace — am sensitive to most essential needs of a corporate worker, universal themes, well-understood
) habit @ reflection, which has caught countless “mismatched” and lost signals between colleagues and me
) constantly learning and paying conscious attention to this area
) This is one of the areas I’m likely to get better, not worse. Once you trip over a stumbling block, you will spot it next time.

real&&rare value-add to developers@weblogic

Some developers have 3-5 yeras experience on weblogic but offer no valud-add relative to “thinking” develoeprs coming from, say, websphere backgrounds. Value-add ranked in terms of value:

* 1) insight into weblogic trace logs (=> tuning, TR). Each app server’s developers have their preference what to log. Perhaps a neglected aspect of J2EE standardization.
* key limitations of weblogic vs websphere, jboss. There are probably dozens of limitations but real veterans know a few serious ones
* weblogic server tuning
* monitoring (=> tuning, TR)
* weblogic cluster
* JRockit Runtime Analyzer and esp. mem leak detection
* weblogic support for web services — different from other vendors

other value-add without ranking:

* weblogic domain
* weblogic deployment tools
* weblogic JNDI idiosyncrasy
* AR
* dev^production mode
* when not to use weblogic jms
* when not to use weblogic conn pool
* idiosyncrasies of weblogic thread pool

%%value-add as a 5-year batch veteran

Why do employers ask for 5 years experience in batch development? Here are the most important value-add of a real veteran, based on my first-hand observation

(See also %% posts on batch wishlist.)

1) robust and resilient. My experience shows that serious batch jobs can fail for a large number of reasons such as unexpected input or network delays

2) Flexibility for change. I think batch apps are seen as quick-and-dirty, and flexible. People ask for more changes cos they assume *cost* of change is lower for batch apps than non-batch apps. Such expectations call for deep experience in batch design.

2A) extensibility, which is slightly different from “flexibility”. Example: adding parallelism, retry.. If not well designed, you often need to throw out old tested codebase and restart from scratch.

) modularization for a development team. Minimize stepping on each other’s toes.

) readability, ease of learning. Batch jobs are often seen as temporary, so documentation and design are lower priorities in batch than non-batch. Many batch applications actually need hand-over and maintenance by a new guy. I think a good system design can ease documentation, learning and knowledge-transfer.

#
* fine-grained control. Consider the monitoring features of JMX and Weblogic
* testability
* performance optimization experience

%%innovation track record

ranked

– [[ multiple inheritance]FTTP module ]]
– snoop oracle client-server conversations to see the actual query and raw data returned
– separate biggies and text files in my personal folder, to minimize its size and ease backup and portability
– always type my passwords on websites, to help me memorize a new zipcode or an often-misspelt word
– dozens of transparent bags hung on pegs, to help find various stuff in my home right after move-in
– since mail2blogger has 5% downtime, i created a googlegroup to publish to multiple blog hosts to compare their uptime
– dd: go vertical, freeing floor space, more stable than tall bookshelf
– typing trainers in append.bashrc.txt
– paste on mirrors torn-out pages of dictionary to help my wife’s english
– daily check on library loans (web scraping)
– put yellow stickers on light-weight luggage for mom
– bookshelf in bathroom
– automated remote login, delete acc.

rare non-tech qualities in developers

An informal list of attractive characteristics in developers. Rare and non-trivial qualities.

— top 10
* willing to take up unpopular or unattractive tasks like …
* documentation — willing, thorough
* unit test — habit of creating and maintaining his own unit tests
* miminize personal favoritism, esp. as a leader. Impossible to be 100%
* unbiased — against race/gender, languages, technologies
* willing to listen && think hard about opposing views when he need not
* communicate in the language of management and users. Many feel this is rare; many disagree.
* technical pdl && “vision” ie long term.
* “Composite capability” is dependant on many factors.

— not so rare
* cooperative — follow agreed but tedious team standards
* Think like the business owners. Many feel this is are; many disagree.
* patience — with novice colleagues
* accurate estimation. set realistic targets to help supervisor’s planning
* no entrenched positions on …
* willing to abandon his own design when appropriate
* publications and open source contributions
* thorough — attention to details …
* articulate — with technical discussions
* quick assessment, even though imperfect
* lifelong learning — fearless
* self-examination before checking others; lead by example. (Experienced developers have many dislikes.)
* ownership — of own code from beginning to end
* flesixible — with OT,
* troubleshooting — systematic, reasoning, Sherlock Holmes. Very few jobs need exceptional troubleshooting; usually a competent technie will do.

* fast learner — some jobs need unusually fast learners. Perhaps #2 asset. Another “Composite skill”
* brank
* proven leadership@@ I think you can be a pretty good leader if you score well on this list. U need not be a motivator. Fairness is much more important