fear@large codebase #web/script coders

One Conclusion — my c++ /mileage/ made me a slightly more confident, and slightly more competent programmer, having “been there; done that”, but see the big Question 1 below.

— Historical view

For half my career I avoided enterprise technologies like java/c++/c#/SQL/storedProc/Corba/sockets… and favored light-weight technologies like web apps and scripting languages. I suspect that many young programmers also feel the same way — no need to struggle with the older, harder technologies.

Until GS, I was scared of the technical jargon, complexities, low-level API’s debuggers/linkers/IDE, compiler errors and opaque failures in java/SQL … (even more scared of C and Windows). Scared of the larger, more verbose codebases in these languages (cf the small php/perl/javascript programs)… so scared that I had no appetite to study these languages.

— many guys are unused to large codebases

Look around your office. Many developers have at most a single (rarely two) project involving a large codebase. Large like 50k to 100k lines of code excluding comments.

I feel the devops/RTB/DBA or BA/PM roles within dev teams don’t require the individual to take on those large codebases. Since it’s no fun, time-consuming and possibly impenetrable, few of them would take it on. In other words, most people who try would give up sooner or later. Searching in a large codebase is perhaps their first challenge. Even figuring out a variable’s actual type can be a challenge in a compiled language.

Compiling can be a challenge esp. with C/c++, given the more complex tool chain, as Stroustrup told me.

Tracing code flow is a common complexity across languages but worse in compiled languages.

In my experience, perl,php,py,javascript codebases are usually small like pets. When they grow to big creatures they are daunting and formidable just like compiled language projects. Some personal experiences —
* Qz? Not a python codebase at all
* pwm comm perl codebase? I would STILL say codebase would be bigger if using a compiled language

Many young male/female coders are not committed to large scale dev as a long-term career, so they probably don’t like this kinda tough, boring task.

— on a new level

  • Analogy — if you have not run marathons you would be afraid of it.
  • Analogy — if you have not coached a child on big exams you would be afraid of it.

I feel web (or batch) app developers often lack the “hardcore” experience described above. They operate at a higher level, cleaner and simpler. Note Java is cleaner than c++. In fact I feel weaker as java programmer compared to a c++ programmer.

Q1: I have successfully mastered a few sizable codebases in C++, java, c#. So how many more successful experiences do I need to feel competent?
A: ….?

Virtually every codebase feels too big at some time during the first 1-2 years, often when I am in a low mood, despite the fact that in my experience, I was competent with many of these large codebases.
I think Ashish, Andrew Yap etc were able to operate well with limited understanding.
I now see the whole experience as a grueling marathon. Tough for every runner, but I tend to start the race assuming I’m the weakest — impostor syndrome.
Everyone has to rely on log and primitive code browsing tools. Any special tools are usually marginal value. With java, live debugger is the most promising tool but still limited pain-relief. Virtually all of my fellow developers face exactly the same challenges so we all have to guess. I mean Yang, Piroz, Sundip, Shubin, … virtually all of them, even the original authors of the codebase. Even after spending 10Y with a codebase, we could face opaque issues. However, these peers are more confident against ambiguity.
Advertisements

[19] 3Deeper mtv2work4more$ after basic ffree

Note sometimes I feel my current ffree is so basic it’s not real ffree at all. At other times I feel it is real, albeit basic, ffree. After achieving my basic ffree, here are 3 deeper motivations for working hard for even more money:

  • am still seeking a suitable job for Phase B. Something like a light-duty, semi-retirement job providing plenty of free time (mostly for self-learning, blogging, helping kids). This goal qualifies as a $-motivation because … with more financial resources, I can afford to take some desirable Phase-B jobs at lower pay. In fact, I did try this route in my 2019 SG job search.
  • I wish to spend more days with grandparents — need more unpaid leaves, or work in BJ home
  • more respect, from colleagues and from myself
  • stay relevant for 25 years. For next 10 years, I still want more upstream yet churn-resistant tech skills like c++.

–Below are some motivations not so “deep”

  • better home location (not size) — clean streets; shorter commute; reasonable school.. Eg Bayonne, JC

— Below are some secondary $-motivations

  • * more time with kids? Outside top 10 motivations.
  • * better (healthy) food? usually I can find cheaper alternatives
  • * workout classes? Usually not expensive

G5 valuable features@2019job

== G3 attitudes that make this hibernation better than qSG phase

  1. not fixated on inferiority. At age 38 I was still feeling at-peak and hence … struggling against (and missing) the expectations repeatedly (boxer taking a brutal beating). Now at age 45 my self-expectations match reality way better
  2. mlp job — not aiming to tackle tough tech domains like c++ or quant. Boss expectation is easier than Macq and Qz. so far no PIP risk
  3. not aiming to “upgrade” my son’s benchmarks

In hindsight, during qSG I grossly underestimated the impact of PIP, damaged goods, stigma, …

== G5 valuable features@MLP

  1. Mgr in NY.. no micro-managing; Colleagues frequently MIA, showing ..
  2. Commute – able to camp out over weekends, or weekday evenings for that matter
  3. Instrumentation … slightly higher self-sufficiency. Independent xx (big gun) will happen quicker.
  4. respectable salary, not one of those 120k jobs. Remember peer-comparison is one of two reasons for first-aid.

Note some of the most valuable features are the absence of (or freedom from) “smells”

— beyond G5

  1. mkt depth — as a key factor to job satisfaction, as explained in the job_satisfaction spreadsheet. Mainstream, churn-resistant domain, similar to mkt data
  2. Extra leaves — not needed really but valuable
  3. not a job primarily using Qz, java1.4, or c#, or plain old C, or javascript/go-lang
  4. flexible or remote working
  5. Med-bx — a purely monetary benefit, more valuable to my peers + family
  6. not a system seldom used, or used by very few people
  7. not a code base that’s never put into production
  8. —— trivialities :
  9. Low-cost fruit shop nearby
  10. No prod emails so far

dev-till-70: 7external inputs

Context — professional (high or low end) programmer career till my 70’s. The #1 derailer is not physical health [3] but my eventual decline of “brain power” including …?

[3] CSY and Jenny Lu don’t seem to agree.

This discussion is kinda vague, and my own thoughts are likely limited in scope, not systematic. Therefore, external inputs are extremely useful. I posed the same questions to multiple friends

Q2: what can I do now given my dev-till-70 plan defined above.
Q1: how do I keep my brain healthy, and avoid harmful stress?

— Josh felt that the harmful stress in his job was worse in his junior years when he didn’t know the “big picture”. Now he feels much better because he knows the full context. I said “You are confident you can hold your end of the log. Earlier you didn’t know if you were good enough.”

— Grandpa gave the Marx example — in between intense research and writing, Marx would solve math problems to relax the brain. I said “I switch between algo problem solving and QQ knowledge”

— Alex V of MS — Ask yourself
Q: compare to the young grads, what job function, what problems can you handle better? My mental picture of myself competing against the young guys is biased against my (valuable) battlefield experience. Such experience is discounted to almost $zero in that mental picture!

When I told Alex my plan to earn a living as a programmer till 70, Alex felt I definitely need a technical specialization. Without it, you have very little hope competing with people 40 years younger. I said I intend to remain a generalist. Alex gave some examples of skills younger people may not have the opportunity to learn.

  • low-latency c++
  • c++ memory mgmt
  • specific product knowledge
  • — I said
  • sockets
  • .. I have a few skillist blogposts related to this

— Sudhir
Mental gymnastics is good, like board games and coding practice and Marx’s math practice, but all of these are all secondary to (hold your breath) … physical workout, including aerobic and strength training! I personally enjoy outdoor exercise more than anything else.

Also important is sleep. I think CSDoctor and grandpa are affected.

Sudhir hinted that lack of time affects sleep, workout and personal learning.

  • Me: I see physical exercise and sleep as fundamental “protections” of my brain. You also pointed out when we reach home we often feel exhausted. I wonder if a shorter commute would help create more time for sleep/workout and self-study. If yes, then is commute is a brain-health factor?
  • Sudhir: Absolutely, shorter commutes are always better, even if that means we can only afford smaller accommodation. Or look for a position that allows working remotely some of the time.

Sudhir also felt (due to current negative experience) an encouraging team environment is crucial to brain health. He said mental stress is necessary, but fear is harmful. I responded  “Startup might be better”.

–Jenny Lu felt by far the most important factor is consistent physical exercise to maintain vitality. She felt this is more important than mental exercise.

I said it is hard to maintain consistency. She replied that it is doable and necessary.

–Junli…. Felt mental exercise and physical exercise are both important.

When I asked him what I can do to support dev-till-70, he identified several demand-side factors —

  • He mentioned 3 mega-trends — cloud; container; micro-service.
    • Serverless is a cloud feature.
  • He singled out Spring framework as a technology “relevant till our retirement time”

— CSY pointed out the risk of bone injury.

He said a major bone injury in old age can lead to immobility and the start of a series of declines in many body parts.

— XR’s demand-oriented answer is simple– keep interviewing. He felt this is the single most effective thing I can do for dev-till-70.

##very few theoretical compSci constructs 4 CIV

Most comp science constructs are too advanced too complicated for a 45-minute coding interview. So reading any comp science book is like my son reading science books not written for his exams. What a candidate need is drills targeted at interviews.” I told friends, based on Leetcode.

A few exceptional interviews (eg: Google) go beyond Leetcode, but still use only simple features of advanced comp science constructs. Here are A few notable comp science constructs to study, mostly advanced data structures

  • [s] trie, suffix array, suffix tree
  • geometry (is dStruct-heavy domain) —
    • nearest neighbor query;
    • query: which (x,y) points fall within this rectangle?
    • line sweep
  • segment tree
  • topological sort – 2 algos
  • [s] disjoint set
  • relationships among in|pre|post-order binTree walks — these insights are valuable for some Leetcode problems.
  • self-balancing algo in RBTree
  • [s] B-tree
  • [s] shortest path algos like BFT
  • [s=only the simple ones are needed in CIV]
  • —- techniques
  • technique: augmented tree
  • technique: back tracking
  • technique: DP and greedy
  • technique bottom-up DP with memoization

Q: How about coding interview books? Can help. They are not as deep or wide ranging as algorithm books.

Q: how about bigO on common data structures? Yes relevant because of the interviewer’s insistence.

 

##[15]types@Work2slow brain aging #campout

This is an important topic for the researchers. As laymen, I will just reflect on my personal experience — looki.

What kind of activity is considered “work”? I feel it’s the responsibility or commitment, obligation, consequences on users -> fear …

Most but not all the “work” is tiring. I suppose creative work can help keep the brain young.

  • IT — Learning a new tech in body-building mode doesn’t feel tiring to me, but how about to XR?
  • IT — A big chunk of everyday IT work (including troubleshooting) is partly creative and investigative. Can be brain-boosting.
  • Kids doing homework – can be tiring but at their age it won’t speed brain aging. How about adults?
— Avoid stressful, demanding jobs that require burning my candle on both ends.

Need to listen to the inside signals when working the long hours. When I work day and night on coding assignments, I was like my Dad and perhaps CSDoctor. I was driven by joy and burning pleasure, not pressure like in Stirt and GS. I feel my body knows if the pressure is positive or negative, pleasure or pain, so better listen to the signals and avoid aggravating the aging process.

Sudhir told me about encouraging environment vs fear… See blogpost on 5 external inputs

## IV skills: disaster-rescue 逃生

I have relied on this parachute over and over again. Therefore, I have real motivation to strengthen this parachute and keep it in good condition.

  • OC — parachute saved me from transfer to BAU
  • 2007 after first entry to U.S., parachute saved me from sitting on bench for months as happened to … Florence
  • Vz — after my contract was cut unexpectedly, parachute saved me from sitting on bench. This proves to be the first of many rescues including 95G, Barclays,
  • [B] Stirt — after the layoff, parachute helped me avoid sitting on bench for months.
  • [B] Macq — after the PIP, parachute saved me from another slow job search on SG job market
  • [B] deMunk — parachute gave me confidence that I had the technical capabilities to escape — My “parachute” was my marketable skillset
  • [B=example of boxer cornered n beaten]

Other people’s examples:

  • [B] Youwei — his java IV skills rescued him from the MS layoff
  • Venkat of OC — it rescued him from a terrible boss
  • Shanyou and Deepak — need better parachutes
  • Jack Zhang
  • Davis Wei

%% absorbency: experiment/SDI imt speed-coding

When you find yourself in high-absorbency mood, favor (#1 most draining) speed-coding. See list of less-draining drills at the bottom.

I can read dry QQ topics for hours each day for many days, but tend to lose steam after coding for a few hours. Speed coding drill drains my “laser” energy faster than QQ reading/blogging/experiment. When my laser is weakened (by boredom), I must drill harder to go through the “brick walls”.

I guess many fellow programmers enjoy coding more than reading. I feel lucky that in my interviews, knowledge tests still outweigh coding test.

Q: What part of coding drill is worst on my absorbency?

A: speed coding implementation is worst. It drains my laser energy fastest. After coding for a few hours I always feel like a deflated balloon and discharged battery and and need a full day to recharge.

I think frustration is the key. Self-expectation (about progress and traction) and self-image create the frustration and the drain.

Instead of traction, I often feel stuck and overspent.

I feel disappointed with myself (Deepak self-identify as “annoyed”. )

Q: Can we stop comparing with others and just compare with our past? Doable sometimes. Consider Leetcode speed-coding contest #Rahul

— Just like yoga

  • if I work on easy problems I feel wasting my time
  • if I work on tough problems I feel painful, draining and want to give up. After the practice i need hours to recover.

Q: … So can we find easier coding drills that I could enjoy (as Rahul suggested)? Definitely not easy. I think the first difficult step is self-acceptance that I can’t improve much at this age.

Q (excellent question): What type of “coding” drill can I do for hours like reading/blogging?

  • pseudo-code algo on paper/whiteboard is lighter. No ECT so I am swift and efficient. Less draining/frustrating.
  • SDI is most fun, least boring, not draining/frustrating. I can spend hours on a SDI. I feel a bit of accu. More like QQ less like coding drill.
  • concurrency coding questions are less draining as other guys are not faster
  • c++/java language feature QQ experiments are more like QQ. I can spend hours on a QQ experiment. More interesting as there’s no time-line no benchmark no frustration. Also other guys are not stronger. I feel some accu exactly like reading on these me features
  • review of my previous code is much less draining (than writing new solutions) as there’s no time-line and code is already working
  • analyzing patterns and reusable techniques (very few) in past problems. Thick->thin is the holy grail. I work hard towards it.
  • reading syntax and ECT tips in books and my blog

 

[16] Fwd: techies’ common complaints about jobs

We complain about high churn, but why the hell none of us go teach math?

We complain low $ROTI but how many percent of techies get any $ROTI from personal investment or self-learning?

We complain about low (if any) lasting social value in our work, but why the hell none of us chooses an RnD career?

Hi friends,

Most techies (including developers) probably feel undervalued, and have a lot of potential not utilized on the current job.

We blame our company or our team or our job. Maybe it’s not challenging enough; maybe too repetitive; maybe too niche.

We look up at some high flyer and ask “what if I’m given that role… I may not do better than that person, but surely I will be competent and up to the job.  It may be boring and stressful but Hey I will earn so much more!”

In many over-staffed IT departments, about 20% of the roles are critical and some 20% of the roles are dedicated to “peripheral”systems that no business users care about. Perhaps that system is lightly used, and users don’t trust the output anyway.……

Well, my current (devops) job gives me a lot of opportunities to push myself higher. It’s not too niche (like Quartz/Athena/SecDB). It offers complexity and depth. Not mindless and repetitive. Not something I feel already too familiar with (and jaded). I can see the impact quickly. The impact is on many people. The impact is on front office.

Still I’m not fired-up. I guess there are always better roles out there.

We had better condition our mind not to think that way. Instead make the best use of the current role. “When life gives you lemons, make lemonade”

[19]c++guys becom`very unlucky cf java

On 22 Apr 2019 I told Greg that c++ developers like me, Deepak, CSY.. are just so unlucky — most of the WallSt c++ jobs are too demanding in terms of latency engineering, either on buy-side or sell-side.

Greg agreed that java interviews are much easier to pass. Greg said if you have reasonable java skills, then you can get a job in a week.

I told Greg that the only way Deepak or CSY could get an offer is through one of the few easy-entry c++jobs, but there are relatively few such jobs i.e. without a high entry barrier.

— widespread view that c++ developers are perceived as strongest due to c++ being a hard language

My conclusion

  1. yes in terms of QQ and zbs
  2. no in terms of GTD, as GTD challenge is mostly due to localSys. Even a python codebase can be hard.

##specializations fac`same IV quizzes 20Y on#socket !! c++11

(tagline: the most churn-resistant specializations.)

Socket IV questions have remained unchanged for 20Y — Unmatched stability and churn-resistance, but not necessarily accumulating

  • Preservation of t-investment
  • Preservation of accumulation
  • Preservation of deep learning? Socket programming has modest depth.

Q: Beside the specialization of socket programming, are there other specializations that are dominated by the same old QQ questions 20 years on?

  • [S] classic data structures
  • [S] classic sort/search algorithms on int-array, char-array, list ..
  • [S] classic traversal algorithms on trees, general graphs
  • [s] classic recursive, DP, greedy algorithms beyond graphs
  • [S] pre-c++0x core-C++ (a specialization!) questions are largely unchanged. C++11 questions are rooted in the old knowledge base.. BUT most of the c++11 QQ topics will likely fall out of interview fashion
  • [s] cross-language concurrency primitives.
  • unix file/process/string/signal manipulation
  • unix shell scripting — low market value
  • [S] SQL — including join, index design … but seldom quizzed in depth nowadays
  • [S] regex — seldom quizzed, but often needed in coding
  • [S=classic, well-defined specialization]
  • [s=slightly less well-defined specialization]

Now the disqualified skills

  1. JGC + jvm tuning — high churn over 20Y
  2. TMP — new features introduced in c++11

## strategic TSN among%%abandoned

Q: name the most strategic trySomethingNew domains that I have since abandoned, given up, quit. How about the new plan to take on coding drills as a lifelong hobby?

Don’t spend too much time, because the answers are nothing new even though this is a decent question.

  1. — ranked by surprise
  2. algo trading? actually very few roles spread across a number of firms
  3. c#
  4. drv pricing quant
  5. real time risk as in GS, Qz and Athena
  6. RDBMS tuning
  7. MOM, async, message-driven design knowhow
  8. distributed cache like Coherence and Gemfire
  9. Solaris sys admin and DBA
  10. perl, SQL, Unix/Linux power-user knowledge? No longer a top 10 focus

—-

  • python? Not yet abandoned
  • web dev for dotcom? I did this for years then abandoned it. Many of the tech skills are still relevant like sessions, java thread safety, long-running jobs

c++QQ critical-mass[def2] has started growing!

If I compare myself with young c++ developers or older guys (like CSY, Paul..) I can see clear patterns in QQ and GTD.

  1. QQ and (to a lesser extent) zbs — I’m clearly pulling ahead, sometimes heads and shoulders above them. Some of these guys have wider QQ knowledge but lacks depth. I wrote about “experts” sizing up each other..
  2. zbs: instrumentation — one of the key areas of improvement for me! However, I can see many of the older guys at RTS aren’t more knowledgeable.
  3. GTD: paradoxically, the younger guys are more productive than me or older guys.

— Now let’s retrace the gradual breakthrough

In May 2019, I felt I have achieved enough critical-mass on c++ QQ topics. Critical mass is defined by The two acid test questions.

Q1: without a full-time c++ job, but with enough interviews, will my c++ QQ insight/understanding show resilience against churn and memory fading, as in coreJava?
Q2: thick->thin achieved? Not yet, but cross-reference graph is now built up as a defense against fading memory

This java career review provides a valuable context.

What visible progress gave me this level confidence? Recent technical wins show my improved ranking among c++candidates.

  1. SCB-FM
  2. CVA
  3. SIG
  4. TradeWeb core team

Note I have invested more effort on c++ QQ than java… [18]t-investment: c++now surpassing java

  • — now a sample of critical-mass topics, roughly ranked by importance on high-end interviews, mostly at HFT and ibanks
  • coding tests
  • mv-semantics
  • containers
  • smart ptr
  • heap memory mgmt including new..
  • polymorphism including MI #44 posts in the category
  • TMP — important at high-end but not HFT
  • [e] pthreads + c++11 threads
  • [e] sockets
  • runtime costs of virtual and heap, as Stroustrup explained
  • [e] cache efficiency, compiler optimizations,
  • [e] linux
  • [e] new innovation directions
  • [e] benchmarks involving c++ as Stroustrup explained
  • build tools
  • [e=ecosystem topics]

[18] CONTINUOUS coding drill #Shi

My friend CSY said some students like his son could conceivably focus “all their time” on one skill (coding drill) for fours years in college, so they will “surely” outperform.

I pointed out that I am often seen as such an individual, but my speed coding interview performance is hardly improving.

I pointed at the number of Leetcode problems I solved with all tests passed. It grew by up to 10 each year, but a student can solve 10 leetcode problems in half a day.

I gave an analogy of my Macq manager’s weekly slow-jogging, for health not for performance. Consistent jogging like that is great for health. Health is more important than athletic performance. I said jogging and my coding drill are life-style hobbies and recreations.

For years I practiced continuous self-learning on

  • java, c++, SQL, MOM, swing — IV and GTD growth
  • Unix, python — mostly GTD
  • quant

I consider my continuous self-learning a key competitive advantage, and an important part of my absorbency capacity.

I asked a bright young Chinese grad ChengShi. He practiced real coding for months in college. I said “fast and accurate” is the goal and he agreed. A few years into his full time job, he stopped practicing the same. Result? He said for the same coding problem he was able to make it work in 10 min then, but now probably 20 minutes. I think this is typical of the student candidates.

I asked “Do you know anyone who keeps up the coding drill?” He didn’t tell me any individual but gave a few points

  • he believed the only reason for coding drill is job hunting
  • when I said continuous practice all year round will make a big difference compared to practice only when job hunting, he agreed wholeheartedly and reiterated the same observation/belief
  • but he disagrees that continuous coding drill would lead to significant professional growth as a programmer, so he would probably channel his spare energy elsewhere.
  • I think he has other ideas of significant growth. At my age, I don’t foresee any “significant growth”.

G3 survival capabilities #health;burn rate

Out of the subconscious realm, I hand-picked a couple of personal “qualities” [2] as the (three or five) pillars of my life [1], the most important pillars since 2007. 2007 is the /watershed year/.

These are the pillars for my past, present and my future, immigration plan, family housing, retirement plan, … There’s possibly nothing new here, but this blogpost provides another perspective into the “making” of my career and the foundation of my entire life. As such, this is not a worthless reflection. Still, avoid over-thinking as there are other worthwhile reflections.

I think everyone can consider the same question — “Using words that are as specific as you can find, name 2 or more personal survival capabilities, hopefully the most important ones.”

  • AA) IV prowess — (since Chartered). Compare CSY’s experience.
    • self-renewal to stay relevant — lifelong learning. Am rather successful so far
    • my reflective blogging is actually part of the my career way-finding, motivation…
    • absorbency — of dry, theoretical, low-level domains. /continuous/ plow-back without exhaustion
    • theoretical complexity — aptitude and absorbency
    • lower-level — overall, i’m stronger than my peers at low-level
  • BB) my capacity to keep a well-paying dev job (not as “unique” as the other 2 capabilities). Even in the three Singapore cases, I was able to hold it for about 2 years, or receive a transfer or severance.
    • figure-things-out speed?  Not really my strength but my competence
    • camp-out, extra hours
    • attention to details
    • getting the big picture?
  • [G3] personal burn-rate — important to my feeling of long term security.
    • See the Davis chat on old-timers.
  • — The following factors are less “unique” but I want to give credit to
  • [G3] my healthy lifestyle and bread-earning longevity — in the long run this factor would prove increasingly important. It enables me to keep working long past retirement age.
  • [G5] my bold investment style [2]
  • my education and self-learning capabilities including English skills
  • [G3 = a top-3/Group3 factor]

! benchmarking — AA is a single-winner competition, whereas BB is about staying above minimum standard in the team.

! strengthening — I continue to plow back and build my AA/BB 内力, focusing on localSys + coding drill. A huge amount of energy, absorbency, continuous effort and dedication needed (cf. XR,YH, CSY…), though since 2018 I have noticed the ROTI is not as high as before 2012.

[1] to some extent, my family’s life also rests on these same pillars, but I tend to under-estimate the capabilities of my family members and over-estimate my earning power.
[2] in this post I don’t want to include my financial assets since I want to focus on personal qualities

## UPSTREAM tech domains: defined by eg

It pays to specialize in a domain that helps you break into similar domains. (Unsuccessful with my quant adventure so far.)

Note “wrappers” API are never upstream. Very common in big banks including Qz.

  • #1 eg: [L] socket —- C++ is the alpha wolf, leader of the wolf pack
  • [D] market data —- Nyse/Nasdaq (binary) feed parer seems to be the leader, over the slower FIX feeds
  • [D] execution algo —- sell side equities desk seems to be the leader in terms of volume and variety of algos
  • data analytics language —- python looks like the leader
  • eg: [D] alpha —- equities trading is the leader
  • eg: threading —- java is the leader of the pack. In c++ threading is too hard, so once I master some important details in java threading, it helps me build zbs in c/c#, but only to some extent
  • eg: regex/string —- Perl is the leader
  • eg: [D] consolidated firm-wide risk —- SecDB is the leader. Applicable to buy-side? I think so.
  • lambda —- C# is the leader. Scripting languages follow a different pattern!
  • list/stream —- Linq is the leader
  • drv pricing theory —- Black-Scholes —- feels like a leader or upstream, but not /convincing/
  • [L] heap/stack —- C++ is the leader. Java and c# are cleaner, simpler and hides the complexities.
  • defensive coding —- (and crash management) C++ is the leader, because c++ hits more errors, like Japan on earthquake management.
  • eg: collections —- C++ is the leader of the pack. There’s a lot of low level details you gain in STL that help you understand the java/c#/python collections
  • eg: [L] latency —- C++ is the leader of the pack in low-level optimizations .. In-line, cache-efficiency, footprint..
  • eg: [L] pbref^val —- C++ is the leader. C# has limited complexity and java has no complxity
  • [L = lowLevel]
  • [D= well-defined application domain]
  • ? automated pricing —- no upstream
  • ? Fixed income trading systems —- no upstream. IRS^Treasury^credit bonds (trading platforms) have limited features in common.
  • [D] VaR/risk analytics —- FI derivatives in big banks seem to be the leader, but I feel one trading desk in one firm may not respect another system

##150k@light GTD-load: which FTE

Real deciding factor is coworker benchmark (not PIP/stigma). Are there managers tolerant of team members below the benchmark? Josh, Srini of Citi-muni..? Even in a less demanding company, pressure can be high.

  • — Here are some jobs paying 150k with light GTD-load. Usually don’t attract young bright guys.
  • employer:  slower ibanks like Citi, UBS
  • employer:  some commercial banks like OC, BONY
  • employer:  large traditional buy-side like AIG, Vanguard
  • employer:  3rd type financial firms like exchanges/ECNs, data vendors (Reuters?), financial product vendors,
  • employer:  non-finance like telcos
  • employer:  startups but they tend to use new technologies
  • less glamorous — like mkt data, back office
  • greenfield codebase with short history —  like OC, StirtRisk
  • smaller codebase — like RTS
  • older workforce — like RTS
  • older technologies — like SQL, C, socket

algo-IV^QQ expertise

Background — became expert via external QQ benchmark` !!localSys xx

Q2: how are algo quizzes different from the C++ language expertise tested in job inerviews?
%%A2: I feel this algo expertise is loosely defined — mostly defined by a few hundred “fashionable and common algo interview questions.”

Ironically, the algorithm researchers and inventors may not ace these west-coast algo interviews 😉 Also, the non-trivial algorithms in textbooks are invariably too complicated for a 45-min coding interview.

There’s no deep insight or innovation in this game. This type of “expertise” doesn’t solve tough real-world problems. They can only solve simple real-world problems.

Top performers in this game are possibly students, exam-taking machines. They won’t feel proud of themselves as compared to c++ language expert or algorithm inventor.

Q2b: how are speed coding different?
%%A: in addition to A2, speed coding requires a mastery of a tiny subset of the language features and mastery of many coding idioms based thereon.

9 income sacrifices: often2avoid stagnation

Update — relevant for my hibernation decision.

I demonstrated a habit of giving up higher income, in order to keep learning something of strategic value ( or managing mgr expectation )

  • before it’s “too late” — i.e. before the opportunity disappears
  • The prospect of blood-letting (wasted cycles) is too painful.
  • I felt my peers were moving ahead of me, while I stood still.
%sacrifice higher income forgone lower income to gain … ROI realized intangible ROI strategic?
2002 50% Zed ->SCS presales none brave
2003 40% SCS ->self-employ entre none brave
2004 70% Sperion ->self-employ entre none brave
2007 10% est. UBS/Headstrong GS branding unsure
20012 7% Barc #Swing 150k picked OC 100% c#, quant dev learned enough to crack some c#IV confidence no:(
2015 10% est. sign-on increment Macq lower mgr expectation year1 bonus as a self-image boost #short-lived no:(
2017 10%+ Pimco C++/FI offer 100+/H c2c picked RTS 100% c++ more c++ offers yes  [1]
2017 10% java jobs picked RTS c++ more c++ offers ensued yes 🙂
2018 10% java jobs picked mvea c++ c++ job market cracked c++ skill is harder, rarer conviction
2018 15% CVA picked mvea low-latency high-volume eq trading big OMS framework job satisfaction yes 🙂 [3]
—-  —- —————
2013 not a job [2] UChicago formal math training Master’s degree, branded uni confidence in math; self-confidence no:(

[1] In 2017, I felt my c++ had not reached the critical mass despite my long self-study. A power drill was required to break the stone wall. The RTS full time job turned out to be that power drill.
[2] For the UChicago motivation, there’s a separate blog post.
[3] no regret taking this lower offer!

productivity@past jobs: questionable patterns

Be careful with the assessment — I may not be so productive on my past US projects
Be careful with the assessment — I may not be so un-productive on the recent projects

Pattern – Barc, ML … were java, my turf
… but Citi was java as well. My java skill did help, but I didn’t make use of my strongest java skills.
… but at GS, Perl was my turf too. My Perl experience did help but the colleagues were too strong.

Pattern – Barc, ML … not many peers to compare with me
… but Chien was a direct peer. I think Rao also.
… but later stage GS, I was benchmarked

Pattern – OC, … familiar with the codebase
… but OC codebase was rather big.
… but Stirt codebase wasn’t big.

Pattern – GS, OC … spent long time on the existing codebase. In contrast, Creating code from scratch is easier. … but Stirt was code from scratch

##4+2 games I Aced,visibly #MSFM

I /aced / killed / thrived at /many games. Most visible and most profitable game in this list is tech IV, including

  • 1a) branching out to c# and c++
  • 1b) quant self-study to impress many technology interviewers

The above topic already has many many posts in this blog. Below are other games I excelled in:

  1. excellent grades up to college Year 1
  2. paid off multiple rental properties + my own home, by age 43. All in good locations with reliable rental demand.
  3. Earned MSFM with flying colors at age 42 — sustained focus, self mastery

Some domains are not really competitive “games” but still I excelled visibly:

  1. no belly (as Nick pointed out); weight loss in late 2018, along with pull-up.
  2. keeping burn rate very low, and achieving some form of ffree around age 30 and again at 43

It’s instructive to recognize the pattern.

  • I think in each game, I had some talent, and a lot of consistent effort.
  • External positive feedback is far from powerful , immediate or frequent, so internal motivation is crucial.
  • All individual games, not team games. Note promotion is not my game and I don’t need to kill this game to be comfortable and satisfied.

##how fast I figurED things out relative2teammates

GS — Initially I was much slower than my colleagues. After 18 months I was 80% as fast as them. Everyone needed to learn and memorize lots of details in a big system. Without that “mileage” you can’t be as fast as the rest.

I relied on my colleagues to give me pointers and detailed explanations. Otherwise it was too many details to wade through by one self.

Citi — no comparison — No “team mates” per-se. I had two old-timer colleagues who were very familiar with the system. I was not benchmarked against them.

I didn’t need to understand the ins and outs of the 5-year old codebase. I never became productive with it.

95G — not slower — i joined this greenfield project from beginning (luckily). I was as fast as the other consultant. Also, I designed a few key components, so I had the “original designer” advantage.

Barclays — not slower — i joined this greenfield project from very beginning (luckily), replacing a half-finished c# system. I had very few “team peers” to benchmark against. I designed one of the most valuable and complex components, so I had the “original designer” advantage. Huge advantage. I basically name my price when it came to effort estimates.

I did read some old code to replicate the logic, but I was fairly fast, and no team member ever read the same codebase anyway.

OCBC — not slower — I was seen as slower, but actually I became 70% to 90% as fast as the other team members. I had to spend many long hours on the existing codebase but it’s a small codebase (1 man-year) so I was able to get a hang of it over a few months.

I also designed some minor modules, where I had the “original designer” advantage. The other c# developers had to come to me and understand my designs.

Stirt — I was much slower than the old-timers, but I was not slower than the new joiners. I should have given up earlier. It was hopeless. Codebase was rather opaque and you need to ask many people to understand it.

Other new joiners seemed to ask only within their own team, but I stick out my neck and asked all around my team. It may have made me look slower and /greener/, but I was trying to learn faster. Did I learn faster than them? Not enough time to show.

Macquarie — The benchmark was my predecessor + my boss. I think I was much slower than them. My boss only complained about it in the 2nd year, months after paying me a small bonus.

RTS — not slower — I’m probably faster than some team members. I feel confident how fast I can figure things out now. There are many team members to compare, so from Day 1 it was crucial to hit the average standard.

%%3 GTD concerns as lead dev #instru/code reading..

Below are my biggest concerns when considering a lead dev role. Actually, many of my peers often hints “My role (as a mgr, or architect, or app owner..) is even harder” but I think they are Not more effective if asked to play the hands-on developer role.

  1. code reading — requires focus and stamina.
    • In addition, taking ownership of a big code base requires determination and confidence
  2. instrumentation — using prints or special tools, troubleshooting, error reproducing, ..
  3. pressure due to major feature not completed by impending deadlines

Note none of these is tested in any interview!

  • How about firefighting? I tend to panic but my peers aren’t stronger than me technically!
  • How about design? Not really my weakness. My peers’ designs are not necessarily better than mine
  • knowledge of generic tech (rather than local system knowledge)? I’m actually stronger than most peers
  • quality, attention to details? is usually my strength
  • fine tuning? Most peers are not good at this
(not ranked) bonus peers are .. generic or local nlg?
code reading no impact some are stronger  local sys nlg
investigate boss needs answers usually no diff  local
deadline yes ?  local
firefighting yes not stronger  local
design somewhat visible not stronger  some generic

##9dataStruct(+..)for c++speed cod`

  1. linked node manipulation in a graph context
  2. vector (more useful than array), std::string (more useful than cStr). Many string operations are still unfamiliar
    1. Array-based data structures are required in 80% of my coding tests.
    2. More than 50% of all my coding tests require nothing but arrays.
    3. Most of my toughest coding tests are presented in arrays but may need maps as helpers
  3. string in std::string or c-str
  4. iterator manipulation like erase, lower_bound, find,
  5. sorting, operator<(), upper_bound, binary search … on containers
  6. sorted data structure like std::map
  7. [w] stringstream — ECT to improve

Very few Wall St interviewers would test you on graph or DP. Here are other less important constructs:

  1. [w] binary tree is common and simple, but they can ask very tough questions on it!
  2. [w] double pointer is trickier and my advantage
  3. [w] Node class in a linked data structure.
  4. [w] stack, queue.
  5. [w] grid or matrix
  6. file I/O? only for IDE tests, not pure algo or take-home tests
  7. [w] basic syntax for pointer arithmetic.
  8. [w] dtor, copier, op=? only for design questions, not algo questions.
  9. [w] shared_ptr? Only for design questions, never needed in algo questions!
  10. [w] ref variable only as function I/O.
  11. stl algo? Only Citadel array-shrink
  12. never exception
  13. never template
  14. no (very, very seldom) threading in coding Q
  15. adv: pointer to function
  16. adv: circular buffer
  17. [w = no weakness]

 

c++4beatFronts: which1to LetGo if I must pick1

Q: Which one to let go, given I have limited O2/laser/bandwidth and mental capacity?

  1. give up BP — biggest room for improvement but least hope
  2. give up codility style — Just get other friends to help me. See codility: ignore overflow, bigO

How about pure algo?

  • already decent? Can improve.
  • diminishing return? Smaller room for improvement? but I can learn a few key ideas about the G100 questions

##(IncInc)most effective 10Y direction to INCrease INCome

Reality! We better embrace reality, not ignore it.

  • new skills like py, hadoop, data science? I see low chance of increasing my income, but I could be wrong
  • some specialist role, perhaps
    • risk/pricing analytics? questionable premium
    • low-latency algo trading in c++/java? unlikely … Get Real
  • hands-on architect (different from specialist role)
  • (perhaps maintenance mode) app owner, and grow in a big company?
    • must be a high value application
  • high-end contractor (probably@ibanks)? most practical
  • coding drill? practical 🙂
  • portable instrumentation zbs
  • [p] portable GTD skills (eg instrumentation)
  • [p] improve localSys learning process to speed up get-over-the-hump
  • [m] algo practice
  • socket QQ/GTD? not directly increasing my income but prevent income decline
  • [m] low level QQ topics in ARM, linux, STL, concurrency
  • [m=muscle building for iv]
  • [p=can help me take on a high-paying lead dev role. If too tough, then we can still count on these items to help improve our stress profile; self-esteem; job security; bonus… ]

+! trySomethingNew] sg, what could I have(possibly)got

See also past vindicative specializations

  • I would still do my MSFM
  • I would still fail to get into algo trading or quant dev — too few jobs and extremely high bar
  • I would likely fail to get into leadership roles. I was considered for leadership roles at 1 to 3 companies

However,

  • I could possibly have focused on a specialization such as risk system + some analytics
  • would probably have joined citi, barc, baml, UBS, SC or .. in sg
  • probably java or swing or connectivity
  • would Not have achieved the c#/py/c++ ZBS growth
  • would Not have the skills to get this ICE raw mkt data job or the other c++ job offers.
  • no guarantee to become a manager or app owner. There could be many old timers in the team.
  • possibly less stress and pain. Lower chance of performance stress (#1 biggest stressor), because my GTD/KPI would be higher due to my java/SQL zbs.

[17]orgro^unconnecteDiversify: tech xx ROTI

Update — Is the xx fortified with job IV success? Yes to some extent.

Background – my learning capacity is NOT unlimited. In terms of QQ and ZZ (see post on tough topics with low leverage), many technical subjects require substantial amount of /laser energy/, not a few weeks of cram — remember FIX, tibrv and focus+engagement2dive into a tech topic#Ashish. With limited resources, we have to economize and plan long term with vision, instead of shooting in all directions.

Actually, at the time, c#+java was a common combination, and FIX, tibrv … were all considered orgro to some extent.

Example – my time spent on XAML now looks not organic growth, so the effort is likely wasted. So is Swing…

Similarly, I always keep a distance from the new web stuff — spring, javascript, mobile apps, cloud, big data …

However, on the other extreme, staying in my familiar zone of java/SQL/perl/Linux is not strategic. I feel stagnant and left behind by those who branch out (see https://bintanvictor.wordpress.com/2017/02/22/skill-deependiversifystack-up/). More seriously, I feel my GTD capabilities are possibly reducing as I age, so I feel a need to find new “cheese station”.

My Initial learning curves were steeper and exciting — cpp, c#, SQL.

Since 2008, this has felt like a fundamental balancing act in my career.

Unlike most of my peers, I enjoy (rather than hate) learning new things. My learning capacity is 7/10 or 8/10 but I don’t enjoy staying in one area too long.

How about data science? I feel it’s kind of organic based on my pricing knowledge and math training. Also it could become a research/teaching career.

I have a habit of “touch and go”. Perhaps more appropriately, “touch, deep dive and go”. I deep dived on 10 to 20 topic and decided to move on: (ranked by significance)

  • sockets
  • linux kernel
  • classic algorithms for IV #2D/recur
  • py/perl
  • bond math, forex
  • black Scholes and option dnlg
  • pthreads
  • VisualStudio
  • FIX
  • c#, WCF
  • Excel, VBA
  • xaml
  • swing
  • in-mem DB #gemfire
  • ION
  • functional programming
  • java threading and java core language
  • SQL joins and tuning, stored proc

Following such a habit I could spread out too thin.

##xp@career diversification

  • biz wing — in addition to my tech wing. I learned a bit but not enough. Not strategic
  • quant? diversify. The on-the-job learning was effective and helped me with subsequent interviews, but further push (UChicago) are not bearing fruits
  • data science? diversify
  • big data java jobs? stack-up
  • —-tsn hits good mkt depth, or hits poor leverage
  • 😦 quant dev, real time risk
  • 😦 hft
  • 🙂 py
  • 🙂 mkt data
  • 🙂 socket
  • 🙂 c#
  • 🙂 windows dev
  • —-tsn within the tech space, where I have proven strengths
  • py? bearing fruits. Confidence.
  • unix -> web dev -> java? extremely successful
  • [10 Citi] c++? slowly turning positive
  • [12 Barc] swing? positive experience
  • [12] dotnet? reasonable
  • [14] real time risk, in-house framework (Quartz)? disillusioned
  • [17] raw mktData
  • [18] high-volume, low-latency equity Order Management
  • [18] FIX? diversify

20 C++territories for QnA IV

NO coding interviews — The scope here is the QQ topics i.e. tough IV topics not needed in GTD.

[1] “+” means Interviewers ask tons of question in this domain, more than it deserves. Its weight in the total score is lower than the share of questions would indicate.
[2] “70%/core” means this IV topic is 70% theoretical; core c++. The more theoretical, the more you need to read and summarize, rather than “GTD by any means”
[O] means HFT style optimization-heavy interviews. Not needed in regular jobs

swap-related tricks? advanced and seldom quizzed

  unsorted topics theoretical Q weight  share of Q [1]  
T1 big 4 70% / core  9%  +
T1 vptr-related: dynamic cast,
interface classes, pure virtual
100% / core  9%  +
T1 virtual dtor 100% / core  2%  +
T1 inheritance: MI, PI, hiding,
overload^override, casting, protection
60% / core  5%
T2 ptr as field; return ptr 20% / core  N.A.
no tier OO low level design #swap tricks,
class hierarchy..
60% / add-on 5%  – hard to ask
T1 stack,heap,static …variables 50% / core  3%
T1 pbclone, pbref, ref params 50% / core  4%
T1 object allocation, life time
vs variable declaration
0 /core N.A.
tier 2 STL: containers, 20% / add-on  14%  – asked more in coding IV
tier 2 array 0 / core N.A.
T2 malloc; heap prohibit; leak prevention/detection 40% / core 7%  –
T1 const 90% / core  2%
T3 [O] DAM allocation efficiency 100% / core 1%
T2 threads 80% / add-on 8%  –
T1 smart ptr and variations 30%  6%
no tier c++11: move, rvr 90% / core  4%
T2[O] sockets: a lot of c++jobs I applied
happen to need it
0 / add-on 10%
T3[O] cpu cache optimization 100% / core 1%
T2[O] inline and code-size bloat 100% / core 1%
T2[O] field layout 90% 1%
T2 c^c++ 90% / core 2%
T2 static/shared library  50% / core 2%
—  minor topics  —  4%
boost besides smart ptr 10% / add-on
container of polymorphic type #interface 50%
templates 80%
singleton, factory 100%
pimpl 90%
other design patterns 100%
tier 2 exception safety 90% / core
T1 RAII 50% / core  –

## meaningful endeavor(Now)for20Y career#dnlg;zbs;IV

Q: what efforts go towards 20Y-career building?
A: instrumentation in c++(java is a bit more churn); socket programming; STL internals; JVM internals; shell scripting; pthreads; FIX; linux internal?

This question is slightly vague, but it’s subtly different from

Q2: what efforts create social value over 20Y?
Q3: what efforts are cumulative over 20Y?

I feel the key question in this write-up is about building a deep and broad “foundation”, upon which we can start to plan (possible) social value, retirement planning, parenting/family building etc.

  • 😦 teaching and research —- Somehow I tend to feel research creates long-term value but I suspect most research efforts are not valuable!
  • 😦 open-source software —- I tend to feel OSS projects create social value, but most of them are not successful and have no lasting influence
  • 😦 Deepening your domain knowledge such as (German’s) security lending —- It does increase your value-add to some employers and help you move up to management, but I don’t call it “foundation”.
    • MSFM and other formal training —- I feel this is less effective, more ivory-tower, but it does build a firm theoretical foundation.
  • fitness and health .. more important than I feel (as shown by my action)
  • zbs such as instrumentation skills? Yes zbs helps your KPI, and also indirectly helps you with interviews. It’s portable skill, unlike local system knowledge. Can the value last 20Y? Depends on the technology churn. c++ and java are safer.
  • IV muscle building? My favorite career building effort. 20Y? 10Y yes.
  • English/Chinese writing and vocab

##tough n high-leverage c++topics#IV QQ

I used to feel I have so much learning(absorption) capacity, but now I feel in my finite career I can’t really master and remember all the tough c++ topics.

Practical solution — Classify each difficult c++topic into one of

  1. QQ: high impact on QnA interview, probably the only type of high-leverage tough topic. Largely textbook knowledge. As such I’m basically confident I can learn all the basics on my own (perhaps slower than Venkat), provided I know the topics.
    1. including “coding questions” designed really for knowledge test, rather than algorithm thinking
    2. eg: HFT, Venkat…
  2. high impact on algo coding IV? No such topic. These coding IV are not about knowledge in tough topics!
  3. ZZ: high impact on GTD zbs — inevitably Low leverage during job hunt
  4. 00: no high impact on anything

Q: Is there a tough topic in both QQ and ZZ? I don’t know any.

I think 00 will be the biggest category:

  • [00] template wizardry;
  • [00] template specialization
  • [00] MI;
  • [00] operator overloading;
  • [00] pthread
  • ————-
  • [QQ]
  • [QQ] move semantics
  • [QQ] [p] boost common lib
  • [QQ] optimization tricks. Remember MIAX and SCB IV by Dmitry
  • [QQ] [p] singleton implementation — not really tough
  • [QQ] pimpl — not really tough
  • [QQ] op-new/malloc (interacting with ctor)
  • [QQ] memory layout
  • [QQ] [p] struct alignment
  • [QQ] specific threading constructs
  • [QQ] smart ptr details
  • [QQ] ptr as a field
  • [QQ] implement auto_ptr or ref-counting string
  • [QQ] [p] UDP —
  • [QQ] [p] multicast
  • [QQ] select()
  • [QQ]
  • [ZZ] IDE set-up
  • [ZZ] compiler/linker/makefile details
  • [ZZ] debuggers
  • [ZZ] crash analysis, like segmentation fault
  • [ZZ] c^c++:SAME debugging/tracing/instrumentation skills #ZZ

[p=paper tiger. ]

##respect,$,familyTime,spareTime..all benefit from强项

If I take a 强项 job, I kind of sacrifice my muscle building and TrySomethingNew, so I better get good money …, but now I feel I don’t have to.

A typical 强项 job would use (the most complete list)

  • math, analytics
  • high volume data processing
  • unix wizardry
  • heavy text processing
  • heavy SQL
  • heavy scripting
  • some http programming
  • data analysis, perhaps using SQL, scripting etc
  • some combo of java/c++/c#/swing
  • threading

[17] j^c++^c# churn/stability…

This comparison has a bias towards java. My observation is finance-centric.

Q: why do I feel c# as a t-investment is not as long-living as java and C (the longest-living)?
A: Java is not tied to any OS. Java is used on windows + many unix derivatives including linux and android, whereas c# and objective-C are tied to particular platforms.

However look at Perl. It is cross-platform but was displaced by vbscript on windows and python.

 notes worst – best factor
Microsoft doesn’t care about this c# jav c++ protection of Your investment
c# jav c++ churn
c++ has no single owner making those decisions #worst to best c# jav c++ stability of features
#younger to older c# jav c++ longevity(new tech tends to die young
or changing too much)
JVM is huge help c++ c# jav ease of troubleshooting
c++ c# jav maintainability of app
c# can be back-end but less proven;
c++ can crash easily — no exception to catch
c# c++ jav underlying stability as
long-running server
windows is murky. Java: type erasure c++ c# jav  dark corners undocumented
#high to low c++ c# jav syntax complexity
#worst to cleanest c++ c# jav clean language
#low to high jav c++ c# expressiveness
c++ much higher #low to high c++ entry barrier at senior level
c# can be ez to learn, but java is even cleaner  #easy to hard java c# c++ initial learning
only Microsoft provides #low to high c# c++ jav std+3rdParty libraries
c# — only on windows c++ c# jav popularity
c# — limited to Windows and GUI/web c# c++ jav wide appeal, general-purpose
each lang has x% high-end jobs but c++
percentage (30%) is highest
c# salary
c++ c# java job market depth

[15] a G3 WallSt survival KPI:how fast U figure things out, relative2coworkers

As stated in other posts, it’s all about local system knowledge + tools

Survival (and bonus, stress..) is always linked to benchmark against team peers, not external teams like another department, not the manager himself. If entire team is low-calibre, then it’s a problem for the manager, but rarely your own problem. (It might threaten your team’s survival…)

LG2: system performance. Most guys in the team probably aren’t so good at this kinda thing, unless the  team has a specific mandate to monitor and fix performance.
LG2: test coverage. May affect your reputation, but once I figure things out I can often relax and test
LG2: code smell. May affect your reputation but is clearly a LGlp compared to getting things to work.
LG2: code quality
LG2: readability by the team lead. OK team lead may complain, but if it works then it’s relatively easy to improve it.
LG2: extensibility. Yang is big on this. Many books are big on this, but it’s an art not a science. Many team members probably aren’t good at it.
LG2:  system stability such as occasional hangs. Often a non-show-stopper.
** eg: my VS2015 build tends to hang and I had to research on how to fix it — show-stopper.

All of the above are secondary to … “figuring things out” i.e. how to get something to work, at a minimum usability.

Design? See also posts on arch_job. Can be a real problem, because if your design gets shot down, you waste time on rework.

 

 

IV^GTD – grow as techie@WS

I want to grow stronger/broader/versatile as a survivor, not necessarily grow my value-add. Actually I don’t have to grow.
— IV skills – Compared to GTD skills, these skills give me more confidence, more protection, more stress-relief. It works in big job markets like Wall St.

Lots of theory, which is my sweet spot.
Selective on what IV topics to learn!
coding IV + algo — favored by SiV
— (portable) GTD skills
Lots of tools….
Selective on what GTD topics to learn!

Needed for a lead developer, but such a role is stressful. I guess some good lead developers are also good at IV, but I’m not sure and I assume I’ll not be good at both.

Warning – a lot of projects don’t provide real GTD enrichment. Eg: Quartz, tibrv wrappers, Gemfire wrappers, FIX wrappers
——-
Macquarie environment lets me learn lots of GTD skills.
OC gave me IV (c#) and GTD enrichment.
Stirt – none!

A java environment would give me some additional GTD enrichment but less IV enrichment

In SG, I continued my previous strategy, learning IV skills + GTD skills. Not effective so far. I feel my c# IV skills improved a lot but still not there. C++ IV skills didn’t improve much partly due to distractions.

 

##[15] once-valuable tech skills: personal xp

perl – lost to python
[$] tomcat, jboss. Weblogic lost most market share
apache, mysql
dns, unix network config
autosys – not used after GS
[$] sybase, oracle – after ML edge project I didn’t need it again.
[$] VBA – used once only. But Excel is going strong!

[$ = still has value]

–random list (not “top 10”) of longevity skills
eclipse, MSVS
Excel and add-in
STL
javascript, though I don’t use it in my recent jobs
python
Linux shell
vi
compiler knowledge
make, msbuild
bash and DOS batch scripting, even though powershell, vbscript and python/perl are much richer.

##[15]where in c++am I growing: GTD^IV

Short and sharp answers preferred
–Q: where and how fast am I growing in c++ IV?
A: EPI300/Google style coding test, esp with STL, shared_ptr etc
A: the HFT interviews revealed my weaknesses in some QQ areas.
A: ##10 basic programm`constructs for timed c++ cod`IV  — focus, direction for IV preperation
–Q: where and how fast am I growing in c++ zbs
A: debugging using print or gdb. No longer afraid of funny errors
A: experience reading STL errors
A: [[safe c++]]
A: [[21st century C]]
A: Learned to question what low level details are more than academic
A: Learned the practical value of c++ integration with excel, python etc
A: Learned a bit more about instrumentation including debuggers, esp. on Windows
A: Learned a bit more about boost smart pointers. There are many of them, because there exists different needs.
A: Learned a bit more about IDE, make, cmake and friends

#1@what I enjoy: body-build`

(Avichal asked me …)

#1 of what I enjoy most – body-building. There are various elements that make it challenging, engaging, sustainable

* some value-add for the tech community

* relevant to immediate IV -and- value-add-for-employer, which translates to stable and hopefully rising income…
Example – high performance c++ on linux

* unlocking additional job markets
Example – c++, c#, swing, socket
Example – FX domain knowledge
Example – quant stuff

* Real insight gained only by in-depth study, overcoming a non-trivial entry barrier, and building a lead over competitors
Example – quant trading strategy
Example – threading, STL,
Example – insights into c#
Example – stat risk content, even though not really relevant to my interviews.

* strategic value to my competitive position
counter-example: secDB

2 focuses when learning a new tech skill: IV^GTD

When we learn a new tech skill, we need very, very specific goals and guidelines to help us choose what NOT to dig into too early.

Focus #1: IV topics, a well-defined focus. Basically textbook knowledge. Go deep enough but not too deep on a particular subtopic. Examples — op-new, pure virtual, const ref param of copier, recursive locks, OOM calling new,

Focus #2: GTD i.e. dev productivity — as measured by managers, bench-marked against other developers. Incidentally, perceived productivity sometimes depends so heavily on local system knowledge that a new hire always lags behind no matter what. I feel productivity and GTD has low dependency on #1, as most of the textbook knowledge is irrelevant.

Focus #2b: tools — see post on 2 categories of tools.

Focus #2c? tips on StackOverflow. No such books. No way to

Focus #?: best practices — or Acceptable practices and well-trodden paths

Focus #?: learn all the syntax details to read any source code on books, interviews and online

Focus #: high-level design decisions, for technical leadership roles. Myriads of decisions are made every hour during development, some at high some at low levels. Apparently, a lot of these decisions involve theoretical understanding.

plowback for zbs(+GTD)@@

For many years I was deeply convinced and motivated by the “Shanyou” notion that I ought to “plowback” for zbs and, to a lesser extent, GTD, not just aim at IV as the Mithuns would.

After working for 20 years, I now believe ANY tech knowledge, accu, deepening/stack-up, sustained focus … has extremely low leverage and basically worthless if not relevant to IV

  • GTD skills? dominated by localSys. Tool knowledge can help but localSys is 10x more important.
    • localSys xx is almost always cursory, rather than in-depth since unnecessary digging (pwm-comm bottom-up xx plan) is invariably unsustainable — The local systems are patchworks , destined for rewrite, illustrating few best practices
  • BestPractice design? Almost never mattered in my career. After EMPWorld, I have never risen to decision-maker level. This zbs is overrated by 10 times.
  • BestPractice implementation? What’s “best” is mostly a matter of taste (personal preference) of the manager
  • zbs? Consider the most practical books like [[Pearls]] — the classics. If not relevant to IV then this zbs is only for self-satisfaction

This is one reason why my reinforcement loop completely broke down in Singapore.

… In contrast, my NY years were filled with joys of self improvement, driven by … interviews.

arch xp is overrated as KPI #instrumentation, tools…

See also other blog posts —

// “in the initial dev stage, instrumentation is my #1 design goal”.
// http://bigblog.tanbin.com/2010/09/2-important-categories-of-software.html

Going out on a limb, I’d say that in high-pace projects GTD (Getting Things Done) is more important than many things which are supposed to be more important.

GTD is more important than … integrating deep domain knowledge insight into the system. I think such domain knowledge can sometimes be valuable. It can help dev team avoid wasting time on “the wrong things”.

GTD is more important than … quality of code. For a few months at least, the only guy looking at the code is the original author.

GTD is more important than … “quality of design”, which is seldom well-defined.

GTD is more important than … being nice to others. Many times other people can tolerate a bit of personality if you can GTD. However, you must not offend the people who matter.

GTD is more important than … impressive presentations that address many real customer pains. A user may die for a guy who “understands my pains”, but when it is time to deliver, selling/understanding is an irrelevant distraction.

GTD is more important than … in-depth knowledge of the language or of a software platform/product. Such knowledge is more important than GTD during interviews though. Threading, data structure…

GTD is more important than … uncovering the root casue of (intermittent) problems/errors/surprises — the Toyota drill-down root-cause investigation. Regex engine creator may need to fully characterise every unexpected behavior; AutomateTellerMachine error probably deserve a drill-down investigation but in enterprise apps drill-down investigation simply takes too much time. We developers are paid to build a usable tool, not a fully-understood tool. Live with ambiguity and move on.

GTD is more important than … adding an important test to the automated test suite. Sure that mistake we just found may happen again so we really appreciate adding that test, but in reality, most high-pace environments don’t have an automated test suite. If we are lucky enough to have documented manual test plan, then yes add it, but such a plan seldom is comprehensive, so after a while people won’t follow it religiously. So in reality we just hope developers learn the lesson and avoid making the same mistake. If they do, then we just need someone who can GTD. Any system to incorporate such heard-learned lesson is likely an imperfect system, and whoever investing in such a system is often wasting his/her time.  If a GTD guy doesn’t bother with that system, he will still be respected and loved by manager and users. Basically, any long-term investment is unappreciated. GTD is all about short-term results. This is the reality of quality control in fast-paced teams.

GTD is more important than … adding meaningful error messages. Anyone debugging the system would love the guy who added the meaningful error message, but he is an unsung hero. Manager love the guy who GTD fast. Code quality is invisible and therefore ignored and unrewarded.

To achieve GTD, you must solve tech problems. Tech problems could (occasionally) call for architectural perspective, but less than tool knowledge, debugging experience, or low-level system insight.

In defense of architects, architectural track record is more important in sales contexts, including selling to internal clients and business users.

I should probably get input from Raja, Xiao An, Yi Hai …

remain relevant in a given technology(fundamentals

Q: At age 55, will you still qualify as a swing developer, or SQL developer, or python developer, or Unix admin?

1) An important factor is the stability of the language and the changing demand on specific skillset in that Language (contrast java vs SQL), but let’s focus on another factor — fundamentals.

2) In every language, there’s a body of fundamental knowledge (“knowledge-pearls”) that are Empowering and Instrumental. They facilitate your learning of “superstructure”. Many hiring managers believe strong fundamentals are the most important skill with a given Language.

I feel low-level knowledge will help you remain /relevant/ over 10 years. Some of the most tricky and frequently quizzed sub-topics are low-level — threading, memory-mgmt, collections, subclass memory layout, casting, RTTI, vptr, c++ big 3, …

3) I feel an appreciation of relative strengths/weaknesses of alternative technologies will help you remain relevant. Fundamental knowledge would help you appreciate them.

4) essential libraries of threading, collections, I/O, networking, … are not strictly part of the fundamentals, but part of mandatory skills anyway.

## lasting achievements as a techie@@

Backdrop: Technological innovation is fast (less so in commercial banking, i was told) so “lasting” means … maybe 5 to 10 years (rarely longer)?? Few achievements meet this criteria.

See also http://bigblog.tanbin.com/2010/09/accu-and-longevity-compare-with-civil.html, http://bigblog.tanbin.com/2011/02/classical-engineering-field.html

– open source — software have real lasting values. I guess you can learn to Read their source code — usually in C.
– [L] contributions to foundation modules such as VM/CLR, compilers, threading library, GC, STL — real lasting values

What skills contribute to lasting value-add to an organization?

– tuning – DB — needed in many big and small systems
– tuning – low latency systems — more rare
– [L] instrumentation i.e. refactor/design a given (complex) system to make it easy to trace and follow
– [L] ? introspection – tend to be rather powerful in many new languages
– ? interpreting bytecode; decompilers
– ? system security; white hat hacker

[L = I feel most of these skills are low-level]

In theory almost everything can be learned by a young guy in a few years provided they get full (rare!) access to all source code and manage to make sense of it all. However, look at how many bright young people become kernel developers even though so much open source operations systems exist.

avoid convenience wrappers;xx low-level APIs #RV,socket

See also high^low-level expertise

Many examples of the same observation — convenient high-level wrappers insulate/shield a developer from the (all-important) underlying API. Consequently, in a hardcore interview he can’t demonstrate his experience and mastery in that particular “real” technology — he only knows how to use the convenience wrappers.

  • example — servlet — in one of my GS trading desks, all web apps were written atop an in-house framework “Panels”, which was invented after servlet but before JSP, so it offers many features similar or in addition to JSP. After years of programming in Panels, you would remember nothing at all about the underlying servlet API, because that is successfully encapsulated. Only the Panels API authors work against the underlying servlet API. Excellent example of encapsulation, standardization, layering and division of labor.

Notice the paradox — guys who wrote those wrappers are considered (by colleagues and job interviewers) as stronger knowledge experts than the users of the wrappers. If you think about it, you (as everyone else) would agree that knowledge of the underlying API is more valuable than GTD skill using the wrappers.

  • example — Gemfire — one of my trading engines was heavy on Gemfire. The first things we created was a bunch of wrapper classes to standardize how to access gemfire. Everyone must use the wrappers.

If you are NOT the developer of those wrappers, then you won’t know much about gemfire API even after 12 months of development. Interviewers will see that you have close to zero Gemfire experience — unfair!

  • example — thread-pool — One of my ML colleagues told me (proudly) that he once wrote his own thread pool. I asked why not use the JDK and he said his home-made solution is simple and flexible — infinitely customizable. That experience definitely gave him confidence to talk about thread pools in job interviews.

If you ask me when a custom thread pool is needed, I can only hypothesize that the JDK pools are general-purpose, not optimized for the fastest systems. Fastest systems are often FPGA or hardware based but too costly. As a middle ground, I’d say you can create your customized threading tools including a thread pool.

  • example — Boost::thread — In C++ threading interviews, I often say I used a popular threading toolkit — Boost::thread. However, interviewers repeatedly asked about how boost::thread authors implement certain features. If you use boost in your project, you won’t bother to find out how those tools are internally implemented. That’s the whole point of using boost. Boost is a cross-platform toolkit (more than a wrapper) and relies on things like win32 threads or pthreads. Many interviewers asked about pthreads. If you use a high-level threading toolkit, you avoid the low-level pthreads details..
  • example — Hibernate — If you rely on hibernate and HQL to generate native SQL queries and manage transaction and identities, then you lose touch with the intricate multitable join, index-selection, transaction and identity column issues. Won’t do you good for interviews.
  • G5 example — RV — one of my tibco RV projects used a simple and nice wrapper package, so all our java classes don’t need to deal with tibrv objects (like transport, subject, event, rvd….). However, interviewers will definitely ask about those tibrv objects and how they interact — non-trivial.
  • example — JMS — in GS, the JMS infrastructure (Tibco EMS) was wrapped under an industrial-strength GS firmwide “service”. No applications are allowed to access EMS directly. As a result, I feel many (experienced) developers don’t know JMS api details. For example, they don’t use onMessage() much and never browse a queue without removing messages from it. We were so busy just to get things done, put out the fires and go home before midnight, so who has time to read what he never needs to know (unless he plans to quit this company)?
  • example — spring jms — I struggled against this tool in a project. I feel it is a thin wrapper over JMS but still obscures a lot of important JMS features. I remember stepping through the code to track down the call that started the underlying JMS server and client sessions, and i found it hidden deeply and not well-documented. If you use spring jms you would still need some JMS api but not in its entirety.
  • example — xml parsing — After many projects, I still prefer SAX and DOM parsers rather than the high-level wrappers. Interviewers love DOM/SAX.
  • G3 example — wait/notify — i used wait/notify many times, but very few developers do the same because most would choose a high-level concurrency toolkit. (As far as I know, all threading toolkits must call wait/notify internally.) As a result, they don’t know wait/notify very well. Strangely, wait/notify are not complicated as the other APIs mentioned earlier but are tricky to 90% of developers.
  • G7 example — FIX — one of my FIX projects uses a rather elaborate wrapper package to hide the complexity of FIX protocol. Interviewers will ask about FIX, but I don’t know any from that project because all FIX details were hidden from me.
  • example — raw arrays — many c++ authors say you should use vector and String (powerful, standard wrappers) to replace the low-level raw arrays. However, in the real world, many legacy apps still use raw arrays. If you avoid raw arrays, you have no skill dealing with them in interviews or projects. Raw arrays integrate tightly with pointers and can be tricky. Half of C complexities stem from pointers.
  • G5 example — DB tuning — when Oracle 10 came out, my Singapore (Strategem) DBA friend told me the toughest DBA job is now easier because oracle 10 features excellent self-tuning. I still doubt that because query tuning is always a developer’s job and the server software has limited insight. Even instance tuning can be very complicated so DBA hands-on knowledge is probably irreplaceable. If you naively believe in Oracle marketing and rely on self-tuning, you will soon find yourself falling behind other DBAs who look hard into tuning issues.
  • G3 example — sockets — i feel almost every place where sockets are used, they are used via wrappers. Boost, ACE, java, c#, python, perl all offer wrappers over sockets. RTS is the exception. However, socket API is critical and a favorite interview topic. We had better go beyond those wrappers and experiment with underlying sockets.

I recently had to describe my network programming experience. I talked about using perl telnet module, which is a wrapper over wrapper over sockets. I feel interviewer was looking for tcp/udp programming experience. But for that system requirement, perl telnet module was the right choice. That right choice is poor choice for knowledge-gain.

The Quoine interviewer Mark was the only hiring manager to appreciate wrappers more than underlying socket knowledge. He is the exception that proves the rule.

##[09] (career)safety net for %%family

See also [[#halo]], [[# towards an app architect]], [[http://bigblog.tanbin.com/2006/11/u-cant-be-developer-for-ever.html ]]

For a techie like me, what long term factors help to create a cushion or safety net?

#1) citizenship
#2) Zofia’s professional credentials
#) brank — titles in well-known companies
#) health and insurance
* home ownership + rental income
* English language skill for wife and me
* trec DBA
* trec presales
* trec PM
* post graduate degree -> teach