demanding mental power: localSys imt QQ

I told grandpa that I have noticed signs that my mental power is declining, mostly in localSys and local codebase absorbency. However, localSys has always been a weakness in me. The decline, if any, is very gradual.

QQ learning is still in good progress. Experience, accumulation, deepening.. thick->thin.

Note A.Brooks’ article talks about creativity, innovation, analytical … My issue is mostly memory capacity and speed. Recall the Indeed interview.

## candd are assessed mostly@tech skill !!dnlg

I feel mostly we as candidates are assessed on technical not domain knowledge.

Q: Among your past job interviews which one had the highest emphasis on dnlg? In the interview, which one of the 3 dnlg categories? Usually math I would assume.

I over-invested in dnlg, relying on it to stand out and differentiate. Well, it’s not “reliable” as a competitive advantage just as SQL/Unix skill isn’t such a selective screening criteria. In contrast, I would say about half of my job interviews had highly selective tech screening.

  • eg: west-coast
  • eg: HFT shops
  • pure algo challenge — About 30% are tough, including a few west-coast challenges
  • whiteboard threading — easy for me, but 60% hard for most guys
  • online tests — MCQ (30%) or coding (80% hard)
  • interactive coding, remote/onsite, whiteboard/IDE — 70% are tough
  • core java/c++/c# knowledge — 50% are tough

##dry$valuable topics 4 high(absorbency)period #flight

This ranking was originally compiled for “in-flight edutainment”. Warning — If high churn, low accu, low traction, or short shelf life then the precious absorbency effort is wasted

See also the pastTechBet.xlsx listing 20+ tech topics and comparing their mkt-depth, demand-trend, churn-resistance … My conclusion from that analysis is — any non-trivial effort is either on a niche or a non-growing tech skill, with notable exceptions of coreJava+threading. All mainstream and churn-resistant skills need only trivial effort.

  1. coding drill — esp. the hot picks. Look at tag “top50”. Practice solving them again quickly.
    • 😦 low accu?
  2. java concurrency book by Lea. More valuable than c++ concurrency because java threading is a industry standard reference implementation and widely used.
  3. java tuning, effJava,
  4. c++ TMP?
    • 😦 seldom asked but might be asked more in high-end interviews. TMP is heavily used in real world libraries.
    • 😦 low traction as seldom used
  5. effModernC++
  6. linux kernel as halo?
    • 🙂 often needed in high-end interviews
    • 😦 low traction since I’m a few layers removed from the kernel internals
    • 😦 no orgro
  7. c++11 concurrency features?
    • 😦 low traction since seldom asked in-depth

importance@formal edu4tech career

I tend to dismiss the value of formal education (including degrees) and I tend to overvalue on-the-job quick-n-dirty learning. Compared to Asia, U.S. tech culture is less fixated on formal education.

  • Eg: Some inexperienced developer colleagues seem to have a good grasp of option math
  • Eg: my theoretical knowledge of comp science is completely self-taught, including concurrency, SQL, OO,..
  • eg: A more extreme example of such a domain is DynamicProgramming + Greedy algorithms. An intelligent programmer (regardless of age) can become highly competent without formal training.

Some theoretical domains are really based on field practice such as OO design.

Even if you have a solid education, either formally, or on the job over a few focused years, we all face the same challenge of continuing education —

  1. my web dev experience (self-education) is now outdated, according my interviews at Indeed , ByteDance ..
    • But luckily there are many interviews I can attend and learn from.
  2. data science, machine learning … is not so easy to self-learn
    • luckily there are many online learning resources.


short-term ROTI ] java IV: tuning imt threading

Analogy — physics vs chemistry in high-school — physics problems involve more logic, more abstract, more ; Chemistry is more factual. Similarly, Java threading challenges are more complex. Java tuning knowledge-pearls are relatively easy to memorize and show-off.

“Short-term” here means 3-5Y horizon.

I’m many times more successful with java threading– accu, deepening knowledge; lead over the pack.

I hope to build some minor zbs in JVM tuning, mostly GC tuning. However, many important parts of JVM tuning skills suffer from churn and poor shelf life.

##[18]any portable skills like coreJava/c++

I have found core java and core c++ outstanding domains for my personality. I feel there are not too many similar domains with

  1. feature: standardized knowledge highly portable across teams
  2. feature: requires more than such (simple) knowledge that anyone can pick up in a crash course
  3. feature: low churn high stability

… so we can hope to accumulate. Here are some comparable domains:

  • pure algo 🙂 yes standardized, non-trivial questions. I can accumulate. Better than brain teasers
  • socket and sys programming:) Yes standardized questions, including malloc, sharedMem
  • SQL 🙂 good despite shrinking demand.
    • Can we say the same (“shrinking”) about ANSI-C? Well, lots of c++ questions are C.
  • bond math 🙂 not so deep but good
  • JGC + jvm tuning? 😦 churn
  • python ? 😦 Deep questions are rare and non-standardized, like concurrency, meta-programming..
  • quant-dev domain 😦 questions are standard, but too few jobs
  • algo trading? 😦 i don’t think they ask any standard question

local jargon, localSys architecture #WFH

Background:  3 types: Portable dnlg ] finance IT listed 3 portable types of dnlg (i.e. domain knowledge), but now I feel the local (i.e. non-portable) dnlg is more relevant to

  • your remote work
  • GTD productivity,
  • your survival,
  • your overall value-add to the team,
  • assessment by manager

Interviewers can only ask portable dnlg, Local dnlg include things like

  1. local jargon
  2. local system architecture, local “data flow”

My xx-absorbency[def#1]!=highest #don’t scold boy 耐得住寂寞

See also 耐得住寂寞,也是天分 #absorbency for students

Obviously 99% of us have limited technical talent like learning capacity, memory, problem solving speed .. but today I will focus on another factor “absorbency | 耐得住寂寞” – the capacity to endure repetitive practice, sustained focus, often in solitude, and build the /mileage/ required for mastery. No one has unlimited absorbency. Sooner or later we all reach saturation point, or “breaking point” (Liu Shuo).

Opening example — I often tell my friends “Study is not hard work for me.”

example — When I was trying rock climbing, my finger and forearm would get too tired for me to continue. On the other hand, as a teenager, I could do 600 sit-up in 10 minutes non-stop.

example — jogging and yoga turns out 2 b%%most heroic self-mastery

Absorbency is one of the top 5 most important technical talents, (possibly top 3), esp. in the long run.

Some fellow parents (like Li Yi) said the vast majority of primary school pupils are not very different in IQ. I would add that differences in absorbency is huge. I often feel my son is weaker on absorbency when practicing piano, Chinese handwriting and math. I think he is improving. His swimming practice has always been good. My daughter shows more absorbency for hand-writing, renzi, and piano.

Some individuals are not the brightest/fastest but can tolerate high amounts of practice and can become formidable like 郭靖, 小龙女, compared to 杨过.

Note the word “absorbency” is more precise than ‘effort’. Absorbency is more about capacity, less about attitude, motivation.

Let’s shift focus to my own technical absorbency. I want to point out I’m not the strongest so I should not blame my son.

  1. Coding drill – I can do more than most peers, but then some grads can solve more than 5 problems a day, and pass all Leetcode tests.
  2. Quant study – I did more than half my homework on my own. Some classmates were able to practice and study more.
  3. Getting a grasp on a big codebase – absorbency required, beside learning speed + memory.
  • eg flight — how much technical learning you get on a 15H long flight is a measure of your technical absorbency.
  • eg coding drill — Rahul, me and Abhinav each find joy in coding drill but many of my friends seem to focus on reading the ideas rather than practicing and solving problems independently. Focusing on the end result rather than the journey is no fun. The joy increases your absorbency
  • Positive feedback — is a key factor affecting absorbency. Coding practice and software programming offers immediate feedback.
  • self-doubt — I frequently (once a month?) question my endeavor (and the elusive strategic value) .. detrimental to absorbency. It’s already hard to absorb the practice even without these self-doubts.
    * yoga
    * risk analytics
    * socket

growing popularity@scripting languages #threat2c++

See also sg19 %% attitude on java, relative2c++ and ..

By 2019, I have noticed growing popularity of high-level languages like GO, node.js and python … all dynamic, scripting languages, in contrast to compiled languages. However, java is still fairly relevant. I won’t say too much about the possible reasons — ease of learning, ease of finding and training new hires…

i feel this trend is a continuation + evolution of a longer trend — from c/c++ to java/c#.

One direct impact of this trend — demand for c++ skill is shrinking[3], again. The remaining c++ jobs tend to pay lower than these “easier” languages or require deep expertise (like HFT shops) .. very bad deals for the c++ job seekers.

[3] It’s important to remember that demand for c# and perl developers is also shrinking.

come-n-go /Flash-in-the-pan — In my younger days (early 2000’s), there were also a growing choice of scripting languages but only one became general purpose — Perl. I feel GO, node.js may lose popularity over the coming years. These languages are popular partly because they are new. I described elsewhere that among the scripting languages, only 1 or 2 will dominate. Perl was dethroned by python.

See y re-enter c++if jobs fewer — I still believe c++ is harder and “builds character“, unless you spend 90% of your c++ career using wrappers that other people wrote and don’t bother to look under the hood.

unexpected longevity@FOSS

Conclusion — my tech-bets and investment in many FOSS technologies proved to be correct. In contrast, only a few of my tech bets on commercial softwares are correct — MSVS, Oracle, Sybase, Excel+VBA,

I didn’t want to spend too much effort analyzing the forces around FOSS, but to my surprise, those forces keep growing and evolving.

  • Eg: weblogic was once dominant, but left behind by Tomcat and Jboss
  • Eg: Microsoft has to contend with Linux, Java, Apache
  • Eg: Oracle has to keep developing OpenSolaris, and MySQL
  • Eg: IBM, Oracle … have to support Linux
  • Eg: SUN, HP-UX all lost the battle against Linux. SUN has no choice but OpenSolaris
  • Most of them have to face the stiff challenge by a single FOSS — GNU/Linux

Because a FOSS needs no revenue no payroll to stay alive, there’s no survival risk or financial uncertainty in a FOSS project. Therefore, a FOSS often has better longevity.

Some of the most influential, dominant, enduring and low-churn softwares are FOSS and are unlikely to change:

  1. linux, BSD-unix
  2. java and GCC
  3. python, perl, and most scripting languages
  4. most development tools in *nix
  5. many javascript frameworks
  6. many browsers

Q: what forces power the FOSS and provide the energy, momentum?
A: alpha-geeks who want to create a impact and legacy?

Apparently, you need just one (or a few) alpha-geek to create a formidable competitor to a software vendor’s army of developers.

socket^swing: separate(specialized skill)from core lang

  • I always believe swing is a distinct skill from core java. A regular core Java or jxee guy needs a few years experience to become swing veteran.
  • Now I feel socket programming is similarly a distinct skill from core C/c++

In both cases, since the core language knowledge won’t extend to this specialized domain, you need to invest personal time outside work hours .. look at CSY. That’s why we need to be selective which domain.

Socket domain has much better longevity (shelf-life)  than swing!

learn new tech for IV(!!GTD): learn-on-the-job is far from enough

Example — you programmed java for 6+ months, but you scored below 50% on those (basic) java knowledge question I asked you in skype chat. You only know what to study when you attend interviews. Without interviews, you won’t encounter those topics in your projects.

Example — I used SQL for at least 3 years before I joined Goldman Sachs. Until then I used no outer join no self-join no HAVING clause, no CASE, no correlated sub-query, no index tweaking. These topics were lightly used in Goldman but needed in interviews. So without interviews, I wouldn’t not know to pay attention to these topics.

Example — I programming tcp sockets many times. The socket interview questions I got from 2010 to 2016 were fairly basic. When I came to ICE I looked a bit deeper into our socket codebase but didn’t learn anything in particular. Then my interviews started showing me the direction. Among other things, interviewers look for in-depth understanding of

· Blocking/non-blocking

· Fast/slow receivers

· Buffer overflow

· Reliability

· Ack

How the hell can we figure out these are the high-value topics in TCP without interviews? I would say No Way even if I spend 2 years on this job.

[18] IV (!! GTD)body-build`logbook by skill

I’d like to keep the table columns simple.

muscle delta when duration intensity
2D coding Q 3->5 2018 2D 3
perm/combo 5->8 2017 4D 3
string/array 6->7 2
backtracking 1->6 2018 2D 3
tail recursion 0->5 2017 x hours 1
big O analysis 7->8 2018 1
sorting algo nlg 7->8 2018 x hours 1
advanced recursion 3->5 2018 x Days 3
whiteboard best practice 6->7 2018 2 days 3
shared mem 2->4 2018 2 H 2 coding experiment
c++ for coding Q 3->8 2017 10-20 D 2 bbg
socket 3->6 17-18 x days 2 thanks to IV, not project
rvr/mv/forward 2->5 16-18 x days 2
noexcept 0->5 2018 1 H 1 Trex IV
other c++11 2->4 1
py basics 4 coding test 3->5 2

spend 4H/D self-study@@ #XR

Hi XR,

I called you to highlight to you the realistic amount of hours we can spend on self-study.

Real example 1 —- I monitored it for months in my Master’s program — I would spend about 30 hours a week, including lectures, tutorial classes and homework(about 15-30 Hr/week, often taking up one full weekend-day). If I self-study without the program, I think it will drop to 3 hours a week. Why so low? Read on.

Eg 2 —- Some graduate students prepare for coding interviews by going through some question bank, just as you planned. Could be close to a hundred questions, during a 3M period — just my imagination, not based on any observation. They probably have 4 hours/weekday + 16 hours/weekend-day of free time. How many hours do they actually spend ? I would guess 10 – 20 hours a week. Why so low? Read on

Real example 3 —- In Singapore, I had kids, bills, grocery, household chores, home maintenance, family outing … so I had far less spare time than now, but still I probably had 1H/weekday + 8H/WED (weekend-day) for myself. In reality, I averaged 3 hours a week on self-study, but you know what a motivated person I am. Why so low? Read on

Real example 4 —- in Feb 2018 I had some very promising, valuable interview opportunities (FB). Even on the very last weekend before the interview, my time utilization rate was very very low. I lock myself up in office with no one else to distract me. I sleep only 7 hours and I eat in front my computer. In fact, I tell my friends “I spent entire weekend in office without going out of the building. You would think I could spend 15 hours a day on the preparation. Well, more like 6. Why so low?

It’s not all about motivation. I had high motivation in Singapore. I had the highest motivation over the recent weekend. But the numbers are still much lower than “theoretical limits”. There’s huge difference between wall-clock time vs real time spent:

Example 5 —- My son would spend an hour pretending to study 20 Chinese characters but only learn 3 characters and soon forget all of them. So the real time he spent was … a few minutes?

It’s about intensity. I call it laser energy intensity.

Real example 6 —- in a coding interview, my mind was working at top capacity. Over 15 minutes I solved some very tough problem. The rest of the interview (I was there for 3 hours) was far less intense, though it’s still intense, as intense as any technical interview.

Example 4 continued —- I time myself for white-board coding. Over 20 minutes my mind was working under immense pressure. Over the next 2 hours I sit down at a computer and make the actual code work. It’s still a lot of work, but my mind was at 50% peak capacity.

Use numbers to gauge the amount of workload and your time utilization rate. Say each problem (in homework or a coding exercise) takes average 30 minutes in exam / interview. Beside eating, using toilet, you have 15 hours to spend, so you would want to do 30 problems. How many can you actually complete? Perhaps 6, up to 10. Why so low? Our mind can’t operate at peak capacity for 15 hours!

In Example 4, after I spend 20 intense minutes, I need to wind down. After I spend 2 hours on lower-intensity work, I also need to wind down.

In Example 1, at the lecture/tutorial classes, the intensity was about 30% on average. That’s why I can do 2-3 hours. At a higher intensity the students’ mind would /disengage/ after an hour. All experienced teachers would slow down and give breaks when introducing tough technical topics.

In Example 1, When I spent 10 hours to complete my homework, my intensity was about 50% and I took many breaks. Actually the concentration time spent was more like 6-7 hours.

How does this mental “workload” compare to paid project work in a regular job, including work-from-home?

  • 😦 Project work is often repetitive or mundane. I can’t choose another topic.
  • 🙂 Paid work often has much bigger impact (otherwise why paid) and often a sense of achievement
  • 😦 Paid work stress is harmful, since the impact is high.
  • 🙂 Project work often has a discipline built-in, so we don’t wander off. This is similar to my Master’s program
  • 🙂 Project work requires us to keep going when the going gets tough. Also in my Master’s program

personal learn`]difficult workplace: %%tips #XR

I understand your boss could be pushing hard and you have colleagues around who may notice what you do…. Well, Not much wiggle room for self-study. A few suggestions:

  • try to put some technical ideas into the code. I did manage to put in some threading, some anonymous inner classes, some memcpy(), some local byte array buffer into my project, so I get to practice using them. (I understand the constraints of a tight time line…)
  • it takes a few minutes to write down new findings discovered at work. I put them in my blog. I also try to spend a little more time later on to research on the same topic, until I feel confident enough to talk about it in an interview.
  • I try to identify some colleagues who are willing to discuss technical issues in my project. I try to discuss only when boss is not paying attention. Boss is likely to feel we are taking too much time on some unimportant issue.

If the learning topic is not related to work, then I feel it’s similar to checking personal investment account at work. (In the ICE office now, some employees get a cubicle with 4 walls so they get more freedom than me.) Do your colleagues check their investment accounts at lunch time? I believe they always get a bit of personal time. In GS, on average very roughly 1 out of 9 working hours is spent on personal matters, and the other companies have higher than that. We all need personal time to call insurance, immigration, repair, … The managers might need even more personal time. I would guess at least 60 minutes a day is yours. Question is how to avoid drawing attention. I don’t care that much about drawing attention, so I often print technical articles to read, or read on-line, or blog on-line.

It’s more discrete to write email to record your technical learning. I often send those emails to my blog (email-to-publish) or to my personal email address.

Personal time (be it 60 minutes or 3 hours at some banks) is never enough. We just have to try harder to squeeze a bit more out of the 9 hours. If you are serious about learning in your personal time, then I see two much bigger obstacles
1) family responsibility and distractions
2) insufficient motivation and persistent effort (三天打鱼两天晒网)

In my Singapore years (4.5 years), I felt overwhelmed not by work but family duties, so my weekends/evenings were almost never put to good use for personal learning. I can’t just blame my family members though. I do get quiet time 10.30 pm to 12.30 and many hours on weekends. Somehow, I didn’t put in persistent effort so I didn’t experience significant growth in my technical capabilities.

A very capable colleague was able to do his math research at home and make progress. His wife is probably a full time home maker and takes care of their 3 kids. He is wealthy so he may have a maid and a separate study room at home. However, I feel a more important factor is his persistent effort and focus. A rolling stone gathers no moss. By the way, this same guy runs about 5 miles at least 4 times a week. Determined and efficient. Good time management and correct priorities.

If (a big IF) we are sufficient motivated, we will find time or make time, either the 60 minutes at work, or on trains, or at home, or in Starbucks. In reality, very few individuals have that level of motivation, so I believe some external factors can help, such as (my favorite) —

* jot down some idea in a draft email and do a bit of research whenever I get time to build on the idea, until it’s complete and fairly substantial. Idea could be something I overhead, an idea I’m truly interested in. The learning is mostly in the research but also in the subsequent reviews. If I don’t review the email, I will forget most of it. When I do review it, I not only refresh my memory, but I often discover connections with other things I studied, or find new ideas to learn — 温故而知新. Learning is associative, like growing a spider web.

sense-of-urgency, for kids+professionals #IBM #Alok

If there is no time frame specified, there is no sense of urgency and nothing to measure against. If you set a time frame you will be able to use this as a reason to go back to your boss at regular intervals and update them on the progress, ask for help and set appropriate expectations throughout the year.”

So said an IBM site. “Sense of urgency” is a simple but powerful propaganda slogan introduced by the 1993 incoming CEO of IBM. In my experience, workers with a consistent sense of urgency will be noticed by the boss. A boss without a sense of urgency could be a nice boss but not effective.

I like this slogan, because my son needs it, badly. It should be instilled at a younger age.

If a frog sits in a slow boiler, she won’t have the sense of urgency to jump out.

By default, folks fall into inaction — weight management, regular exercise, or self-improvement. Setting some kind of measurable goal is a simple device. My dad goes a step further — until now, he has daily plans and he gets things done.

Take for example my house search. If I don’t have any sense of urgency, I would simply postpone indefinitely any research or discussion including school-districts. Reality is, sooner or later my son needs a school. The more I postpone my research, the less time I have to do a proper research. So I do feel a sense of urgency, but hopefully I’m not hitting too many colleagues in the office.

Let’s remind each other the sacrifices we made — the cut, living away from family, additional expenses including rent.

[[Focus]] for technical xx

See also my email to Raja, XR and ZengSheng on “the zone”, the “creative engine”…

As the book author, Dan’s concept of focus is more narrow. It’s related to a student’s focus. I told my son he needs better concentration. As a student I was very good with concentration and engaged.

Compared to my engaged “traction” days, I now spend too much time emailing, blogging on non-tech, calling home, research about housing, shopping, food preparation … But that’s not the key. During the work or study hours, I am not focusing as I did in primary school.

Traction/engaged days are with growth, concentration, positive feedback loop, tech blogging….

I think Yang was the first manager who pointed out I need better focus. Perhaps he meant I tried to study all the systems in PWM Comm all at once. I made real progress (with ErrorMemos, SQL, Java …) after I sent my wife/son back to Singapore.

I made real progress with quant stuff during the Barcap days. However, I also had to keep job hunting remotely.

Avichal, who sat beside me, said I had too many distractions… Disengaged.

Similar to dabao’s case, I guess the initial code complexity requires focus… Without sufficient focus, the complexity is just too much for my capacity. Better shut out everything else and focus on delivering something non-trivial. Remember PWM, Quartz, …

Grandpa had a lot to say about focus…

##study subjects I enjoyed as student(+professional)

(You don’t have to read this lengthy blog post.)

All my examples below required a long period of persistent effort. My son is unwilling to put in even 30% of my effort, so he wouldn’t find any enjoyment.

After the initial few hours of curiosity, learning any subject was always challenging, boring, or repetitive. The mental workload always wears us down, so we always needed conscious effort to stick to it I.e. stay focused and apply ourselves. The joy comes as a light at end of the tunnel. No exception.

In contrast, watching TV, playing electronic games, eating, sight-seeing, shopping … don’t require any effort. When we get bored with it, we are free to break away — there’s no measurable progress to worry so give-up has absolutely no consequence. We choose to stick to it (say, a game) only because we enjoy it. Therefore no effort required.

If a teenager takes on professional gaming as a paid job, he will invariably find it monotonous, repetitive, uninteresting, dull, tiring, before he can earn any money. Those gamers who do earn money are able to earn money precisely because they put up with the boring stuff — persistent effort.

— Now the study subjects I enjoyed, ranked by amount of effort:

I enjoyed blogging on technical subjects. My QnA interviews directly benefit from my blogging but this is not the biggest reason for the enjoyment. In fact, most of my technical blog content is not required for interviews. I can see my blog improving in quality (and quantity) as I revise and expand each article progressively. Blogging let’s me achieve deeper understanding on many tough topics like sockets, reflection, c++template, move-semantics, quant, complex SQL, … I call it "真本事". I then notice the superficial knowledge in fellow programmers, who studied only 20% of what I read.

I enjoyed memorizing English words, in my teenage years. I developed my own method. I was able to memorize 30 – 100 words a day for a few months (I refresh each word 20 to 40 times spaced out over 6 months). The fast progress was "breathtaking". Self-confidence grew. I actually learned some words my English-educated Singapore classmates didn’t know!

I enjoyed Chinese composition in my secondary Year 2-3, when I found my literary "voice", my style of writing. For about 2 years before that break-through, I was always in the bottom 10% in my class in terms of composition and every assignment was a pain like giving birth to a short, boring baby. The pain, self-despise, self-pity was one of the deepest in my childhood. I felt inadequate almost like crippled. The breakthrough was a liberation. I achieved it by writing daily essays for a few months. My output slowly improved from very dry, simple "流水账" to have some meaning, some color, some personality, some nice expressions. It became creative expression.

(Paradoxically, as a student I was very strong in math/physics, but I never had the same enjoyment as I had learning English and Chinese composition. I think my joy was in overcoming highly visible personal limitations — sense of breakthrough and liberation.)

I enjoyed piano playing after years of repetitive, boring, tough practice. In my life I never again put up with this much practice — 5000 repeated practices x 2 minutes/repetition in some cases.

I know more c++than c#(spent X years full time)

Action: spend enough (30?) hours revisiting my 1000+ cpp (including socket, stream..) blog posts and explore just a little beyond them.

(I blogged about exactly the same observation before …) Here’s a paradox — i spent 2 years in a full time c# job, arguably longer than my full time c++ experience. However, sometimes I feel I understand dotnet less than C++.

Reason? Many aspects of the languages are never used in-depth on work projects, so length of experience != depth of experience. We need spare time self-exploration to learn those aspects like:

– template meta programming
– custom operator new/delete
– memory tuning
– memory leak detection
– threading
– async communications
– pure virtual
– ++i vs i++

(… not a complete list by any measure.) Interviewers are notorious for testing these obscure or advanced, theoretical topics.

create positive-stressful env n keep fit #UChicago;c++job

Hi XR,

We talked briefly about this human nature — no one has that “perfect” will-power, perseverance and persistent motivation, therefore as a result the environment we put ourselves in can help induce a significant personal effort.

Here’s the situation we discussed — you are in a perm job, you won’t feel motivated to keep up interviewing skills. I guess our workload + family responsibilities will “crowd out” any personal study plan to keep learning and stay fit for interviews. (This looks like one example of the boiling frog — a frog won’t jump out of hot water if on a slow cooker.)

I see many examples of this human nature:

  • I mention my blog and my github in my resume, so I sometimes worry that prospective interviewers may spot mistakes. That fear keeps me motivated to improve my “published” work. When
    I review my work, I often learn something, and I often feel excited that I’m slowly building up a positive “public” profile.
  • Without taking my current c++ job, how many (spare time) hours would I be able to put into c++ self-study? I think average 2hr/week is achievable but not easy. Nowadays I actually put in about 5 to 10 hours outside 9-6 work hours, on my c++ projects.
  • In Singapore, my wife can put in some effort to improve English but she doesn’t have time or energy. She finds it difficult to put in the effort. In the U.S. she WOULD indeed put in more effort learning English. On the other hand, many students in China could build a formidable English vocab simply by reading — they have a persistent motivation.
  • 30 minutes moderate exercise 5 times a week — is a basic guideline, but I know very few adults actually doing that consistently. I bet when one’s health condition requires that much exercise, there will be more motivation and more effort.
  • Once I paid course fees, I routinely put in 30 hours a week on my financial math studies. In theory I could self-study the same, but how many hours would I put in on average over a year? Perhaps 30 minutes/week!

(Most of my examples are about self-improvement and self-motivation. )

it takes effort to remain in financial IT#AshS

NY vs NJ — cost is basically the same my friend, in terms of rent, transportation, food … Most of my friends probably live in New York city suburbs. (I know no one who choose to live in NYC, paying 4% city tax.) My cost estimate is always based on my experience living in NYC suburbs.

The “relief” factor you identified is psychological, subtle but fundamental. It resonates in my bones. It’s a movie playing in my head every day.

In the present-day reality, at my age I probably can still find jobs at this salary in Singapore or Hongkong, with growing difficulties. How about in 10 years? Huge uncertainty. If you were me, you have to take a long, hard look into yourself and benchmark yourself against the competing (younger) job seekers. You would find negative evidence regarding your competitiveness on the Asia financial IT job market. That’s the get-the-job aspect. The 2nd aspect is keep-the-job and even tougher for me. Therefore in the U.S. I make myself a free-wheeling contractor!

As I age, I take the growing job market competition as a fact of life. I accept I’m past my prime. I keep working on my fitness.

On both [get|keep ]-the-job, U.S. offers huge psychological relief. Why? Ultimately, it boils down to long term income (and family cash flow). Since I believe U.S. employers can give me a well-paying job more easily, for longer periods, I feel financially more secure. I will hold meaningful jobs till 65 (not teaching in polytechnic, or selling insurance etc). I can plan for the family more confidently. I could plan for a new home.

There are many more pros and cons to consider. Will stick to the bare essentials.

Singapore offers security in terms of the social “safety net”, thanks to government. Even if my Singapore salary drops by half, we still enjoy subsidized healthcare, decent education, frequent reunions with grandparents, among many other benefits (I listed 20 in my blog..).

Besides, I always remind my wife and grandparents that I have overseas properties, some paying reliable rental yield. Under some assumptions, passive income can amount to $5k/mon so the “worry” my dad noticed is effectively addressed. Another huge relief.

As a sort of summary, in my mind these unrelated concerns are interlinked –

· My long-term strength/weakness on job markets

o coding practice, lifelong learning

o green card – a major weakness in me

· family cash flow

o passive incomes

o housing

· Singapore as base camp

Do you notice I didn’t include “job security” in the above list? I basically take job-Insecurity as a foregone conclusion. Most of my friends are the opposite – taking job security as an assumption. This difference defines me as a professional, shapes my outlook, and drives me to keep working on “fitness”.

If I had no kids, I would have already achieved financial freedom and completely free of cash flow concerns. As a couple, our combined burn rate is S$3k-4k/mon but our (inflation-proof) passive income has/will reach that level. Also medical and housing needs are taken care of in Singapore. In reality, kids add S$1k-2k to our monthly burn rate. We are still on our way to financial freedom. So why the “worry”?!


Sent: Thursday, 16 March 2017 8:07 AM
To: Victor Tan
Cc: ‘Bin TAN (Victor)’
Subject: Re: it takes effort to remain in financial IT

That jewish guy and avichal are exceptions . Most of developers struggle and get success by hard work, by spending time with code for atleast 1-2years in any new company. I have not met other fast learners like them. Those are rare to find.

Regarding the worry that your dad always notices , i think this is quite obvious “more money more stress” but yeah this is the right time for you to move to USA where there are plenty of tech jobs . This move will surely give you a shy of relief.

Regarding the cost if living, why would you want to stay in NY , it is indeed costly. How about NJ?

QQ^ZZ mutual exclusion: c++/java..

Background — The QQ/ZZ framework was first introduced in this post on c++ learning topics

Focus on tough not ordinary topics in c++ or java.

QQ: By and large, tough topics in job interviews are never needed in projects.
ZZ: By and large, tough topics in work projects (zbs) are never tested in job interviews.

Mutually exclusive =>

  • effort (laser energy) I spent on textbook knowledge during job hunt basically hurts my work,
  • effort I spent on tough zbs topics for a project very seldom help with job interviews.

Therefore, some optimizing candidate would make the minimum effort to get by on projects, and channel the energy to QQ knowledge or algo challenges.

master a skill^depend@colleagues – you decide

Nikhil surprised me when he told me he frequently asks for help… Until then, I always thought I need to go through the problem solving by myself, unaided, or I will never learn.

Context — trading engine team workload is often 3 times the normal, comfortable level. Headcount is often below 33% normal headcount. (This is a result of time-to-market delivery experience — The fewer developers, the lower communication overhead, the more nimble.)

Challenge – Suppose you are suddenly asked to finish something quick and dirty in half the time previously allocated.

Higher pressure usually means you need to depend more heavily on other developers in your own team. Many things you should not *want* to understand. You need to decide what analysis/research/investigation to “outsource”, but still retain control.

A component could be your colleague’ responsibility, but familiarity improves your *velocity* and reduces your reliance on the author. With mastery, you often discover tweaks and shortcuts, saving time when there’s no time at all. You can become more knowledgeable than the author about its usage!

Look at GS colleagues Nikhil, Anil, Nicholas… They ask each other dozens of questions for everyday BAU, but each mastered how to read and experiment with JIL, sybase stored proc, CVS, unix commands, …They don’t master every subject, and they don’t lean on others in every domain.

Eg: how to programatically insert reference data into an empty data store, rather than through a complicated (buggy) GUI. The GUI route is un-automated if you must repeatedly rebuild from scratch.
Eg: key tables. Out of 20 to 50 frequently used tables, about 2 to 5 tables should be familiar to everyone, like Comm, trade, offer. Before you go through these steep learning curves, you aren’t initiated into the club.
Eg: how to locate and parse the logs
Eg: how to test-send and test-receive messages
Eg: how to restart some of the important servers
Eg: how to identify common issues in each essential server
Eg: parse a few cornerstone config files
Eg: parse autosys JIL to see dependencies and flows

But as a Greenfield dev consultant, you aren’t expected to get too deep into prod support or BAU.

A short-term greenfield developer consultant is not a “Subject Matter Expert”.

which programming languages changed toooo much

No elaboration please. Pointers will do.

perl 6?
python 3?  but the excessive changes are not taken up!
php5 was better received.
C# ?
MFC -> winforms -> WPF ? Different technologies. More like delphi/powerBuilder/coldFusion declining than a language evolving.
java 8? not much change. Lambda is the biggest
c++11? not much change. Rval ref is the biggest

[14] less stressful job in financial IT@@ #XR

You wrote:

Pros: Less time spent in learning new things, since  most new technical skills will be abandoned in several years, so a lot of time is saved.

I agree that saving time is important, for stressful daddies like you and me. I try to identify those technology fads and avoid wasting time on them, except when the fad (like silverlight) is really needed on my job.

While working for big banks I feel I’m paying a price in terms of family life and my health (i.e. stress level).

Q: Is a less stressful job more healthy in the long run? not sure.

My OCBC job was slower after the initial 6 months, but my learning slowed down too much. I became less competitive less strong less marketable on the job market. That won’t help with long term well-being. So in this case answer is No.

My Citigroup contract project was also slower than other Wall St jobs, but I kept myself busy by learning c++. I guess the ideal situation is low-stress high-motivation, highly engaging. Perhaps a job we could cope well, but gives us motivation to exceed expectation. So in this case answer is Yes.

Overall, I felt (a bit) less stressful on Wall St as a contractor than now as an employee. As a result, my health condition was probably (a bit) better in U.S. Paradoxically, I think my health condition has a lot to do with the job market demand — Wall St relatively easy to find a job, so I didn’t need to work so hard to keep a job.

Before entering finance IT, was my work stress lower? Sometimes yes sometimes no. Some of the stress on Wall St we can learn to cope, so it won’t affect health in a massive and destructive way. However, some of my non-finance jobs (in Singapore) were extremely unstable — one company let me go within 1 month, another company extended probation period. Stressful? What stress? Livelihood stress? I was a bachelor and didn’t feel stressful but I imagine a father of two would feel traumatized! In the early 2015 layoff, I was feeling really stressed and depressed after getting so few calls on the job market. I felt the local financial IT job market here was dead. The banks were not hiring. I was happy just to see a Barclays Singapore role below AVP. A Commerz bank java opening was taking 2-3 months to get approved.

Perhaps the key factor is the feeling of in-control — When I felt I could handle the turbulence, the ups and downs, the sudden changes… then I felt OK. Now, with kids, I need stable income and time to spend with family. I don’t like anything to rock my boat.

You don’t have to read these but my earlier comments are at

On Thu, Oct 2, 2014 at 9:18 AM, you wrote:

Talking about learning, some people actively learn, just like what you did: plan what to learn, then take spare time to learn it. While some others passively learn, such as they are forced to learn in work. For example, one developer work in a company for several years, and then company wide technology upgrade to some main stream skills. So the developer can still keep pace with relatively new skills. Although those relatively new skills are not the latest, they should be proved to be accepted by big financial firms.

The latter guys won’t easily get an ideal job, but they should be able to get a job in most market conditions.
Cons: Difficult for them to pursue the ideal job. Dare not to change job just for better payment.
Pros: Less time spent in learning new things, since some most new technical skills will be abandoned in several years, so a lot of time is saved.

On Wed, Oct 1, 2014 at 5:34 PM, Bin TAN (Victor) wrote:

Forgot to mention that my study of c#, c++ and financial math study
will, I believe, help with my career ….somewhere down the road. When
i get interview questions on these topics, I’m sure I can handle them
better thanks to my recent study. I tell myself to be patient with
this “investment” in myself.

As mentioned in the last mail, In the last 2 years after I came back
to SG, the study has slowed down quite a bit, but i believe i still
improved my knowledge and understanding in certain areas.

Another potential long-term return of my financial math study is ….
teaching as a career. A lot of finance professionals switch gear to an
academic career after a certain age. I keep this option open by
working hard on my studies. If i get decent grades i might qualify to
teach. Compared to financial IT, teaching is hopefully less stressful,
even though i don’t want to underestimate the job duty of a dedicated
teacher. A good teacher can really impart useful knowledge. I may not
be a top teacher but i have my strengths in this area.

On Mon, Sep 29, 2014 at 1:38 AM, Bin TAN (Victor) wrote:
> I believe our mind (esp. our memory) is like a muscle. If we don’t
> seriously use it, it tends to age, weaken and lose capacity.
> Over the last 5 years, i spent 2009-2011 in the US on my own —
> tremendous learning and improvement, perhaps the most active learning
> period in my professional life. Then after I came back to SG, i
> studied financial math program, c#/dotnet, and some c++.
> I feel the serious study keeps my mind active. However, over the last
> 18 months, I notice various signs of my learning capacity reducing,
> but it’s not all due to aging —
> * biggest factor is lack of concentration, due to kids and family commitment
> * not enough time to periodically re-visit each topic,
> After all, I feel it’s vital and paramount to keep our mind in
> constant learning mode. 活到老学到老.

##[15] once-valuable tech skills: personal xp

perl – lost to python
[$] tomcat, jboss. Weblogic lost most market share
apache, mysql
dns, unix network config
autosys – not used after GS
[$] sybase, oracle – after ML edge project I didn’t need it again.
[$] VBA – used once only. But Excel is going strong!

[$ = still has value]

–random list (not “top 10”) of longevity skills
eclipse, MSVS
Excel and add-in
javascript, though I don’t use it in my recent jobs
Linux shell
compiler knowledge
make, msbuild
bash and DOS batch scripting, even though powershell, vbscript and python/perl are much richer.

local jargon,local domain knowledge = localSys knowledge

I agree that finance IT skills tend to “converge” across big banks, so we poor developers need not learn so many damn shit new stuff like git, python, continuous build…

On the domain knowledge side, the “local system knowledge” has no convergence as such, so each time I take up a new system, I must spend the time on the stored procs, table relationships, batch jobs, feed files, etc. There are also a lot of non-technical local knowledge. I hear you when your manager complained “what do you know about the system?”

Local system knowledge can be a long learning curve. Those who write the code in the first place has a huge advantage —

“It takes me 3 months to write this damn shit, then we add new features and fix bugs, but it’s still familiar to me. A new guy would take a year to get familiar code-wise, provided he has chance to explore all the key components. If he’s only exposed to half the system, then he would never know  the other half.”

(Therefore, I like to rewrite stuff.)

In I mentioned 3 types of domain knowledge — Jargon, Math, and Architecture.The Math part is stable. Local system knowledge includes a lot of local jargon. Sometimes generic jargon can help us pick up the local jargon.

constant study(over last5Y)keeps brain young#like muscle

I believe our mind (esp. our memory) is like a muscle. If we don’t seriously use it, it tends to age, weaken and lose capacity. Over the last 5 years, i spent 2009-2011 in the US on my own — tremendous learning and improvement, perhaps the most active learning period in my professional life. Then after I came back to SG, i studied financial math program, c#/dotnet, python and more c++.

I feel the serious study keeps my mind active. However, over the last 18 months, I notice various signs of my learning capacity reducing, but it’s not all due to aging —

  • biggest factor is lack of concentration, due to kids and family commitment
  • not enough time to periodically re-visit each topic,

After all, I feel it’s vital and paramount to keep our mind in constant learning mode. 活到老学到老.

Learning a tech is not 100% all about getting the job done or getting better jobs, even though that’s about 99% of it, admittedly.

A few colleagues (a tiny minority) at my various jobs seem to enjoy the learning process, even proprietary technologies with low market value. Learning tech can be a joy. In such a context, or such a geek, don’t mention “saving time”.

As mentioned in the title + first email, I feel our brain is like learning machine, and like a muscle. (I guess chess is also a kind of brain exercise…) Any such exercise is never a waste of time.

Some people make a decision to learn something (Spanish? Wu-bi Chinese input? musical instrument) even when they are as busy as we are. I guess Learning a technology is sometimes like that.

However, I (grudgingly) agree it’s not so fun learning, say, java struct, for years and then see it falling out of favor. I carefully pick the low-churn technologies, like you would pick watermelons at the supermarket:) See my earlier email

what c++ topics are valuable in the long run on WallSt

When you first pick up a language (like c++) from 11 textbooks, you form a view of what topics (know-how/insights) are important, and worth spending time on.

However, once in the field you will get a different view. Over the years, I realized
++ remote debugger is critical

++ lots of C techniques are widely used on Wall St.

++ MSVS is a dominant tool, so is GNU toolkit.

++ multiple inheritance is still relevant

— most whiteboard coding will use C constructs + vector/string, not fancy c++ features

— STL containers are widely adopted, but not everything in STL is equally important.
— portability isn’t a issue.

— template techniques are powerful but seldom used on Wall St
— boost shared_ptr is widely used, but not other boost libraries

— strings — many teams have their in-house development, so the std string is less imp

— I feel OO techniques are not as popular/easy as in java/c#

So over 50 years (if I were to work that long), i’m going to discover all the important topics. Now question is how to quickly discover them, so as to focus on the most valuable.

Answer — I feel a beginner is better off changing project frequently. The more diverse projects you take on, the faster you discover those important “topics”.

The rolling stone gathers no moss? Well, i feel this is not the case for the beginner dynamic traveling consultant:)

localSys knowledge about a Live system #3 sources#GS

When you work in financial IT, real insights into a live financial [1] IT system comes from

A) hands-on experience, often local system knowledge

B) user interaction — so you know the 80% of the codebase that are NOT important.

B2) production problem solving, with user-impact or business impact. As explained elsewhere, these steers our effort to the key sections of codebase. 20/80 rule — 20% of the codebase holds 80% of the system functionality, business logic and production issues.

Without B, you as a developer can be familiar with many sub-systems but without confidence to support users on your own, implement numerous changes, assess impacts, estimate effort, identify root causes… In many organizations developers aren’t assigned such tasks, but GS was very different and very effective.

C) good domain knowledge — This is required only in quantitative systems. Otherwise, I feel A+B will give us (system developers) more than enough domain knowledge. When C is relevant, then A+B will make you productive without “insight”.

[1] including real-time trading systems, reporting systems, financial analysis systems, STP etc.

remain relevant in a given technology(fundamentals

Q: At age 55, will you still qualify as a swing developer, or SQL developer, or python developer, or Unix admin?

1) An important factor is the stability of the language and the changing demand on specific skillset in that Language (contrast java vs SQL), but let’s focus on another factor — fundamentals.

2) In every language, there’s a body of fundamental knowledge (“knowledge-pearls”) that are Empowering and Instrumental. They facilitate your learning of “superstructure”. Many hiring managers believe strong fundamentals are the most important skill with a given Language.

I feel low-level knowledge will help you remain /relevant/ over 10 years. Some of the most tricky and frequently quizzed sub-topics are low-level — threading, memory-mgmt, collections, subclass memory layout, casting, RTTI, vptr, c++ big 3, …

3) I feel an appreciation of relative strengths/weaknesses of alternative technologies will help you remain relevant. Fundamental knowledge would help you appreciate them.

4) essential libraries of threading, collections, I/O, networking, … are not strictly part of the fundamentals, but part of mandatory skills anyway.

learning common code smells #time-savers


Every experienced and self-respecting developer cares about maintainability, because everyone of us has first hand experience tracing, changing, and “visualizing” convoluted logic in other people’s code. Once you know the pain, you intuitively try to reduce it for your successors — altruistic instinct.

At the management level, every experienced /application-manager/ recognizes the importance of maintainability since they all have witnessed the dire consequence (earthquake, explosion, eruption…[1]) of making changes to production codebase without fully understanding the codebase. More frequently, management knows how much developer Time (most Wall St teams are tiny — nimble and quick turnaround) it takes to find out why our system behaves this way. Bottom line — unreadable code directly impacts an application team’s value-add.

Technical debt is built up when coders take shortcuts, hide dirt under the carpet,  and “borrow time”.

If we ask every developer in each of our past teams to write down their dreaded code smells, some common themes will emerge — like rampant code duplication,  VeryLongVeryUgly methods, VLVU method signatures or 10-level nested blocks. Best practice means no universally-hated code smells.

Beyond those universally-hated, code smells can be widely hated, hated by some, frowned-on or tolerated … — different levels of smelliness.

I am trying to understand the common intolerance of the common developers (not the purists). You seem to suggest there’s no point trying to. If someone says there’s code smell in my code, i’d like to know if it’s universally hated, or hated-by-some.

In many case, our code is seldom debugged/modified/reviewed by others. In those cases, it doesn’t matter that much.

Happy new year …

[1] In reality, they often get the “patient” to sigh on the not-responsible-for-death before operating on the patient. They also have a battery of UAT test cases to make sure all the (common) use cases are covered. So the code change is a best-effort and may break under unusual circumstances.

3 types: Portable dnlg ] finance IT

Domain knowledge is specialized, hard-to-find, and an entry barrier. I see 3 broad categories of domain knowledge — Lingo (jargon), Math and Architecture

— 1) Lingo (Jargon) — Technical or Non-technical professionals in a given trading system must share a few hundred terms, phrases, verbs and adjectives. Each jargon term often has a superficial “face” meaning and connects with other jargon terms. The full meaning in context often requires a wikipedia page. Some Random examples that came to my mind — fixings; basis risk; option knock-in; tick DB; Basel; lockfree; move-semantic; rvalue ref.

Roughly half the jargon terms involve math. An IT guy can either memorize the derived mathematical conclusions or examine the underlying math. Random examples of such jargon include — volatility surface; yield; price sensitivities; yield’s impact on FX options; when to use stress testing vs VaR. I feel many people in IT don’t really have a firm grip on these. We go into a restaurant of financial jargon and look for fast-food. I call it fast-food culture, or quick-answer culture. Anything we don’t easily understand, we like to say we don’t need to understand. If we don’t make a conscious effort, we will always stay a financial laymen. As a result, a “conscious” programmer completely new to finance for 6 months can understand concepts deeper than a 5-year veteran.

Rather than “Jargon” I prefer “Lingo” — more general and less technical.

— 2) the Math part is a body of knowledge created by quants and PhD’s, over the past 50 years or so, perhaps starting with bond math. Needs high school math and later some calculus. For many IT folks who left school 5+ years ago, even the high school math pieces are not a piece of cake. If I don’t invest hours of spare time, i won’t fully understand half the theories in bond math which is all high-school.

The math in finance looks simple but needs to be rigorous. I feel pricing calculations in fixed income and in options often use a precision of 5 to 10. A shallow understanding could overlook details.

Financial math is a sizable body of theory, but most trading IT developers need only the basics. If I could say that I needed 6 months to grow from zero-finance-knowledge to be competent enough to help build trading engines, then obviously my roles didn’t need a lot of financial math.

— 3) system architecture + infrastructure + best practices — components like pricing, risk analytics, high speed market data, pnl explains, stress test, visualization, exchange gateways, SOR, DMA, mark-to-market, trade capture …
It’s possible to talk about an architecture of a complete suite of components but I prefer to talk about architectures of individual components. I prefer this because architectures are vastly different in terms of volume and real-time nature.

Banks want to hire experienced developers to help them re-architect, so as to stay competitive in the “arms race”.

Architecture domain knowledge changes faster than math or jargon knowledge. Just like jargon and math, architectural knowledge is specialized and not a commodity skill, so relatively few people in finance IT has it, creating a has-vs-hasnot divide among candidates. It’s difficult to gain that insight and knowledge because developers don’t need to know all the design decisions once made to fix up the live system they are now supporting. Team is set up such that you are given a lot to do just to get-things-done so you have no spare bandwidth to learn other components, but non-trivial insight into neighboring components are necessary for an architectural understanding.

I have seen a few impeccable blue-prints that fail in practice and need major changes.  Anyone can come up with great-sounding architectures but most of them are /sub-optimal/. Reason can be “hard to debug”, “inflexible”, “learn curve”… Therefore real world, proven architectural domain knowledge is rare and valuable.

Most architectures rely on solid, time-honored infrastructure software like Message-oriented-middle-wares, RDBMS, distributed cache and grids, cross-platform RPC/ORB, thread libraries, xml. Vendors always say “we are perfect” but good knowledge about their weaknesses and limitations are hallmarks of real architects.

Some say fixed income has more domain knowledge but that’s true only in the math area. Equities (and FX?) HFT probably have a larger body of domain knowledge in the architecture space.

Market data systems have substantial jargons + some architecture.

Risk systems probably (based on hearsay) has all three, but I feel only a small percentage of the software developers need math.

[11]traction[def1]^learning curve gradient

Context — learning a new software language, API, entire server (OS, DB..) or toolkit, with non-trivial concepts embedded therein.

The more common pattern is the learning curve. Initial gradient is often higher as pick up speed. After 6 months (or 1, or 12..) it flattens and tapers off and you experience diminishing returns.

A 2nd pattern is “gaining traction” . For the first 4 weeks (or 1 or 20..), you spend a lot of time reading and experimenting but without growing confidence…

  1. after a while, you start to connect the dots via thick->thin, often in a series of incremental breakthroughs.
  2. thick -> thin is not merely (superficial) accumulation of knowledge
  3. you often need perseverance and sustained focus. See focus+engagement2dive into a tech topic#Ashish
  4. knowledge gap build-up above the new entrants
  5. high ROTI
  6. high retention rate
  7. gaining-traction is opposite of wheel-spinning

I experienced and overcame this wheel-spinning process in ..

– swing
– C++, – java, c#
– python
– drv pricing
– options
– yield


A related pattern is engagement

which (vendor-)implementations to invest in

I may have talent in comp science theories. But I place my highest emphasis on familiarity with implementations — tools, utilities, products, packages, jars, building blocks — download-able stuff.

With theories, a talented student can progressively go deeper, higher, sharper, and in less time as her foundation strengthens and broadens. I feel less so with tools. Familiarity with implementations takes a hell of time.

For example, when something doesn’t work with something else, it often took me a few minutes to a few days. Therefore I have a strong aversion for new technologies and a strong affinity to mature, time-honored tools.

For example, countless designs work great on paper or during interviews. The #1 thing about any tool is the limitations you are likely to hit. Even a minor drawback can derail your entire project.

Some tools I’m investing into (either my old turf or new targets, the +++) and some (–) I’m divesting
++ c/java debuggers
++ sybase, oracle, mssql
+ rv, MQ (JMS is a spec, not an implementation)
+ unix sockets
+ eclipse
+ pthreads implemented in linux and solaris
+ python + solaris/linux latency tuning
+ pl/sql?
– spring / hibernate
– bash customization
– http
– mysql, php
– vi — needs 2 years. See [[Productive Programmer ]]
– any local system knowledge

harmless productivity tools ^ frameworks like ..Spring

[[The Productive Programmer]] by Neal Ford advocates learning a large number of _simple_ tools for productivity. Tools could be a complex but we use them in a non-intrusive, simple manner. Many other tools promise to help productivity but are traps – spring, hibernate, jaxb, xslt. Once you employ them they tend to become your master.

I have no such issues with perl/shell scripting, testing tools like junit, unix utilities, text editors, eclipse, maven/ant (?) A critical feature is, the option to use the tool in part of my project, and use alternative tools for the remainder. The tool should not be possessive and exclusive.

Hibernate is more gregarious than Spring, as you can mix Hibernate and straight JDBC in different modules of your system. Spring, being a framework, tends to engulf your entire project. You are then constrained by that framework developers’ decisions.

avoid convenience wrappers;xx low-level APIs #RV,socket

See also high^low-level expertise

Many examples of the same observation — convenient high-level wrappers insulate/shield a developer from the (all-important) underlying API. Consequently, in a hardcore interview he can’t demonstrate his experience and mastery in that particular “real” technology — he only knows how to use the convenience wrappers.

  • example — servlet — in one of my GS trading desks, all web apps were written atop an in-house framework “Panels”, which was invented after servlet but before JSP, so it offers many features similar or in addition to JSP. After years of programming in Panels, you would remember nothing at all about the underlying servlet API, because that is successfully encapsulated. Only the Panels API authors work against the underlying servlet API. Excellent example of encapsulation, standardization, layering and division of labor.

Notice the paradox — guys who wrote those wrappers are considered (by colleagues and job interviewers) as stronger knowledge experts than the users of the wrappers. If you think about it, you (as everyone else) would agree that knowledge of the underlying API is more valuable than GTD skill using the wrappers.

  • example — Gemfire — one of my trading engines was heavy on Gemfire. The first things we created was a bunch of wrapper classes to standardize how to access gemfire. Everyone must use the wrappers.

If you are NOT the developer of those wrappers, then you won’t know much about gemfire API even after 12 months of development. Interviewers will see that you have close to zero Gemfire experience — unfair!

  • example — thread-pool — One of my ML colleagues told me (proudly) that he once wrote his own thread pool. I asked why not use the JDK and he said his home-made solution is simple and flexible — infinitely customizable. That experience definitely gave him confidence to talk about thread pools in job interviews.

If you ask me when a custom thread pool is needed, I can only hypothesize that the JDK pools are general-purpose, not optimized for the fastest systems. Fastest systems are often FPGA or hardware based but too costly. As a middle ground, I’d say you can create your customized threading tools including a thread pool.

  • example — Boost::thread — In C++ threading interviews, I often say I used a popular threading toolkit — Boost::thread. However, interviewers repeatedly asked about how boost::thread authors implement certain features. If you use boost in your project, you won’t bother to find out how those tools are internally implemented. That’s the whole point of using boost. Boost is a cross-platform toolkit (more than a wrapper) and relies on things like win32 threads or pthreads. Many interviewers asked about pthreads. If you use a high-level threading toolkit, you avoid the low-level pthreads details..
  • example — Hibernate — If you rely on hibernate and HQL to generate native SQL queries and manage transaction and identities, then you lose touch with the intricate multitable join, index-selection, transaction and identity column issues. Won’t do you good for interviews.
  • G5 example — RV — one of my tibco RV projects used a simple and nice wrapper package, so all our java classes don’t need to deal with tibrv objects (like transport, subject, event, rvd….). However, interviewers will definitely ask about those tibrv objects and how they interact — non-trivial.
  • example — JMS — in GS, the JMS infrastructure (Tibco EMS) was wrapped under an industrial-strength GS firmwide “service”. No applications are allowed to access EMS directly. As a result, I feel many (experienced) developers don’t know JMS api details. For example, they don’t use onMessage() much and never browse a queue without removing messages from it. We were so busy just to get things done, put out the fires and go home before midnight, so who has time to read what he never needs to know (unless he plans to quit this company)?
  • example — spring jms — I struggled against this tool in a project. I feel it is a thin wrapper over JMS but still obscures a lot of important JMS features. I remember stepping through the code to track down the call that started the underlying JMS server and client sessions, and i found it hidden deeply and not well-documented. If you use spring jms you would still need some JMS api but not in its entirety.
  • example — xml parsing — After many projects, I still prefer SAX and DOM parsers rather than the high-level wrappers. Interviewers love DOM/SAX.
  • G3 example — wait/notify — i used wait/notify many times, but very few developers do the same because most would choose a high-level concurrency toolkit. (As far as I know, all threading toolkits must call wait/notify internally.) As a result, they don’t know wait/notify very well. Strangely, wait/notify are not complicated as the other APIs mentioned earlier but are tricky to 90% of developers.
  • G7 example — FIX — one of my FIX projects uses a rather elaborate wrapper package to hide the complexity of FIX protocol. Interviewers will ask about FIX, but I don’t know any from that project because all FIX details were hidden from me.
  • example — raw arrays — many c++ authors say you should use vector and String (powerful, standard wrappers) to replace the low-level raw arrays. However, in the real world, many legacy apps still use raw arrays. If you avoid raw arrays, you have no skill dealing with them in interviews or projects. Raw arrays integrate tightly with pointers and can be tricky. Half of C complexities stem from pointers.
  • G5 example — DB tuning — when Oracle 10 came out, my Singapore (Strategem) DBA friend told me the toughest DBA job is now easier because oracle 10 features excellent self-tuning. I still doubt that because query tuning is always a developer’s job and the server software has limited insight. Even instance tuning can be very complicated so DBA hands-on knowledge is probably irreplaceable. If you naively believe in Oracle marketing and rely on self-tuning, you will soon find yourself falling behind other DBAs who look hard into tuning issues.
  • G3 example — sockets — i feel almost every place where sockets are used, they are used via wrappers. Boost, ACE, java, c#, python, perl all offer wrappers over sockets. RTS is the exception. However, socket API is critical and a favorite interview topic. We had better go beyond those wrappers and experiment with underlying sockets.

I recently had to describe my network programming experience. I talked about using perl telnet module, which is a wrapper over wrapper over sockets. I feel interviewer was looking for tcp/udp programming experience. But for that system requirement, perl telnet module was the right choice. That right choice is poor choice for knowledge-gain.

The Quoine interviewer Mark was the only hiring manager to appreciate wrappers more than underlying socket knowledge. He is the exception that proves the rule.

##growing in java: where to focus when learning c++/c#

I feel stronger than a rookie java developer because we are comfortable with both GTD and IV:
1) IDE
** (remote) debugging. Also consider my Perl sss()
2) threading — idioms, principals
** thread creation, synchronized, wait…
** threadlocal
** thread pool
** immutable
3) reflection + proxy
) syntax of overriding, constructors, final, singletons, generics
) design patterns — large idioms
) unit test

other signs of strength:
) generics — creating generic classes
) serialization
) garbage collections
) inner classes
) collections and data structures — choosing

What about spring, hibernate …? I feel these fads come and go.

##java trading interview topics

* annotations? I don’t know why so often asked
* multi-threading
* concurrent data structures
* thread pool
* deadlock prevention
* deadlock investigation
* wait-notify, synchronized
* dependency injection. I don’t know why so often asked
* garbage collection
* singletons
* serialization, externalization(??), probably needed for stream messaging
* generics
* immutable
* deep copy — needed for serialization and cloning
* producer/consumer

##[10] hard CORE c++java skills across industries

— c++
void ptr
double pointer
ptr to ref; ref to ptr
ptr to method
new/delete global overload to intercept new/delete operations
scoped typedef, namespace
static initialization sequence

Note: MOM, spring, hibernate, data grid … aren’t core java and out of scope.


— Below are needed only in rare situations, IMHO
memory management — leak detectors, garbage collection, weak (soft) references, memory profilers,

jdk and cglib proxy
custom class loaders
byte code engineering
JVMTI — (eg jprobe) powerful tools to analyze production systems.
real time java

learning curve in AutoReo

(Written in May 2010)

I thought about some friend’s suggestion to “deepen your java”. Now I feel the more I complain about being slower than others and having a lot of difficulties learning the code base, the better!

Learning curve in the Autoreo team is steep, but project risk is not too high.

By contrast, in other projects I don’t feel challenged because I was so familiar with the technologies and tools. I won’t improve. One day I step into a big trading system, I would not be prepared for the complexities. Project risk would be too high – I won’t be given a lot of time to learn.

I don’t fancy learning java in a school or getting a Master’s. I worked for 12 years and studied by myself for a long time. I know what I need to learn – stuff used in real (imperfect) systems. So my current project is probably the best classroom for my personality. I’m grateful.