[20]what protects family livelihood: IV^GTD skill #AshS

Hi Ashish,

After talking to you about PIP (i.e. performance improvement process) in my past and in future scenarios, and considering my financial situation (wife not working, 2 young kids + 2 elderly grandparents) over the 20Y horizon , I came up with this question —

Q: Between two factors: AA) my competitive position on the tech hiring market, BB) job security at MLP, which factor has more impact on my family livelihood

Answer: I remain as convinced now as 10 years ago: AA is the dominant factor. I won’t allow myself to rely on some employer to provide my family a stable income for 20Y, even if I do a good job. There are countless stories among my peers who worked hard but lost jobs.

Answer (during covid19 mass layoff): I’m even more convinced that AA is the dominant factor. MLP is doing well, but MLP owner is not my dad. See my email to my sister sent on 19 Aug.

If I do a good job in the current team, what’s the chance of keeping this job for 10Y? 10%? There are individuals (like my manager) who stay in one team for 10+ years, but I think anyone like him has witnessed dozens of coworkers who have no choice but leave, for various reasons (not counting those who had a choice to stay but hopped higher elsewhere.)

That’s the basic answer to my opening question, but there are a few important sub-factors to point out.

Family livelihood includes housing, medical and education. In the U.S., I would incur $3k/M rental + 2k/M health insurance. Therefore, livelihood in the U.S. is more precarious, less secured.

My Health — is a big hidden factor. Stamina, mental capacity has a direct impact on our “performance” in the competition, both on job market and on-the-job. I think you probably need a lot of physical and mental energy, stamina,,, to deep dive into an unfamiliar local system or codebase, to become so confident, right?

company stability — is a sub-factor of BB. Some investment banks (GS, Barclays, MS) are known to aggressively cut headcount even in profitable years, to stay lean and mean.

Aging — is a factor affecting AA slightly more than BB. Age discrimination is something I seem to notice more as I grow older. So far I’m able to maintain my “cometptive fitness” on job market. If I rely on BB too much as I age, then I believe I would pay less attention to AA, and grow older and weaker. To strengthen the foundation of my family livelihood as I age, I tell myself to see the reality — as I age I would face a less friendly job market + instability on any job. Therefore I need to give priority to AA, by maintaining/improving my tech skills for competitive interviews.

Demand — for developers continue to grow in the job markets. This is a fundamental reason why AA is so valuable and reliable. This robust demand doesn’t help BB at all.

Overall, my view is biased in favor of AA. This is deliberate. With PIP or without PIP, any high-paying tech job (like yours or mine) comes with an expectation and risk of job loss. AA is the parachute.

— A follow-up mail
My employer is very successful this year. Nevertheless, I never assume my employer’s success, strength and stability gives me an iron rice bowl. Things can turn bad for any reason, at any time, but I always have 3 dependents to feed (my wife stopped working last year).

So my answer remains — AA is the dominant factor.

However secure my employer is, I am fundamentally insecure.  Reality is — just about every job is insecure. Paradoxically, the more we see through the smoke and mirror and recognize the inherent insecurity in any job, the more cool-headed we can become, the more likely we can find motivation to invest in our own competitiveness on the job market.

I don’t know why so few thought leaders point out the truth that’s fundamental in many careers — that tech interview skill is the most important survival skill for many developers like us, in good times or bad times.

What if you are a very senior develoepr, perhaps library developer, or some well-known open source contributor. I guess the next hiring manager would still use some screening test ! Your previous value-add is not easily portable, so you must learn a new codebase, and grow your value-add from scratch. Your value-add is what the next employer pays for.

automation scripts for short-term GTD

Background — automation scripts have higher values in some areas than others

1. portable GTD
   • Automation scripts often use bash, git, SQL, gradle, python/perl… similar to other portable GTD skills like instrumentation know-how
2. long-term local (non-portable) GTD
3. short-term local (non-portable) GTD

However, for now let’s focus on short-term local GTD. Automation scripts are controversial in this respect. They take up lots of time but offer some measurable payoff

• they could consolidate localSys knowledge .. thick -> thin
• They can serve as “executable-documentation”, verified on every execution.
• They reduce errors and codify operational best practices.
• They speed up repeated tasks. This last benefit is often overrated. In rare contexts, a task is so repetitive that we get tired and have to slow down.

— strength — Automation scripts is my competitive strength, even though I’m not the strongest.

— respect – Automation scripts often earn respect and start a virtuous cycle

— engagement honeymoon — I often experience rare engagement

— Absorbency — sometimes I get absorbed, though the actual value-add is questionable .. we need to keep in mind the 3 levels of value-add listed above.

@outage: localSys know-how beats generic expertise

When a production system fails to work, do you contact

• XX) the original developer or the current maintainer over last 3Y (or longer) with a no-name college degree + working knowledge of the programming language, or
• YY) the recently hired (1Y history) expert in the programming language with PhD and published papers

Clearly we trust XX more. She knows the localSys and likely has seen something similar.

Exception — what if YY has a power tool like a remote debugger? I think YY may gain fresh insight that XX is lacking.

XX may be poor at explaining the system design. YY may be a great presenter without low-level and hands-on know-how.

If you discuss the language and underlying technologies with XX he may show very limited knowledge… Remember Andrew Yap and Viswa of RTS?

Q: Who would earn the respect of teammates, mgr and external teams?

XX may have a hard time getting a job elsewhere .. I have met many people like XX.

measurable productivity@work, as a whip

https://bintanvictor.wordpress.com/wp-admin/post.php?post=31918&action=edit

Many managers use measurable productivity, and peer benchmarking as a whip.

big_gun + laser_gun on localSys

I like the imagery of laser gun and big gun

Backdrop — My brain power vs the brain power baked into the local system.

My laser focus is a blaster-gun that can break down the stonewall of opacity in the local system. This laser-gun doesn’t require “independence” but IFF I become independent early on, then I can make use of my weekend quiet hours and apply my laser. My weekend quiet hours are a real big gun in the local competition among the team members [1].

However, my quiet hours tend to get spent on research rather than localSys. See office-hour productivity#1 distraction==research 

I could also have an engagement honeymoon.

[1] My advantages and weaknesses are fairly obvious when compared to my colleagues.

Many of my peers seem to have that laser gun more often I had.

• ==== my victorious war stories were mostly in brown field arenas. In each case I needed the same sustained focus for a few months
• +ve xp: PWM error memos
• xp: PWM AICE
• +ve xp: RTS
• -ve xp: at mvea I was not independent.
• -ve xp of CSDoctor, but I won’t spend too much time on hearsay war stories.

plowback: zbs+GTD,shields

See also invest salary{peak earning phase

–update on plowback
I often feel the urge to plowback and strengthen the shields..

I think zbs and GTD (except tool knowledge) are overrated in terms of plowback. In contrast, IV body-building is a worthwhile plowback.
Industry-wide dnlg? Not so effective as a form of plowback.
fitness? Excellent Plowback
stock investment? Too time-consuming, not worthwhile.
parenting? Yes takes huge amount of time but 90% inefficient in terms of plowback
English (and Chinese) vocab?

..plowback my savings to buy a shorter commute
* increase nonwork income to support lease spread
* spend time (huge amount) helping kids’ studies
* save up for 43R model
* save up for a private school in an average school district

——

For many years I was deeply convinced and motivated by the notion that I ought to “plowback” for zbs (and, to a lesser extent, GTD) not just aim at job interviews as the Mithuns would.

After working for 20 years, I now believe ANY tech knowledge, accumulation, deepening/stack-up, sustained focus … has extremely low leverage and basically worthless if not relevant to Interviews —

• GTD skills? dominated by localSys. Tool knowledge can help but localSys is 10x more important.
  • localSys xx is almost always cursory, rather than in-depth since unnecessary digging (pwm-comm bottom-up xx plan) is invariably unsustainable — The local systems are patchworks , destined for rewrite, illustrating few (if any) best practices
• BestPractice design? Almost never mattered in my career. After EMPWorld, I have never risen to decision-maker level. This zbs is overrated by 10 times.
• BestPractice implementation? What’s “best” is mostly a matter of taste (personal preference) of the manager
• zbs? Consider the most practical books like [[Pearls]] — the classics. If not relevant to IV then this zbs is only for self-satisfaction

This is one reason why my reinforcement loop completely broke down in Singapore, despite the plowback into zbs, GTD etc.

… In contrast, my NY years were filled with joys of self improvement, driven by … interviews.

---

Now let’s switch gears and talk about other (important) forms of plowback.

— enough time for exercise is the best plowback

— stay relevant to the job market

— investment nest eggs

— annuities, shield plans

— spend plenty of time with kids, on academic learning and beyond.

[19] localSys proficiency]1st 4M #Macq

Arguably my #1 weakness is the initial learning curve on localSys. For that reason, My figure-things-out speed would be slower than other guys.

Due to my superior interview performance, I “sometimes” show underwhelming learning curve, depending on the other new hires. In some teams like RTS, Stirt, … some new hires were slower, less motivated, more distracted.

Camp-out — is my advantage, provided I have an engagement honeymoon.

self-independent learning — Ashish mentioned it many times. Not sure how realistic. If I can achieve it, I would have a big-gun, as described in big_gun + laser_gun on localSys

respect — Ashish pointed out independent discovery would win me respect from team members. I can recall many experiences to support it.

burning pleasure —

virtuous circle — the respect would spur me on.

annual leaves — It’s OK to take some leaves, but if I can spend more time at work in first 3-6 months I have a higher chance of creating the virtuous circle.

Q: At Macq, was the initial 6M ramp-up effective for localSys, GTD?
A: It did help prevent the Stirt mistake. Key is capture of absorbency and burning pleasure, in the engagement honeymoon. In 2015 I didn’t have a lot to capture 😦
A: Consider set-up-to-fail . I would say even with a quick ramp-up, I would still fall below the bar.
A: now i feel 6M is hard to maintain. Realistic honeymoon is 4M, but sometimes it lasts longer.

frequent rests@work → productivity drop

As I get older I believe my brain needs more frequent mini-breaks, but I think sometimes my breaks become distractions. The more treacherous “breaks” tend to be

• blogging
• personal investment — Eliminate !
• generic tech learning, unrelated to current project

— sense of urgency — When taking these breaks, A sense of urgency is what I need, but I favor a relaxed sense of urgency.

— focus — frequent mini-breaks should not affect my focus at work. I might need to avoid chitchats.

I think it’s possible to maintain or enhance my focus by taking breaks.

— stay late — Frequent breaks often require longer hours. I should prefer coming early, but in reality I often stay late.

— I also want to avoid long sit-down sessions. The breaks give relief to my eyes, neck, back, shoulder etc.

##Design competency based@localSys nlg

Have I blogged about this topic elsewhere? Perhaps some elements of this topic. Today I’m specifically addressing — Your competitive strength in tech designs based on localSys.

My self-assessment — I think I’m better than average with my design skill, assuming roughly equal level of localSys know-how. I’m more detailed, so I tend to take care of many localSys constraints.

Q: do other people see my strength in this space?
%%A: some colleagues do.  Some managers do, such as Yang, Sundip, Ravi, Anand, Saurabh (bbg module), Venkat@RTS (on config generator)

I basically chose not to care much about their appraisal. The risk is — The more I care about quality of my design and other people’s appraisal of it, the more contention, more emotional attachment

Therefore, My attitude — I put in (creative) effort but if they don’t like my design I won’t insist.

Nowadays I avoid design confrontation as I risk losing my reputation. I got my fingers burnt before, including Macq, GS .. when I cared too much about my designs. I think Rahul, Zhurong .. are unlike me. They put their heart into it … and cared about the designs.

— Scale — I usually refer to module-level, usually an enhancement or small feature. Bigger projects seldom come to me as a contractor or new-comer but here are some examples

• 95G: race condition control via stored proc
• 95G: wait/notify
• Barclays everything
• RTS
• SwapBust
• SIM trailer replacement for AISE
• mtg comm

— job hopping factor  — I feel I have benefited from frequent job changes. I feel I had more success stories to tell. However, due to limited localSys, I wasn’t confident/competent to create bigger designs.

— Strength/weakness view
I think my competitive weakness is brown field GTD — figure-out-speed, localSys learning speed, speed code reading …

In green field, or design projects, I’m not so weak, but not always strong either.

I think some guys can design against ambiguity more effectively.

## localSys note-taking: choices

Other choices include jira

				MSOL
	paper	Windows10 StickyNotes		MSOL Notes #can hide 🙂	Tasks	mail folders	recoll #1 default
Encourages Periodic refresh	AAA	unsuitable		B # everything  is temporary	C # “browse”	F #needs discipline	F #needs discipline
browse		A		B #click to see details	A	B #copy-edit subject	B #click to see details
searchable	F	unsuitable		unsuitable	A	D #too many	C
Organization of 30+ topics	Unsuitable	F #intrusive until hidden		D	C # categories	B	AAA # hierarchical
Radiator	AAA	B # Kyle use it long-term		F	F	D	F
Screen shot	F	F		F	A	A	B
ez2edit	B	A			A	Mostly readonly	A
locate by title +!search within small population	only if in front			A iFF tiny population	B #no tree but can put into one category	D #must search	B #filename not always fully used

camp_out]ff: %%coping strategy/strength

Exec summary: If I find myself engaged, I would want to spend extra quiet hours including weekends working in office and make progress on localSys. I would be able to apply myself and match my brain power against the accumulated brainpower baked into the local code base.

By “Camp-out” I mean all forms of extra time spent working in office. Camp-out is a concrete form of engagement. Usually the engaging period is a honeymoon but hopefully it can become a more sustained focus on localSys. If it does happen (I won’t push myself too hard…) I would hit better GTD, productivity, and earn some respect from some colleagues (not necessarily the boss).

More importantly, I would feel proud of myself and free of regret or guilt. See warning below.

Capture the mood. Eg: You get an engaging project that give you insight into portable GTD, tools, or localSys.

• Xp: Qz — I did capture the mood a few (like 3+) times. I felt really good even though the camp-out didn’t “save me” in the end. Like U.S. presidential election, the “end” result depends on many many things so it’s unwise to use it as a big paint brush and wipe out all the effort.
• Xp: Macq — I made big efforts to camp out to work on c++ build system and python. No regrets. The camp-out didn’t really build my foundation in localsys as the expectation was too high. No regrets!
• xp: OC — I did a few times, mostly to learn c#. No regrets.

— Warning: camp-out may not earn me respect, GTD, enough localSys mileage. Ashish feels I would survive.

I feel I may still get PIP, but after such an all-out effort, I would be decisive to leave. Consider Qz and Macq.

— I see this as my competitive strength and an effective coping strategy.

I used to feel I’m kind of inadequate if I have to camp out and sacrifice family time just to keep my head above water. Wrong! Most of my peers simply don’t have the absorbency, and productive solitude to do this. Many can’t even imagine it.

I said my son may have more or less math abilities, but his effort (chiefly absorbency) is probably lower than his classmates. My daughter’s abilities is unknown, but her absorbency is better than her brother. Similarly, my absorbency is higher than my peers as demonstrated by camp-out.

— Initially, May need 6M-9M to get over the initial hump. I can even rent a temp home for 6M if I feel engaged. See kids only on weekends.

— As contractor I feel the motivation but zero pressure to camp out; As employee I feel pressure to camp out. Clearly I feel better as contractor.

— Q: In my experience, part of the camp-out hours are often spent on blogging (flight control) and morale-boosting workout. OK?

— Q: However, if code is too opaque, I won’t be able to figure things out 😦 What can I do?

— Q: What if I receive a PIP.. will I be able to persuade myself to camp out?
A: Yes. I did in GS, Qz and Macq. No honeymoon, but I have been able to overcome the huge negative energy, convert it partially to motivation and get myself to camp out and focus on GTD.

I like this imagery — I felt like a boxer on my back-foot, not hopeful but not pessimistic either.

— Q: If in the mood for localSys but need to sleep? Shall i go home?
A: don’t go home to sleep. Sleep in office. Go home to eat then come back to office.

 

##strongER trec taking up large codebases

I have grown from a sysAdmin to a dev
I have grown from web and scripting pro into a java pro then c++ pro !
Next, I hope to grow my competence with large codebase.

I feel large codebase is the #1 diffentiator separating the wheat from the chaff — “casual” vs hardcore coders.

With a large codebase, I tend to focus on the parts I don’t understand, regardless that’s 20% or 80% of the code I need to read. I can learn to live with that ambiguity. I guess Rahul was good at that.

In a few cases, within 3M I was able to “take up” a sizable brown field codebase and become somewhat productive. As I told Kyle, I don’t need bottom-up in-depth knowledge to be competent.

In terms of count of successes taking up sizable brown-field codebase, I am a seasoned contractor, so I would score more points than a typical VP or an average old timer in a given system.

• eg: Guardian — don’t belittle the challenge and complexity
• eg: mvea c++ codebase — My changes were localized, but the ets ecosystem codebase was huge
• eg: mvea pspc apportionment/allocation logic + price matching logic
• eg: mtg comm (small green field), integrated into a huge brown field codebase
• eg: StirtRisk personalization — I made it work and manager was impressed
• eg: StirtRisk bug fixes on GUI
• eg: Macq build system — many GTD challenges
• eg: Moodles at NIE
• eg: Panwest website contract job at end of 2016
• ~~~ the obvious success stories
• eg: AICE — huge stored proc + perl
• eg: Quest app owner
• eg: error memos
• eg: RTS

burn-out: intensity^overtime #CNA

For 99.5% of us, high intensity is unsustainable almost by definition. I believe Intensity doesn’t equal GTD output, which is usually measured by some numbers, or subjectively by observers. As a student, I had a strong foundation (“abilities”), so a moderate intensity was able to create high output. Intensity factors include (not limited to)

• non-stop mental focus
• truly high-pace workplace
• creative engine or analytical engine
• deadlines
• multi-tasking and context switching
• quick recall of huge amount of details
• GS style time booking

https://www.channelnewsasia.com/news/commentary/working-relentless-pace-wont-help-career-prospects-10596240?cid=h3_referral_inarticlelinks_24082018_cna is a good online article

During summer, the firm works Just four days — a total of 32 hours. The company’s summer workload must fit reduced hours, Mr Fried insists. This employer doesn’t offer “flexible” hours in exchange for intensity

Employers and policymakers focus a lot on the excessive hours, but compared with overtime, work intensity predicts much greater reductions in well-being and career-related outcomes.

accumulated localSys Xp=brain stress relief@@ limited

The most common source of everyday hazardous stress on my brain is coworker benchmark by speed of figure-things-out on localSys[1].

My focus today is on this hazardous stress.

If you stay for x years, then the localSys experience would enhance the speed of figure-things-out, and reduce the hazardous stress on the brain. This accumulation takes place naturally, but you do need to /apply yourself/, otherwise you won’t know even the basics.

In rare cases, a bright new joiner can acquire better localSys knowledge than old timers. Eg: Avichal, Yang. Viswa

bigger picture — Josh pointed out that knowledge of the bigger picture greatly reduces the hazardous stress.

[1] Note when the figure-out is NOT about localSys, then I am NOT slower than colleagues. Among other things I’m good at online research and reading documentation.

On a side note, staying too long in one system could lead to boredom and stagnation, not good for the brain, but I believe this is not as harmful as the hazardous stress. Note by “boredom” I don’t mean the initial honeymoon of engagement, but rather the boredom after 5Y. Learning some new technology is one “easy” and safe way to keep the brain active and engaged. Relatively low expectation.

deadly delays@ project^feature levels

insightful article: managing tight deadlines is the background

— deadly delay at feature level, without user impact

This is real stressor primarily because there exist team colleagues who can GTD faster on this same task.

For the serious delay, user impact is … exaggerated! This is true across all my projects — user impact doesn’t really matter. In fact, the feature may not be used much whether you deliver it in high or low quality, within or exceeding budget.

— blatant delay at project level when you are architect / team lead

In theory if you promise to delivery some new feature i.e. green field project, then it can be tough to deliver on time. In reality, project time/budget overrun is very common. You only need good explanation.

Users never care that much about any new system cf current production systems. New systems often hit various teething problems and functionally unreliable for months.

OK , Users don’t really care that much, but there’s visible technology budget impact which make the technology MD look bad. MD must look for an “explanation” and may cut headcount as in StirtRisk and RTS-Trep

Whose fault? Many fingers point at you the architect, but often it’s partial fault of the user-facing manager due to immaturity and overconfidence with timeline estimate.

Delay is mostly architect’s fault if another architect can deliver faster but I can’t imagine two architects working on two comparable projects at the same time. Therefore it’s never “mostly architect’s fault”. In reality, management do apply pressure and you suffer, but it’s really for political reason (like sacrificial lamb or scapegoat)

eg: RTS Trep
eg: PWM Special-Investment-management
eg: Post-Saurabh GMDS — multiple delays and scope reductions

密码保护：##G5 tips: pain relief ON next sg job

此内容受密码保护。如需查阅，请在下列字段中输入您的密码。

密码：

highest leverage: localSys^4beatFronts #short-term

Q: For the 2018 landscape, what t-investments promise the highest leverage and impact?

1. delivery on projects + local sys know-how
2. pure algo (no real coding) — probably has the highest leverage over the mid-term (like 1-5Y)
3. QQ
4. –LG2
5. portable GTD+zbs irrelevant for IV
6. obscure QQ topics
7. ECT+syntax — big room for improvement for timed IDE tests only, not relevant to web2.0 onsite interviews.
8. Best practices — room for improvement for weekend IDE tests only, not relevant to web2.0 shops.

Q: passive income ⇒ reduce GTD pressure#positive stress

See also 3stressors: FOMO^PIP^ livelihood[def]

My (growing) Passive income does reduce cash flow pressure… but it has no effect so far on my work GTD pressure.

Q: Anything more effective more practical?

1. take more frequent unpaid leaves, to exercise, blog or visit family
2. expensive gym membership

How about a lower salary job (key: low caliber team)? No I still want some challenge some engagement, some uphill, some positive stress.

your brain power^brain power baked into localSys

Background – “how fast you figure things out relative to your peers”.

For each team member AA, the struggle is the same — AA’s brain power vs the cumulative brain power that has gone into the local system which measures the complexity. If the local system complexity is too high then AA would struggle and take a long time (before he gives up).

The “local system” could include firmwide frameworks, or something open-source.

I prefer a local system created by low site-specific brain power, like one with standard SQL/stored-procs, standard noSQL, standard data encoding (FIX, Json..), standard java/c++ libraries, including OSS.

• RTS and OC – relatively small amount of site-specific brain power in the system.
• PWM comm – actually small amount of local system complexity but time given is too short
• Barc – brand new codebase .. low site-specific brain power.
• Quartz — the worst

localSys learning: get over the hump

• eg: nyse xtap
• eg: OC Quest ownership handover
• eg: Same with my son’s piano … Same with my son’s math problem solving ..

Once I get over the hump … relief ! and then I tend to keep going for a few hours and gain more local system insight.

There are various signs of the hump…. It’s nice to notice “Hey, we might be at the hump”, so we could turn on “supportive self-coach”

• block out distractions/interruptions … put on earphone
• ask for help rather than pushing too hard on our own
• take frequent breaks with food or mini-workout
• recognize that even 30 minutes of immersion is tough, valuable and an achievement to be celebrated [1], with or without visible progress.

[1] I WILL reward my son for that.

localSys insight: Self-learning

Q: by correlating the logs, config, source code, queries … on your own [3] in your spare time [2], do you think you can gain insight into local system?

This question is at the heart of my GTD challenges, including code-reading weakness.

I don’t need to be super-fast to make progress.

[2] I often have some spare time, not a lot but a few hours a week. Sometimes my work hour is exactly allocated to this. In 2018, I have plenty of weekend hours 🙂

[3] self-learning — There’s a balance to strike between asking and self-learning. I’m reasonably good at guessing :).

I guess some developers rely on asking others and then making rapid progress. It has not been very successful for me, and I have not tried it many times. For me, it’s almost always a personal journey, not guided tour.

Proven GTD: worthless in candidate ranking #JackZ

I feel that Jack Zhang is competent with localSys GTD but weak on c++ and comp science.

Does he have working knowledge of c++? I assume so. Working knowledge is attainable in a couple of months for a clean language, and up to a year for c++

The required level of working knowledge and basic skill is very low for localSys GTD.

His c++ knowledge is probably barely enough to do the job. Remember I didn’t know what things live on java heap vs stack.

Based on my guesstimate, he would fail any c++ interview and any algo interview. He can write simple SQL in an interview, but I am not sure if he can write complex joins.

The fact that Jacn and DeepakM are are proven on GTD is useless and lost in the conversation.

How about CSY? He can solve many algo problems without practice, but he is reluctant to practice.

I think the self-sense of on-the-job competency is misleading. Many in their positions might feel GTD competency is more important than IV skills. They are so afraid of the benchmark that they don’t want to study for it.

When the topic of tech interview comes up, I think they wish to escape or cover their ears.

[10] Can interviewers see through our soft skill weaknesses@@

Now I feel even experienced interviewers (I had chitchat with some recently) can only judge a candidate’s non-technical qualities to a very limited extent.
Experienced interviewers try to focus on “thought process” and “problem solving skill” (soft skills) rather than practical experience and textbook knowledge. White boarding, expanding a given problem’s scope — “what if we need …”, “What if we can’t …” … By the way, my Google interviews are all of this type. Heavy on algorithm and almost language-agnostic algorithms. Some of my most in-depth interviews with good companies seem to follow similar patterns. These are not designed to be traditional technical screening as they test analysis and problem solving, under pressure.

In addition, Many interviewers ask about java’s fundamental design principles, though we developers seldom need to know those. Examples include those wildly crazy questions on hash table implementations, wait/notify, equals() and hashCode(), generics implementation in Java 1.5, beauty of dependency injection, reentrant locks .. These questions are less about “productivity/competency” or design skill and more about “depth of understanding“. Perhaps they want to know how deep the candidate thinks. I consider these non-technical qualities.

Obviously interviewers watch out for personality issues like arguing, strong opinion, arrogance, refusal to admit defeat, insufficient give and take, lack of confidence… Luckily none of my serious weaknesses are visible to them — such as my weakness in office politics.

Taking a step back, if we analyze why a person A fails on a job, obviously a lot of responsibilities are on other people. But if we focus on the weaknesses in A herself and try to classify the common weaknesses in workers, very loosely i see 2 types
1) a range of technical weaknesses
2) a range of non-technical weaknesses

I feel the art and science of candidate screening has advanced more on the first front then the 2nd front. Sharp interviewers can now find out technical weaknesses more than they can non-technical weaknesses. I’m talking about things like real (not fake) honesty, ownership and initiative, efficiency, follow-up, prioritizing, sense of urgency, can-do attitude, client-service attitude, dedication, knowledge sharing, helping out, give and take, push-back, persistence and determination, respect for cultural differences, bias, fairness, attention to detail, code testing attitude, ethical standard on code quality, professionalism, self-starter, motivation, hard working, personal sacrifice … ( … some of my strongest and weakest parts!)

I think these are important to a hiring manager and can break or delay a project, but these personal qualities are invisible to even the smartest interviewers I can imagine. However, I was told some interviewers can be unbelievably smart, beyond my imagination. I tend to feel a perceptive person can be very sensitive but she can’t be sure about personalities based on a brief conversation.

My conclusion — once you pass the technical screening, then interviewers can’t easily find out your relative weaknesses in communication, personality, attitude .. provided you don’t make an obvious mistake.

RTS: Don’t rely@QA2test your code@@

insightful article: managing tight deadlines is the background

My ICE manager said "Don’t rely on QA team to test your code. Do your thorough testing before passing to QA."

This is Impractical given the thousands of detailed items in a 50-page spec. Thorough developer testing would take more time and I didn’t do it. A colleague was able to give QA team a well-tested rpm and received only 30 bug reports (mine received 60 to 100), but did it win him any award? No.

• I could explain my requirement is more complex. Manager may or may not accept.
• Sometimes we could explain one of us is new
• Which system has more crashes in QA/production? Each crash can take a long time to figure out. Thorough developer testing doesn’t help much here.
• In the end, which system goes live faster? That’s the visible yardstick. 2nd visible yardstick is how quickly we start QA, so by cutting corners on developer testing, I was able to start QA earlier. A more thorough developer test won’t help with any of these visibles.

susFocus^engage^absorbency^visPgress^traction #Clarified

I hope to differentiate these related terms, and reduce proliferation of tags

• traction — is kind of vague and high-level, so no “t_traction”
• traction — often describes learning curve breakthrough. Defined in [11]traction(def)^learning curve gradient #diminishing ROTI
• visPgress — is more concrete and well-defined than “traction”
• (dis)engaged — describes a mental state, like “absorbed”, “enchanted”
• (dis)engaged — can be felt only from inside
• t_distract — specific distractions, often short-term like kids, e-banking…
• sustainedFocus — is often required to overcome tough learning obstacles
• sustainedFocus — can be sustained by self-discipline (absorbency) whereas engaged is often due to luck and sustained interest.
• sustainedFocus — is more a specific form of being “engaged”
• sustainedFocus — is a longer phrase compared to the honeymoon of engagement
• absorbency — is most specific
• absorbency — describes the difficulty of staying engaged despite dry, repetitive, focused learning
• t_focus@GTD tag and gzGTD category are broadly similar

Grandpa is a model of

• engaged
• sustainedFocus — but without too much self-discipline needed
• absorbency

In comparison to grandpa,

• I want to have more sustained focus
• I want longer engagement, because I tend to lose interest too soon
• I know my engagement lasts only hours, so I frequently feel a bold justification to capture the moment, perhaps at a financial cost
• my absorbency capacity is not as good as I wish
• i feel a real need to capture commute time
• I need a job that gives me more personal time, even during work hours.
• … I think these factors have profound consequences, such as my career decisions

expertise ≠ GTD@localSys

Unlike GTD, zbs is indeed expected of you if you are a true expert.

[17]productivity@past jobs: questionable patterns

Be careful with the assessment — perhaps I was not so productive on my past US projects
Be careful with the assessment — perhaps I was not so un-productive on the q3SG projects

Consider deleting this blogpost completely.

Pattern – Barc, ML … were java, my turf
… but Citi was java as well. My java skill did help, but I didn’t make use of my strongest java skills.
… but at GS, Perl was my turf too. My Perl experience did help but the colleagues were too strong.

Pattern – Barc, ML … not many peers to compare with me
… but Chien was a direct peer. I think Rao also.
… but later stage GS, I was benchmarked

Pattern – OC, … familiar with the codebase
… but OC codebase was rather big.
… but Stirt codebase wasn’t big.

Pattern – GS, OC … spent long time on the existing codebase. In contrast, Creating code from scratch is easier. … but Stirt was code from scratch

##how fast I figurED things out relative2teammates

GS — Initially I was much slower than my colleagues. After 18 months I was 80% as fast as them. Everyone needed to learn and memorize lots of details in a big system. Without that “mileage” you can’t be as fast as the rest.

I relied on my colleagues to give me pointers and detailed explanations. Otherwise it was too many details to wade through by one self.

Citi — no comparison — No “team mates” per-se. I had two old-timer colleagues who were very familiar with the system. I was not benchmarked against them.

I didn’t need to understand the ins and outs of the 5-year old codebase. I never became productive with it.

95G — not slower — i joined this greenfield project from beginning (luckily). I was as fast as the other consultant. Also, I designed a few key components, so I had the “original designer” advantage.

Barclays — not slower — i joined this greenfield project from very beginning (luckily), replacing a half-finished c# system. I had very few “team peers” to benchmark against. I designed one of the most valuable and complex components, so I had the “original designer” advantage. Huge advantage. I basically name my price when it came to effort estimates.

I did read some old code to replicate the logic, but I was fairly fast, and no team member ever read the same codebase anyway.

OCBC — not slower — I was seen as slower, but actually I became 70% to 90% as fast as the other team members. I had to spend many long hours on the existing codebase but it’s a small codebase (1 man-year) so I was able to get a hang of it over a few months.

I also designed some minor modules, where I had the “original designer” advantage. The other c# developers had to come to me and understand my designs.

Stirt — I was much slower than the old-timers, but I was not slower than the new joiners. I should have given up earlier. It was hopeless. Codebase was rather opaque and you need to ask many people to understand it.

Other new joiners seemed to ask only within their own team, but I stick out my neck and asked all around my team. It may have made me look slower and /greener/, but I was trying to learn faster. Did I learn faster than them? Not enough time to show.

Macquarie — The benchmark was my predecessor + my boss. I think I was much slower than them. My boss only complained about it in the 2nd year, months after paying me a small bonus.

RTS — not slower — I’m probably faster than some team members. I feel confident how fast I can figure things out now. There are many team members to compare, so from Day 1 it was crucial to hit the average standard.

%%3 GTD concerns as lead dev #instru/code reading..

Below are my biggest concerns when considering a lead dev role. Actually, many of my peers often hints “My role (as a mgr, or architect, or app owner..) is even harder” but I think they are Not more effective if asked to play the hands-on developer role.

1. code reading — requires focus and stamina.
   • In addition, taking ownership of a big code base requires determination and confidence
2. instrumentation — using prints or special tools, troubleshooting, error reproducing, ..
   • ##transparent^semi-transparent^opaque languages
   • %%GTD xp: 2types@technical impasse#难题
   • ## 2 heaviest work stressors 
3. pressure due to major feature not completed by impending deadlines
   • ##a few projects technically too challeng` 4me

Note none of these is tested in any interview!

• How about firefighting? I tend to panic but my peers aren’t stronger than me technically!
• How about design? Not really my weakness. My peers’ designs are not necessarily better than mine
• knowledge of generic tech (rather than local system knowledge)? I’m actually stronger than most peers
• quality, attention to details? is usually my strength
• fine tuning? Most peers are not good at this

(not ranked)		bonus	peers are ..	generic or local nlg?
code reading		no impact	some are stronger	local sys nlg
investigate		boss needs answers	usually no diff	local
deadline		yes	?	local
firefighting		yes	not stronger	local
design		somewhat visible	not stronger	some generic

[19] perceived productivity: 5factors #Kyle

As I discussed with Kyle, a worker’s sense of productivity is complex and subtle, far from objective. It’s influenced by many factors.

The sense of productivity is an important part of job satisfaction, sense of inadequacy + inferiority, duty ^ guilt, self-hate ^ compassion.

I tend to use this incredibly vague concept as a whip to beat myself up. I guess some managers (Mark, Venkat, Stirt) also use it on me…. harsh.

• It depends on the visible progress on project assignments. Projects are stuck frequently– like wheel-spinning. eg: StirtRisk projects were often stuck and I felt less productive. I had a short-n-sharp discussion with Rahul, who identified context-switching as the productivity killer
• It depends on visible impact on users — the #1 most visible sign of value of my output.
  • Eg: RTS, 95G, mortgage commissions, Quest,
  • proficiency in prod support has big impact. It is seen as low value, associated with ops team in Barc, 95G, Citi, but perceived as high value in GS
• It depends on localSys + understanding of user’s logical bubble. Those developer in close contact with users would have better gauge of which to-do (and which solution) scratches a big itch. 四两拨千金, 事半功倍.
• It depends on Perceived impact of my output relative to coworkers
  • eg: outstanding in Barclays
• It depends on our personal (highly inaccurate) estimate of how many focused hours/day. If “visible” items unavailable, then this is my default basis.
• It depends on My vague perception of coworker productivity. Kyle felt the perception is not that vague.
• It depends on How early coworkers come and leave; Coworkers working-from-home. Note Many hiring managers say (honestly) that if an efficient worker can get things done in time, then boss doesn’t mind shorter hours or WFH.
• It depends on How much leisure time I see in coworkers’ day, including chitchats
• It depends on Role model set by manager. Eg: Saurabh often took Friday as a lite day.
• It depends on User’s feedback on my output, often in brief emails — RTS
• It depends on Coworker feedback on me – OC, Macq
• figure-out-by-self — is a controversial yardstick. In most places (Ashish) it is a top 5 sign of productivity, but in GS I was told to not waste time researching. My GS colleagues always ask each other and I had to.

tags: respect^bmark^GTD^localSys

These tags/categories overlap. Not a big problem.

• respect — respect from MANAGER only. Synonyms include approval
• coworker benchmark — usually the main reason for PIP, though Macq is exception
• GTD — the basis of coworker benchmark
• localSys knowledge — the #1 source of GTD power

gdb skill level@finIT

I notice that, absolutely None of my c++  veteran colleagues (I asked only 3) [2] is a gdb expert as there are concurrency experts, algo experts [1], …

Most of my c++ colleagues don’t prefer (reluctance?) console debugger. Many are more familiar with GUI debuggers such as eclipse and MSVS. All agree that prints are often a sufficient debugging tool.

[1] Actually, these other domains are more theoretical and produces “experts”.

[2] maybe I didn’t meet enough true c++ alpha geeks. I bet many of them may have very good gdb skills.

I would /go out on a limb/ to say that gdb is a powerful tool and can save lots of time. It’s similar to adding a meaningful toString() or operator<< to your custom class.

Crucially, it could help you figure things out faster than your team peers. I first saw this potential when learning remote JVM debugging in GS.

— My view on prints —
In perl and python, I use prints exclusively and never needed interactive debuggers. However, in java/c++/c# I heavily relied on debuggers. Why the stark contrast? No good answer.

Q: when are prints not effective?
A: when the edit-compile-test cycle is too long, not automated but too frequent (like 40 times in 2 hours) and when there is real delivery pressure. Note the test part could involve many steps and many files and other systems.
A: when you can’t edit the file at all. I have not seen it.

A less discussed fact — prints are simple and reliable. GUI or console debuggers are often poorly understood. Look at step-through. Optimization, threads, and exceptions often have unexpected impacts. Or look at program state inspection. Many variables are hard to “open up” in console debuggers. You can print var.func1()

 

%%GTD xp: 2types@technical impasse#难题

See also post on alpha geeks…
See also post on how fast you figure things out relative to peers
See also ##a few projects technically too challeng` 4me
See also https://bintanvictor.wordpress.com/2017/03/26/google-searchable-softwares/
see also https://bintanvictor.wordpress.com/2017/05/29/transparentsemi-transparentopaque-languages/

tuning? never experienced this challenge in my projects.
NPE? Never really difficult in my perience.

#1 complexity/opacity/lack of google help

eg: understanding a hugely complex system like the Quartz dag and layers
eg: replaying raw data, why multicast works consistently but tcp fails consistently
eg: adding ssl to Guardian. Followed the standard steps but didn’t work. Debugger was not able to reveal anything.
Eg: Quartz dag layers
Eg: Quartz cancelled trade

#2 Intermittent, hard to reproduce
eg: Memory leak is one example, in theory but not in my experience

eg: crashes in GMDS? Not really my problem.

eg: Quartz preferences screen frequently but intermittently fails to remember the setting. Unable to debug into it i.e. opaque.

effi^instrumentation ] new project

I always prioritize instrumentation over effi/productivity/GTD.

A peer could be faster than me in the beginning but if she lacks instrumentation skill with the local code base there will be more and more tasks that she can’t solve without luck.

In reality, many tasks can be done with superficial “insight”, without instrumentation, with old-timer’s help, or with lucky search in the log.

What if developer had not added that logging? You are dependent on that developer.

I could be slow in the beginning, but once I build up (over x months) a real instrumentation insight I will be more powerful than my peers including some older timers. I think the Stirt-tech London team guru (John) was such a guy.

In reality, even though I prioritize instrumentation it’s rare to make visible progress building instrumentation insight.

[17] 5 unusual tips@initial GTD

See also https://bintanvictor.wordpress.com/wp-admin/edit.php?s&post_status=all&post_type=post&action=-1&m=0&cat=560907660&filter_action=Filter&paged=1&action2=-1

* build up instrumentation toolset
* Burn weekends, but first … build momentum and foundation including the “instrumentation” detailed earlier
* control distractions — parenting, housing, personal investment, … I didn’t have these in my younger years. I feel they take up O2 and also sap the momentum.
* Focus on output that’s visible to boss, that your colleagues could also finish so you have nowhere to hide. Clone if you need to. CSDoctor told me to buy time so later you can rework “under the hood” like quality or design

–secondary suggestions:
* Limit the amount of “irrelevant” questions/research, when you notice they are taking up your O2 or dispersing the laser. Perhaps delay them.

Inevitably, this analysis relies on the past work experiences. Productivity(aka GTD) is a subjective, elastic yardstick. #1 Most important is GTD rating by boss. It sinks deep… #2 is self-rating https://bintanvictor.wordpress.com/2016/08/09/productivity-track-record/

##transparent^semi-transparent^opaque languages

With a transparent language, I am very likely (high correlation) to have higher GTD/productivity/KPI.

Stress — correlates with opacity. Would I take a job in php, SQL, javascript, perl?

Confidence to take on a gig — The more transparent, the higher my confidence — like py, javascript

Bootstrap — with a transparent language, I’m confident to download an open source project and hack it (Moodle …). With an opaque language like C++, I can download, make and run it fine, but to make changes I often face the opacity challenge. Other developers are often more competent at this juncture.

Learning — The opaque parts of a language requires longer and more “tough” learning, but sometimes low market value or low market depth.

Competitiveness — I usually score higher percentiles in IV, and lower percentiles in GTD. The “percentile spread” is wider and worse with opaque languages. Therefore, I feel 滥竽充数 more often

In this context, transparency is defined as the extent to which you can use __instrumentation__ (like debugger or print) to understand what’s going on.

• The larger systems tend to use the advanced language features, which are less transparent.
• The more low-level, the less transparent.

–Most of the items below are “languages” capable of expressing some complexity:

• [T] SQL, perl, php, javascript, ajax,
• [T] stored proc unless complex ones, which are uncommon
• [T] java threading is transparent to me, but not other developers
• [S] java reflection-based big systems
• [T] regular c++, c# and java apps
• [O]… but consider java dynamic proxy, which is used in virtually every non-trivial package
• [T] most python scripts
• [S] … but look at python import and other hacks. Import is required in large python systems.
• [O] quartz
• [S] Spring underlying code base. I initially thought it was transparent. Transparent to Piroz
• [O] Swing visual components
• [O] C# WCF, remoting
• [T] VBA in Excel
• —-below are not “languages” even in the generalized sense
• [S] git .. semi-transparency became stressor cos KPI!
• [O] java GC … but NO STRESS cos so far this is not a KPI
• [O] MOM low level interface
• [S] memory leak detectors in java, c#, c++
• [O] protobuf. I think the java binding uses proxies
• [T] XML + simple SAX/DOM
• [S =semi-transparent]
• [O =opaque]

 

zbs/GTD/KPI/effi^productivity #succinctly

See zbs^GTD^QQ

• zbs — real, core strength (内功) of the tech foundation for GTD and IV.
• GTD —
• KPI — boss’s assessment often uses productivity as the most important underlying KPI, though they won’t say it.
• productivity — GTD level as measured by __manager__, at a higher level than “effi”
• effi — a lower level measurement than “Productivity”

##google-searchable dev technologies(!!Qz):lower stress

This topic is Rather important to my stress level, my available time for learning, my available time for family….

For example, with Quartz, I must ask around and experiment (millions of times). Slow to build a coherent understanding. Slow ramp-up. (In contrast, with Python I could do that by reading good articles online.) So my productivity lag/gap remains even after a few years.

Other Negative examples – Tibrv, Autosys, Accurev, Less-known boost libraries,..

MSVS? Most of the search results are about c#, so it’s somewhat harder to solve problems.

Eclipse CDT? Most of the search results are about Eclipse java.

Positive examples – vbscript, DOS batch,

Yet, this stressor is mild compared to “performance warnings”.

stress@jobHunt^GTD

Both mental stress and physical stress. Let’s take a step back and compare the stress intensity during job hunt vs GTD stress on the job.

Many people say it’s too stressful and tiring to keep interviewing compared to a long-term job in a big company. Well, I blogged many times that I need to keep interviewing…. The stress is so far manageable.

On a regular job, the GTD stress levels range from 5 to 7 on a scale of 10 (Donald Trump on women;). Often rise to 8.

Became 9 or 10 whenever (as FTE) boss gave a negative feedback. I know from several past experiences. In contrast, contract projects felt much better.

(To be fair, I did improve after the negative feedback.)

During my job hunt including the challenging Singapore lap, my stress level felt like 4 to 7, but often positive stress, perhaps due to positive progress and positive energy.

Conclusion — I always felt more confident on the open market than recovering from setback on a job.

assessing GTD competence with c++/python etc

See also tag t_GTD_zbs

Q: What are some of the yardsticks to assess someone’s practical GTD competence in a specific technology like c++?

Note every time I say “practical”, there’s actually some theoretical knowledge required, so a guy without a middle-school education probably can’t do a good job.

10 fundamental factors:

1. Local system knowledge

2. Troubleshooting

3. instrumentation tools and other tools

4. Devops tools for the chosen language – IDE, build, automated test

5. 3rd-party libraries in the chosen language

6. Only a small subset of theoretical knowledge is relevant to GTD

7. Practical architecture and low-level design

8. Practical tuning in the chosen language

Here are some specific areas to assess a candidate’s GTD competence:

· Finding the best way to make a (localized) change in the local system

· Practical abstraction and refactor to avoid code duplication

· (Devops) Practical error reporting

· (Devops) Practical logging and log management

· (Devops) monitoring, profiling

· Architecture to support multi-module development, often requires a language-specific architecture

upgrade c++GTD 6→7 #j=8

Focus here is zbs not IV.

I told XR that a few coding interviews have brought my c++ GTD by half a point. I felt it was still at 5 as of 2015. The Mac job didn’t bring it higher, but the ICE job did. As of late 2017, it’s at 6. It’s feasible to bring it up to 7 —

• reading code
• gdb
• linker, compiler
• basic threading
• basic templates? LG2

##waste time – java features needed4GTD !! IV

These java features are powerful and practical at work, but seldom asked in IV, so ROTI is lower during job hunting.

reflection
bytecode engineering
dynamic proxy
annotations for code generation
generics
serialization (Chi-hao)
NIO

##a few projects technically too tough4me

See also https://bintanvictor.wordpress.com/2017/05/29/transparentsemi-transparentopaque-languages/

I was technically very, very confident in my 20’s to early 30’s until I was bogged down in some tough projects. I think many successful and competent techies experience the same at least once in their career. They though they were powerful, invincible but then a tough project was given to them that they struggled to get done. Later someone else got it done without much effort. (For a public domain example, look at the kernel project in GNU.)

It can be harmful to dive so deep into past personal limitations but then a hard, honest review could be life-enhancing. I see myself as a tough grown-up guy so I can take the pain.

• Eg: GS rule-engine re-architecture in 2007 — too early
• Eg: code generator in Mac Quant team
• eg: see the opacity issues in https://bintanvictor.wordpress.com/2017/06/12/xp-3typestech-zbs-challenges/
• eg: the mysterious “tp” in the g++ command line when I run “make debug”

X years needed2become effective java GTD guy

Update — It would be good to have some measurable yardsticks for GTD competence. Right now the g_GTD_zbs tag includes at least 50 blogs.

XR, (letter sent in July 2010)

Here the goal is not getting the job but keeping the job. You said there’s a difference between a knowledgeable java student and an experienced java developer.

I said I only did java for 3 years.. well maybe 4. I think much of that time I was doing repetitive work, not learning. About half the time (1-2 years) I was learning, by experimenting, reading, discussing, debugging… (I actually learned a lot of java from our long phone conversations.)

I feel if a bright young guy is given a good training project, then in 6 – 24 months he might be able to reach my level of java proficiency.

I also said a lot of young coders could become faster than me with a specific java task after a few months of java learning. Well … an apprentice carpenter could become faster than his master at sawing along a line, but can’t replace the master completely. I feel the hundreds of decisions made each week by an experienced java developer are often based on more experience.

C# is probably same thing – 6 to 24 months to become effective. A very experienced c# friend told me “3 months”. I spent about 6 serious months and another 12 repetitive months on c#…

Venkat, one of the fastest-learning developers I have worked with, said (in Lau Pa Sa) “To get really competent with a new language (like java, c#, python) honestly you need at least one year.” Venkat had the strongest C++ skill I know. I witnessed how he picked up c#. He later struggled a bit with java.

The 64 million dollar question is, what are the really effective learning strategies. I don’t have good answers. Guo Qiao was an advocate of programmer online learning. Some people even suggest taking part in open source projects, to learn from practicing master programmers… I’d love to hear your input.

Thanks.

##most used(GTD+)Generic know-how4WallSt developers

#1 java — the ecosystem
#2 c/c++ — not yet my chosen direction
#3 dotnet including c#, WCF, remoting, windows debugging …
#4 py — purely automation in most places(easy instrumentation); advanced py systems in a few places

# unix power user tools – including scripting, instrumentation… More widespread but depth is seldom required compared to SQL and python. More well-defined as a tech skill than windows dev.

# SQL/stored proc — losing pole position in low latency and big data environments, still dominant in traditional apps

# Excel add-in and automation

# javascript

warehouse your curiosity google-search,esp.during work hours

Many coworkers (including my managers) have a habit — immediately search on-line when a particular news catches their attention. Or it could be a name, a product, a jargon, a fact, a number, a question, something search-able, but something not related to their immediate task and don’t require immediate investigation. If we warehouse the curiosity, very often we realize that topic is not worth searching. This is similar to a help desk’s unpublished policy:

Depending on the urgency, wait N minutes before reacting to a request. When you do reach out to the user, she will likely need it less urgently than if you ask her now.

Statistics — when I look back at my google searches, most of them are low value and no urgency. I wonder if you can ask google to show your own search history.

This rule requires self-discipline, something most adults and kids lack.

This rule applies not only during office hours. It applies whenever you need concentration. Non-trivial learning is one major area affected. This is esp. important when we find ourselves in the absorbency/engagement mode. Therefore,

… need to train ourselves to identify it, like identifying fatigue while driving.

How do we warehouse?

• Write it on a piece of paper
• git blogging
• take a picture
• voice recorder

establish tech stronghold(GTD+nlg)]each team

Note – focus is job performance, not IV.

To survive in a team,need unique technical specialty in a key area.

With the possible exception of the Citi Muni team, I have not seen a banking IT team to tolerate a “dead weight” guy. To do reasonably well in any financial IT team, I often need some “hard” strength (like an subject matter expertise), not just soft skill. If within your team you have a unique technical strength in a complex and important sub-system, then you can “name your price”, and you are kind of irreplaceable. Perhaps a go-to person. Some team members don’t have any expertise but survive on strong relationships. I’m unable to do that. In the past teams, I think many team members have such a strength. If I don’t have it, my position would be less established, less solid.

• 😦 😦 stirt? Worst experience. Was good at the insignificant preferences framework; canceled trades
• 😦 GS? Uncomfortable position. My Perl was not top notch. My java was below Yang. I became good at Error Memos (EOS) and trailers/upfronts
• 🙂 OC? was good at Bloomberg adapter and Guardian builtin web server to show the log files
• 🙂 🙂 95 Green? My DB design was convincing to Ravi. My wait-notify based design was pretty hard to match.
• 🙂 Barc? FMD, fmath
• 😦 Macquarie? In the quant team, whatever I accomplish seems trivial to manager, since he’s too strong. I’m seen as very slow on small tasks.

%%strength: GTD with non-trivial research/innovation

If the research is too easy, then other developers can also do it, often faster.

If the research takes years, then it’s really for a researcher. I’m not a researcher (yet)

The project could have high value!

eg: Barcap proj
eg: b2bTradingEngine
eg: bloomberg adapter in GMDS
eg: home-made web server using WCF

##[18]tips: increase lazer@localSys #esp.SG

• volunteer to take on more prod support, selectively.
• accumulate some email threads and go through some relevant ones with a buddy once a week, but need to be strategic and get manager buy-in
• reduce distraction — muscle-building. Tough but doable.
• reduce distraction — FSM. Rationalize time spent on FSM, until I’m over the hump
• reduce distraction — kids. Rationalize time spent on boy, since it didn’t work well. I think I can bring him out less often, since outing is way too costly in terms of my time.
• camp out and focus on 1) project delivery 2) localSys
• absorbency — when you are in the mood to learn local sys, increase your absorbency capacity with good food, exercise etc
• sunshine — is needed on localSys tough effort. My son’s effort need the same sunshine

label: cope^pain^threat^GTD

I would say choose one among

gzCope, gzPain, gzThreat, GTD

In rare cases, use simultaneous categories.

GTD is more specific than Cope. In a sense, all GTD posts are also part of Cope but actually Cope is more about job market, moving up, choosing specializations.

[17] GTD=reasonable ] stirt/OC/Macq

Update — In ##what actually determined%%past job satisfaction  I concluded that respect(zbs) was the #1 determinant of my job satisfaction. In these 3 Singapore jobs, my efficiency and zbs were insufficient compared to whatever their benchmark is. Even in java, theoretical, complex domains, without distractions … my colleagues could be superfast. Even if I get a lot done, I can still be seen as under-performing — low respect low bonus. Consider GS.

Update — my RTS GTD/effi was lower than Singapore, much lower than Stirt and Macquarie.

It’s kinda hard to measure 2 developers’ output. Given my background within my past teams, I now feel I did reasonably well on GTD, not super productive or super slow. Some periods in OC and Stirt were highly productive. My productivity was not excellent in Citi, Verizon, even 95Green, not higher than in Stirt. Conclusion — my GTD productivity self-assessment is swayed by my income and boss’s assessment.

• In OC, I took care of Guardian, Quest, Excel interface, Bloomberg and other part of GMDS. I wrote the web-based log viewer in WCF…. I think my OC 1st-year GTD productivity was reasonable, probably higher than Citi, and not lower than barcap.
  • The donut bonus was at least 50% due to manager relationship.
  • In contrast, at 95Green and Barcap my GTD was not much higher, but the value of my output was higher than team peers. I spent my spare time on c++, swing, c#, quant…
• In Stirt and Macquarie, manager felt I was good enough to be transferred to other teams — sign of respect
• Stirt — I had lower value-add than the veterans, but probably higher value-add than most new joiners. Technically I was rated competent by everyone, otherwise they wouldn’t set up the transfer. My Stirt GTD effi was clearly higher than at citi and not lower than Barcap or 95G. Perhaps only lower than GS.
  • One /speed bump/ was the perfectionist team lead who approves everyone’s commit
  • Another speed bump was lack of online resources for hacking (i.e. self-study).
• Macqarie role has very high requirements in terms of communication, leadership, but in terms of technical expertise I was clearly appreciated, otherwise they wouldn’t hint the internal transfer, and they wouldn’t pay me a $10k bonus.
  • Kevin criticized my effi but given his high standard on quality, my effi was higher than Citi and OC, and probably not lower than barcap.
  • ##enough localSys⇒GTD #but respect≠guaranteed has many details about Macq

[15] a G3 WallSt survival KPI:how fast U figure things out, relative2coworkers

As stated in other posts, it’s all about local system knowledge + tools

Survival (and bonus, stress..) is always linked to benchmark against team peers, not external teams like another department, not the manager himself. If entire team is low-calibre, then it’s a problem for the manager, but rarely your own problem. (It might threaten your team’s survival…)

LG2: system performance. Most guys in the team probably aren’t so good at this kinda thing, unless the  team has a specific mandate to monitor and fix performance.
LG2: test coverage. May affect your reputation, but once I figure things out I can often relax and test
LG2: code smell. May affect your reputation but is clearly a LGlp compared to getting things to work.
LG2: code quality
LG2: readability by the team lead. OK team lead may complain, but if it works then it’s relatively easy to improve it.
LG2: extensibility. Yang is big on this. Many books are big on this, but it’s an art not a science. Many team members probably aren’t good at it.
LG2:  system stability such as occasional hangs. Often a non-show-stopper.
** eg: my VS2015 build tends to hang and I had to research on how to fix it — show-stopper.

All of the above are secondary to … “figuring things out” i.e. how to get something to work, at a minimum usability.

Design? See also posts on arch_job. Can be a real problem, because if your design gets shot down, you waste time on rework.

 

 

IV^GTD – grow as techie@WS

I want to grow stronger/broader/versatile as a survivor, not necessarily grow my value-add. Actually I don’t have to grow.
— IV skills – Compared to GTD skills, these skills give me more confidence, more protection, more stress-relief. It works in big job markets like Wall St.

Lots of theory, which is my sweet spot.
Selective on what IV topics to learn!
coding IV + algo — favored by SiV
— (portable) GTD skills
Lots of tools….
Selective on what GTD topics to learn!

Needed for a lead developer, but such a role is stressful. I guess some good lead developers are also good at IV, but I’m not sure and I assume I’ll not be good at both.

Warning – a lot of projects don’t provide real GTD enrichment. Eg: Quartz, tibrv wrappers, Gemfire wrappers, FIX wrappers
——-
Macquarie environment lets me learn lots of GTD skills.
OC gave me IV (c#) and GTD enrichment.
Stirt – none!

A java environment would give me some additional GTD enrichment but less IV enrichment

In SG, I continued my previous strategy, learning IV skills + GTD skills. Not effective so far. I feel my c# IV skills improved a lot but still not there. C++ IV skills didn’t improve much partly due to distractions.

 

##c++(GTD+)learning-aids to upgrade 6→7

Which yardstick? First must be IV. 2nd would be common (not the obscure) GTD skills

For IV, need to analyze how I was beaten in past interviews. For GTD zbs, a few major home projects has significantly increased my proficiency.

• see recoll -> c++Q&A.txt
• [ZZ] try out debug tools in the gdb book? LG
• [ZZ] experiment with sample code in [[fin instrument pricing using c++]]
• [ZZ] proj: valgrind (linux) – get hands-on experience. Might take too much time to get it working
• problem – test if a linked list has cycles
• problem: all permutations of 0 or more of abcde
• problem: write skeleton c++ code for ref counted string or auto_ptr
• problem: test if a given container is in ascending order
• [ZZ means … see https://bintanvictor.wordpress.com/2017/02/14/low-leverage-ctopics/]

WallSt friends’ comment@slow coder,deadlines

This is life-n-death:  if you are not adding enough value you are out…

With important exceptions (Stirt, Lab49..) Wall street systems are stringent about time line, less about system failures, even less about maintainability or total cost of ownership or Testing. I feel very few (like 5%) Wall St systems are high precision and I include the pricing, risk, trade execution systems. Numerical accuracy is important to the users though, because those numbers are about the only thing they can understand. Their job is about control on those numbers.

In City muni, Boris’s code was thrown out because it didn’t make it to production. Any production code is controlled not by dev team but many many layers of control measures. So my production code in Citi will live.

If you are slow, Anthony Lin feels they may remove you and get a replacement to hit the deadline. If they feel it’s hard to find replacement and train him up, then they keep you – all about time lines.

Hou Li felt your effort does protect you – 8.30 – 6pm everyday. If still slow, then manager may agree estimate is wrong. She felt deadline and effort estimate are arbitrary. However, if you are obviously slower than peers, then boss knows it.

python/perl ^ java/c++ GTD tricks

label – z_jobHunt

[[Python cookbook]] chapters on meta-programming, performance … have powerful techniques for projects, but seldom quizzed on Finance IT interviews, including quartz, even after so many years of python adoption.

##[15]where in c++am I growing: GTD^IV

Short and sharp answers preferred

–Q: where and how fast am I growing in c++ IV?

A: EPI300/Google style coding test, esp with STL, shared_ptr etc

A: the HFT interviews revealed my weaknesses in some QQ areas.
A: ##10 basic programm`constructs for timed c++ cod`IV  — focus, direction for IV preperation

–Q: where and how fast am I growing in c++ zbs

A: debugging using print or gdb. No longer afraid of funny errors
A: experience reading STL errors
A: [[safe c++]]

A: [[21^st century C]]

A: Learned to question what low level details are more than academic

A: Learned the practical value of c++ integration with excel, python etc

A: Learned a bit more about instrumentation including debuggers, esp. on Windows

A: Learned a bit more about boost smart pointers. There are many of them, because there exists different needs.

A: Learned a bit more about IDE, make, cmake and friends

KPI operational skills:under-valued #eg IDE…

I know Barclays (or any big bank) pays a VP salary to experienced developers. Would they pay the same for someone very familiar with IDE, CVS, automated build/release/test, script automation, back-up … lots of tools?

No because the latter skill is a commodity “operational” skill compared to the “creational” skill of a product developer. In a way, developers are a revenue center. Developers need analysis, design, deep experience, domain knowledge…

I feel interviewers ask too much about algorithm/data structure, problem solving, financial domain knowledge, and not enough about those operational skills. I guess in 50% of dev team there’s no one specializing in those infrastructure stuff, so every developer must pick up this skill, which can really impact productivity and trouble-shooting.

I think instrumentation skills (see other posts) are closely related to this.

[15]essential GTD/zbs to build algo trading system

Hi

(See also http://bigblog.tanbin.com/ post on HFT)

Just to share some observations and reflections. More than one Asia (and to a lesser extent US) recruiters have reached out to me as a potential lead-developer for a HFT engine, to be created from scratch. I believe there are not many old hands in Singapore. Even in the US, this is a relatively small circle. Not a commodity skill.

A small trading shop would have very different needs than a big bank, so their HFT engine will use off-the-shelf tools for all but the most critical, customized modules. (I had a brief blog post on it.) What are the 10 essential know-how i.e. Essential functionalities you must know how to create (prove)?

• see also my 2018 post on execution system^OMS^FIX…
• executing strategy for order origination, i.e. machine trading
  • market data processor, sockets
  • order book replicated from market data? perhaps at the center of the engine
  • OMS (Kenny) ? in some cases we leverage the exchange query API or 3rd-party solutions
• [W] FIX or other order submission protocol
• in-sample testing. I used Matlab for this. Can be much simpler than the production system.
• [3] real time pnl, updated by market data
• [3] barebones blotter, or perhaps command line interface?
• [3] store historical data for analysis. Perhaps SQL or KDB.
• [3N] reference data
• [3] post-trade STP

Low-level tech knowledge
• [N] threading
• [NW] Boost
• debugger
• [N] memory leak detection
• [W] socket programming

[3 = 3rd party products could be good enough in Every component, but this one is more likely]
[N = Not needed in every shop, but often required by interviewer]
[W = my weakness, relatively speaking]

2 focuses when learning a new tech skill: IV^GTD

When we learn a new tech skill, we need very, very specific goals and guidelines to help us choose what NOT to dig into too early.

Focus #1: IV topics, a well-defined focus. Basically textbook knowledge. Go deep enough but not too deep on a particular subtopic. Examples — op-new, pure virtual, const ref param of copier, recursive locks, OOM calling new,

Focus #2: GTD i.e. dev productivity — as measured by managers, bench-marked against other developers. Incidentally, perceived productivity sometimes depends so heavily on local system knowledge that a new hire always lags behind no matter what. I feel productivity and GTD has low dependency on #1, as most of the textbook knowledge is irrelevant.

Focus #2b: tools — see post on 2 categories of tools.

Focus #2c? tips on StackOverflow. No such books. No way to

Focus #?: best practices — or Acceptable practices and well-trodden paths

Focus #?: learn all the syntax details to read any source code on books, interviews and online

Focus #: high-level design decisions, for technical leadership roles. Myriads of decisions are made every hour during development, some at high some at low levels. Apparently, a lot of these decisions involve theoretical understanding.

some skills are key to IV, some key to GTD

GTD skill – debugging

GTD skill – tools

IV – performance profiling, tuning

IV – algo, data structure, threading

IV – prog. language details

IV – low-level or in-depth knowledge

IV – design skills including OO and agile best practices

fear: c# iwt java/python

The more my GTD “survival” depend on tools or environment, the more fearful and the less confident I feel.

Extreme eg:  When programming Perl, php (and python) I don’t rely on any complicated tool. The libraries I use tend to be simple and small.

Extreme eg: Excel VBA is fully inflicted with infrastructure dependency. Any idiosyncrasy or nuance of Excel can give inexplicable subtleties.

— Java —

Java is a powerful, mid-level language with a remarkably robust sandbox. The ugly, dirty world outside is largely kept away from us programmers, so we operate in a clean room, sandbox, or a constant-temperature green house.

However, when I had to debug into Spring-JMS I hit the same infrastructure-dependency.

JNI — The sandbox leaks when you play with JNI. Infrastructure dependency on a massive scale. You now need a lot of “platform-knowledge” about the “platform” outside. It’s a lot dirtier than inside the sandbox.

Eclipse Debugger — is less well-behaving and less understood than other aspects of java development. When the source code is out of sync with the executing binary, things could keep going but would fail in strange ways — undefined behavior.

How about stack overflow. No exception. Just crash.

— c# —

The c# language is more feature-rich than java, but still consistent and structured. No complaints. However, when you use c# for real projects you hit infrastructure dependency. I mentioned in numerous blogs how much platform knowledge needed.

Unlike the famed write-once-run-anywhere portability of java, Many really, really important parts of C# development are tied into the “platform” outside, meaning developers need that platform knowledge. Just a few examples

* COM

* threading implemented on top of win threads

* many critical tasks need MSVS

* Excel integration

* windows service integration

Fwd: pure language complexity ^ tool complexity

(“Tool” includes the entire system used in the dev process.)

YH,

You asked me why my OC manager likes to beat me up. I thought about it. Here’s one reason.

My manager complains I’m slow with my development projects. In this job (as in many other jobs), people don’t say my code quality or design is bad. I think most of the code they write isn’t great either. We are just hoping to churn out working code on time. I do hope to but won’t try too hard to make my code beautiful, low-maintenance, testable, easy to change, easy to understand. Implicitly, that’s the unspoken agreement with the boss. One quality where I exceed minimum requirement is error-condition handling — i want my code to behave reasonably under any conceivable condition. I don’t want “undefined behavior”. I don’t want surprises. Anyway, my boss only complains about how slow I complete projects, not about quality. Invariably it boils down to troubleshooting skill like googling, asking around, understanding logs, and diagnostic tools. Sometimes not easy for anyone. Most of these technical problems involve more than code complexity in my c# codebase, and now I realize that’s the key.

Now I believe my c# language skill is perhaps better than the average c# developer. For eg, I’m confident taking on technical challenges involving threading, linq, complex data structures, complex delegates+events. (The toughest coding interviews on the west coast sometimes cover nothing but a little tight/dense algorithm. These can be extremely IQ-intensive, more than I can handle but I feel comfortable that I can beat most c# developers at this game.) In a nutshell, if the problem is purely about c# language, I am confident. Most real world technical problem-solving goes beyond that. There are too many of these problems to enumerate. Below are just a few examples.

eg: When the same code works in one environment not in another environment, I feel 30% of the time it’s due to some permission or security issue. Error message is usually misleading. Even worse when nothing happens, without any error msg. Can take ages to uncover the root cause.

eg: GUI troubleshooting always involves some tool and some infrastructure that’s not written by me…

eg: GUI system is invariably a client to some server-side. There’s always some networking involved.

eg: Async communication might be considered a pure language complexity but in reality troubleshooting often involves some vendor-supplied infrastructure code which sets up the communication and threading.

eg: WCF and remoting always involves tools to generate proxy, network layer, serialization…

eg: My biggest source of troubleshooting headache is VisualStudio… I had many problems with debugger, with build, with project references…

In order to reproduce a problem, I often need to check out and build many projects, configure many modules, hunt down all the required libraries, set up all the dependencies, install a bunch of infrastructure software… The set-up can take days for the uninitialized, and is error-prone and not fully documented. This happens everywhere so I’m not complaining, but this often leads to a lot of delay and complaint from boss.

[[debug it]] c++, java.. — tips

I find this book fairly small and practical. No abstract theories. Uses c++  java etc for illustrations.

Covers unix, windows, web app.

=== debugging memory allocators
–P170
memory leaks
uninitialized variable access
varialbe access after deallocation
–p199
Microsoft VC++ has a debuging mem allocator built in. Try http://msdn.microsoft.com/en-us/library/e5ewb1h3(v=vs.90).aspx

Electric Fence

===
–P201 DTrace – included in Mac OS X
–P202 WireShark, similar to tcpdump
–P203 firebug – client-side debugging
edit DOM
full javascript debugging

–P188 rewrites – pitfalls

–A chapter on reproducing bugs — quite practical

familiarity with localSys(+mainstream tools) is KPI for GTD

(I have been learning this lesson since 2007, but Not before.)

We techies including the coolest techies, can get stressed up by technical issues, by deadlines, by users and managers. These really are the same kind of issue — the damn thing doesn’t work. This not-working issue constitute a major stressor though not the only stressor. When things are seriously bad, our basic competency becomes a question, meaning managers question if we are up to the job. When this stressor gets bad,

we feel like treated differently than other team members
we lose manager’s trust, and we get micro-managed,
we are required to justify our (big or small) designs
we are forced to abandon what we wrote (99% working) and adopt another design imposed on us
we are asked to explain each delay
we are forced to work late
we voluntarily decide to cancel our vacation
we worry about our job
we have no time/energy to hit the gym
we lose sleep and lose appetite
…

On the other hand, when we are clearly capable of handling this type of issue, we feel invincible and on top of the challenges. We can GetThingsDone — see blog post on GTD.

To effectively reduce this stressor, we really need to get comfortable with the “local system”. Let’s assume a typical financial system uses java/c#/c++, SQL, sproc, scripts, svn, autosys … In addition, there are libraries (spring, gemfire, hadoop etc) and tools like IDE, Database tools, debuggers… riding on these BASE technologies.

We are expected/required to know all of these tools [1] pretty well. If however we are slow/unfamiliar with one of these tools, we get blamed as under performing. We are expected to catch up with the colleagues. Therefore each tool can become a stressor.

[1] see also http://bigblog.tanbin.com/2012/11/2-kinds-of-essential-developer-tools-on.html

Therefore a basic survival skill is familiarity with all of these tools + the local system. If I’m familiar with all the common issues [2] in my local system then I can estimate effort, and I can tell my boss/users how long it takes. Basically, I’m on top of the tech challenge.

[2] If some part of java (say the socket layer or the concurrent iterators) never gives me problems, then I really don’t need that familiarity.

Q: how about non-mainstream tools like spring-integration, jmock? Just like local system knowledge. Investing your time learning these is not highly strategic.

When I change job, again there’s a body of essential know-how I need in order to / fend off / the stressors. Part of this know-how I already have – the portable tech knowledge. Frequently, my new team would use some unfamiliar java feature. More seriously, the local system knowledge is often the bulk of the learning load. If I were in a greenfield development phase I would write part of the local system, and I would have a huge advantage.

A major class of tools are poorly understood with insufficient proven solutions —
– About half of Windows tools.
** OC GMDS systems kept crashing for no reason.
** When I got started with perl/php/javascript/mysql vs VBA/dos FTP, I soon noticed the difference in quality.

arch xp=overrated as predictor of KPI #instrumentation, tools…

See also other blog posts —

// “in the initial dev stage, instrumentation is my #1 design goal”.
// http://bigblog.tanbin.com/2010/09/2-important-categories-of-software.html

Going out on a limb, I’d say that in high-pace projects GTD (Getting Things Done) is more important than many things which are supposed to be more important.

GTD is more important than … integrating deep domain knowledge insight into the system. I think such domain knowledge can sometimes be valuable. It can help dev team avoid wasting time on “the wrong things”.

GTD is more important than … quality of code. For a few months at least, the only guy looking at the code is the original author.

GTD is more important than … “quality of design”, which is seldom well-defined.

GTD is more important than … being nice to others. Many times other people can tolerate a bit of personality if you can GTD. However, you must not offend the people who matter.

GTD is more important than … impressive presentations that address many real customer pains. A user may die for a guy who “understands my pains”, but when it is time to deliver, selling/understanding is an irrelevant distraction.

GTD is more important than … in-depth knowledge of the language or of a software platform/product. Such knowledge is more important than GTD during interviews though. Threading, data structure…

GTD is more important than … uncovering the root casue of (intermittent) problems/errors/surprises — the Toyota drill-down root-cause investigation. Regex engine creator may need to fully characterise every unexpected behavior; AutomateTellerMachine error probably deserve a drill-down investigation but in enterprise apps drill-down investigation simply takes too much time. We developers are paid to build a usable tool, not a fully-understood tool. Live with ambiguity and move on.

GTD is more important than … adding an important test to the automated test suite. Sure that mistake we just found may happen again so we really appreciate adding that test, but in reality, most high-pace environments don’t have an automated test suite. If we are lucky enough to have documented manual test plan, then yes add it, but such a plan seldom is comprehensive, so after a while people won’t follow it religiously. So in reality we just hope developers learn the lesson and avoid making the same mistake. If they do, then we just need someone who can GTD. Any system to incorporate such heard-learned lesson is likely an imperfect system, and whoever investing in such a system is often wasting his/her time.  If a GTD guy doesn’t bother with that system, he will still be respected and loved by manager and users. Basically, any long-term investment is unappreciated. GTD is all about short-term results. This is the reality of quality control in fast-paced teams.

GTD is more important than … adding meaningful error messages. Anyone debugging the system would love the guy who added the meaningful error message, but he is an unsung hero. Manager love the guy who GTD fast. Code quality is invisible and therefore ignored and unrewarded.

To achieve GTD, you must solve tech problems. Tech problems could (occasionally) call for architectural perspective, but less often than it calls for tool knowledge, debugging experience, or low-level system insight.

In defense of architects, architectural track record is more important in sales contexts, including selling to internal clients and business users.

I should probably get input from Raja, Xiao An, Yi Hai …

self-rating in java GTD+theory/zbs #halos#YH

Hi YH,

Self-rating is subjective, against one’s personal yardsticks. My own yard stick doesn’t cover those numerous add-on packages (swing, spring, hibernate, JDBC, xml, web services, jms, gemfire, junit, jmock, cglib, design patterns …) but does include essential core jdk packages such as —

• – anything to do with java concurrency,
  • reading thread dump
• – anything to do with java data structures,
• – [2017] Garbage collection
  • memory profiling (jvm tools)
  • GC log analysis
• – [2017] java.lang.instrument, mostly for memory
• – networking/socket, file/stream I/O
• [!2017] generics, esp. type erasure
• – RMI,
• – reflection, dynamic proxy, AOP
• – serialization,
• – class loaders
• – JNI, esp. related to memory
• – JMX, jconsole,
• – difference between different JDK vendors,
• – real time java
• – reading bytecode

(The highlighted areas are some of my obvious weaknesses.) Now I feel my self-rating should be 8/10, but i still feel no one in my circle knows really more about the areas outlined above. I guess that’s because we don’t need to. Many of these low-level implementation details are needed only for extreme latency, where c++ has a traditional advantage.

Another purpose of this list — answer to this

Q: what kind of java (theoretical) knowledge would give you the halos in a Wall St interview?

am fast at GTD, as WallSt developer

* I grew fast enough for GS. Compared to other banks, GS has 50% of the headcount for any given workload.
* I was fast enough for the Lab project lead. Their standard is even higher than GS. The Lab project lead didn’t complain about my delivery speed though he pointed out Piroz was more experienced and faster.
* Citi senior mgr didn’t complain about my delivery speed at all. Not even once. I was working at a comfortable pace.
* I worked alongside Lab49 consultants who are battle tested fast-coders over many years.
* in every job i managed to steal time for self-study – swing, python, secDB, rv, options, c++, non-blocking I/O, c++ debugger, bond math…
* many trading desk dev teams are elite teams with tiny headcounts. I survived some.
* I generally pay more attention to details than other developers. I could, if I want, switch off this attention.

I guess there are super-developers who are really faster in some specific contexts, but overall it’s hard to be a fair judge considering quality, prod support capability, error checking, knowledge-transfer, automated-testing, help to other team members, client-orientation, maintenance cost, system flexibility, readability, architecture soundness…

%%broad GTD valueAdd cf3Y-younger java programmer

 (Written in 2009)
– i managed difficult teams and difficult customers for years
– real AR experience with perl
– 5 years of Unix system admin experience
– i took part in AR decisions of many projects. 1st-hand AR experience
– I dealt with more diverse challenges of trouble-shooting and problem-solving
– In small projects, I was usually the customer-facing single point of
contact, not working behind the interface person. i was the only one
to negotiate with customer.
– Over the same 3 years, i dealt with many small customers rather than
1 or 2 big customers
– since i took part in more prj over the same 3 years, i witnessed
more failed prj
– i witnessed 1st-hand the “decline” of many technologies — rare xp

How do you identify high workload]the team #XR

These are not the heaviest stressors — see ## 2 heaviest work stressors 

• Q: Do you need to record how you allocate time to assignments in blocks of 2 hours, 4 hours or 8 hours?
  • Only happened in GS
• Q: Do you need to everyday describe, verbally, what you did in the previous day? I never did.
• Q: When neck, back, or eyes feel tired, do you always feel free to get away from computers for a while? I always do
• Q: Do regular developers feel relaxed enough to write, if they like to, documentation in wiki or for other people? I usually do
• Q: Do you get time to make 30-minute chitchats with friends on office phone during office hours? I usually do.
• Q: Do you feel pressure to return to work after 45 minutes at lunch with a friend, assuming no official policy about how long lunch break is.
  • Only happened in GS, BAML, Barclays
• Q: do you feel relaxed enough to read department newsletter? Exclude brief light periods after a project.
• Q: do you feel relaxed enough to read technology newsletter? Exclude brief light periods after a project.
• Q: Do you think twice about going out for lunch once a week?
• Q: Do you ever work out, however briefly, before leaving office?
  • I did in BAML, Barclays, Macquarie, ICE
• Q: Suppose you are stuck waiting for an answer or for a build to finish, do you feel you can just stop working for 15 minutes? What if you get such “waiting periods” everyday? Does your manager make it a rule that you should take up some of the numerous other tasks during such a short wait?
  • Only happened in GS
• Q: As a developer, during office hours do you feel relaxed enough to read business publication related to work? I believe (even lead) developers never have to read such things to do a decent job. Required business knowledge is highly site-specific and accrue during projects. Knowledge about industry is never a must.

## low latency – key expertise4GTD

I talked to a few big and small low-latency shops. I feel latency optimization is “low-logic, high speed, high throughput”. Below are the Expertise required, but in any real system, you will see diminishing return on each aspect, so remember to prioritize the cost-effective areas to optimize. See my post on 80/20 (http://bigblog.tanbin.com/2012/03/8020-rule-dimishing-return.html)

* threading
** parallel lock free threading but sometimes we can’t avoid inter-thread-comm (condVar)
** [L] try every technique to avoid locking (such as immmutables)

* [L] low-level memory management — custom allocators, custom (or disabled) garbage collectors
** care and feeding of java GC
** try every way to avoid non-deterministic garbage collection

* in-memory data stores — KDB etc
** This is how secDB compensates for some of the performance drawbacks
** state maintenance in memory — important in many OMS, DMA, exchange connectivity engines.

* try every techniques to avoid RDBMS
** use flat files

* low-latency messaging — 29West, tibrv messaging appliance …
** [H] async MOM is the only choice for high volume systems
** multicast is the de-facto standard

* OS monitoring and optimization
** dtrace
** network utilization
** paging activity
** cpu utilization

* [L] socket programming customized
* avoid ethernet collisions
* [H] avoid choke points, promote multi-lane highways
* [H] connectivity, collocation
* [L] serialization – customized
* [L] powerful hardware, FPGA
* [H] scale out – important
* 64 bit
* realtime java vs C++
[L] = low-level
[H] = high-level architectural feature

marathon – strengthen your(GTD+)lead in the pack#sg high-end

Y Lin pointed out those high-end jobs tend require specialized skills. I now feel concurrency is an example, in addition to —

latency – c++ is the default for extreme latency
latency – sockets
latency – market data
latency – java?
dnlg – math — better entry barrier, closer to business.
dnlg – jargons in FX, Rates…
dnlg – arch
FIX
MOM

—
real projects are completed in lowly tools like sql, java, C, scripts, but interviewers need to see much much more than those.

5 ways my GS managers make new guys so productive so fast

# 1) My GS managers have a feel about how much production support each developer is taking on. Therefore managers can monitor the learning pace of the new hire.

(Note Tier 1 or Tier 2 production support teams take on a lot, but they can’t read source code. Developer team always receive a lot of escalated prod support requests.)

Sooner or later, we realize there need to be a way to numerically measure the accumulation of _local_system_knowledge_ by the new hire. Many new hires intuitively feel “now i know close to half the systems”, or “now i have 75% of the system knowledge of my mentor”. A more objective way to measure is to simply count the number of system issues (big or small) resolved by each developer *independently*. I also count in all user questions as system issues. Together, they represent the “load” on the production support team. The more a new hire can help “offload”, the better. If a new hire is not taking on enough after 6 months, improve her training.

Crucially, only the knowledge relevant to “offloading” is true “system knowledge”. Manager needs to know the difference — a lot of know-how isn’t relevant and doesn’t constitute system knowledge.

#1b) Some of my GS managers realize that every proficient prod support person need to get their hands really dirty with logs, autosys JIL, DB investigation, tracing through java/SQL/scripts — an overwhelming body of “traces” or clues. A foundation of the “local system knowledge”.

In other banks, some new hires grow familiar with 1/3 of those traces in a year — too slow. It’s possible to achieve 70% familiarity in a year. 100% is defined as the minimum level among primary production support guys.

#1c) my GS managers asked me to spend a session each week with a system knowledge expert colleague, going through the weekly 300+ emails sent to the team. A lot of production support tasks originate in such emails. Personally, I feel any amount of such knowledge sharing can’t replace resolving an issue independently. A new hire needs to use her own head and interpret, guess, rule-out-possibilities, not just follow a cookbook. A lot of the “traces” are misleading or incorrect.

# 2 ) Perhaps the manager’s most important guide is the determination and conviction that “we can and will train this person up in 6 months to start delivering real business value”. A crude analogy — once you pay big money for a power drill before home renovation, then you want to get real value from the investment. Given this conviction, the manager makes everyone in the team aware new hire’s survival depends on their help, so “be prepared for the hundreds of questions new hire may ask”. In another bank, a senior mgr told me to reduce the volume of questions and learn to “slide into the new team”.

My GS manager spent lots of time trying to understand how I learn, how i spent my time, what questions I ask, where I often get stuck. This is what mentors do. Micro-managing? Perhaps.

3 ) My personal favorite (possibly biased opinion). Some of the new hires were put into “intensive production support” mode, kind of immersion training, spending 70% their time on prod support, far more than on green field or BAU. This lasts 3 months to almost a year.

) This is nothing special in GS — Let a new hire take ownership of a production support task and follow up till closure. I feel other banks too easily allow a new hire to pass on a “hard” problem to colleagues. In GS, New hire is encouraged to ask many “why” questions afterward, and mentor/colleagues should give in-depth answers. Both sides should be aware that some questions are less relevant and too early.

My GS managers, just as in other fast-paced banks, like to mention note taking. New hire’s note taking is frequently criticized in many banks. He can’t write down everything he hears since what he hears don’t make sense to him anyway. He can’t afford to spend too much time going into all the “why” either.

) This is nothing special in GS — Sometimes there’s a dedicated “mentor” to transfer basic system knowledge. A good mentor can make a big difference, such as pointing out the wrong things to delve into. Some new hires prefer asking many different colleagues.

I was a decent mentor. I share quick tips to set up tests, point out the tools worth learning … I spent a lot of time sitting with the new hire.

) In my GS department, people are generally more available to help or answer questions than in other companies. This extends across department boundaries. If I need help from another business division, I can send mail or call them and get immediate answers, or they will get back to me.

[11] BAU vs green field – WallSt

My Citi mgr divided my work into ProdSupport + Greenfield + BAU i.e. (strategic or quick) enhancements on current production codebase.

In ML, effort is divided into BAU 66% + 33% Greenfield

GS is 75-25

One of the hardest part in Greenfield is replicating BAU behavior. The closer we get to prod release, the more we rely on prod system knowledge. Those nitty-gritty BAU behaviors become necessary features and show stoppers.

I think recruiters generally see Greenfield as more attractive. More design involved. More experienced candidates required.

A friend said green field becomes stressful as release date approaches.

code-cloning && other code smells in fast-paced GTD WallSt

Update — embrace these code smells to survive the coworker-benchmarking

In fast-paced Wall St, you can copy-paste, as long as you remember how many places to keep in-sync. In stark contrast, my ex-colleague Chad pointed out that even a one-liner sybase API call should be made a shared routine and a choke point among several execution paths. If everyone uses this same routine to access the DB, then it’s easier to change code. This is extreme DRYness. The opposite extreme is copy-paste or “code cloning” as some GS veteran described. Other wall st developers basically said Don’t bother with refactoring. I’m extremely uncomfortable with such quick and dirty code smells, but under extreme delivery pressure, this is often the only way to hit deadlines. Similarly,

*Use global variables.
** -Use global static collections. Remember my collection of locks in my ECN OMS kernel?
** -Instead of passing local variables as arguments, make them static or instance fields.
** -Use database or env var as global variables.
** -Use spring context as global variables.

* Tolerate duplicate variables (and functions) serving the same purpose. Don’t bother to refactor, simplify or clean up. Who cares about readablility? Not your boss! Maybe the next maintainer but don’t assume that.
** Given the requirement that a subclass field and a parent field pointing to the same object, due to bugs, sometimes they don’t. Best solution is to clean up the subclass, but don’t bother.
** 2 fields (out of 100) should always point to the same object, but due to bugs, sometimes they don’t. Best solution is to remove one, but don’t bother.
** a field and a field of a method param should always point to the same object, so the field in the param’s class is completely unnecessary distraction. Should clean up the param’s class, but don’t bother.
** a field and method parameter actually serve the same purpose.
*** maybe they refer to the same object, so the method param is nothing but noise. Tolerate the noise.
** tolerate a large number (100’s) of similar methods in the same class. Best to consolidate them to one, but don’t bother.
** tolerate many similar methods across classes. Don’t bother to consolidate them.
** tolerate tons of unused variables. Sometimes IDE can’t even detect those.

– use macro to substitute a checked version of vector/string. It’s cleaner but more work to use non-macro solutions.
– don’t bother with any validation until someone complains. Now, validation is often the main job of an entire applicaiton, but if your boss doesn’t require you to validate, then don’t bother. If things break, blame upstream or downstream for not validating. Use GarbageInGarbageOut as your defence.
– VeryLongVeryUgly methods, with multi-level nested if/while
– VLVU signatures, making it hard to understand (the 20 arguments in) a CALL to such a method.
– many methods could be made static to simplify usage — no need to construct these objects 9,999 times. It’s quite safe to refactor, but don’t bother to refactor.
– Don’t bother with null checks. You often don’t know how to handle nulls. You need to ask around why the nulls — extra work, thankless. If you can pass UAT, then NPE is (supposedly) unlikely. If it happens in production, you aren’t the only guy to blame. But if you finish your projects too slow, you can really suffer.
– Don’t bother with return value checks. Again, use UAT as your shield.
– use goto or exceptions to break out of loops and call stacks. Use exception for random jump.
– Ignore composed method pattern. Tolerate super long methods.
-Use if/else rather than virtual functions
-Use map of (String,Object) as cheap value objects
-Forget about DRY (Don’t Repeat Yourself)
-Tolerate super long signatures for constructors, rather than builder pattern ( [[EffectiveJava]] )
– don’t bother to close jdbc connections in the finally block
– open and close jdbc connections whenever you like. Don’t bother to reuse connections. Hurts performance but who cares:)
-Don’t pass jdbc connections or jdbcTemplates. In any context use a public static method to get hold of a connection.
– create new objects whenever you feel like, instead of efficiently reuse. Of course stateless objects can be safely reused, creating new ones are cheap. It hurts performance but who cares about performance?
-Use spring queryForMap() whenever convenience
– Tolerate tons of similar anon inner classes that should be consolidated
– Tolerate 4 levels of inner classes (due to copy-paste).
– Don’t bother with thread pools. Create any number of threads any time you need them.
– tolerate clusters of objects. No need to make a glue class for them
-Use public mutable fields; Don’t both with getters/setters
-JMX to open a backdoor into production JVM. Don’t insist on password control.
-print line# and method name in log4j. Don’t worry about performance hit
-Don’t bother with factory methods; call ctor whenever quick and dirty
-Don’t bother with immutables.
– tolerate outdated, misleading source documentation. Don’t bother to update. Who cares:)
– some documentations across classes contradict each other. Don’t bother to straighten.

knowledge@vendor products is far from enough4GTD

Consider typical non-trivial financial app based on 1 or a couple primary vendor products + some database + some JMS server + some app server + some web server. The #1 “primary vendor software” could be a distributed cache, spring, ETL, or something like ION or something else.

Q: How much complexity is in the vendor software, and how much is in the site-specific code including localized customizations of the vendor software?
A: 10% vs 90%

How about the ML Edge b2b trade engine? 70/30% at this stage (March).

##wall-blaster handy techniques to impelement features

Challenge — to implement non-trivial logic quickly, cleanly, with assurance and peace of mind.

Default (stupid) solution is a java/c++/c# implementation – painful. The challenge feels like a stone wall to drill through. The following technologies are good dynamites. (Additional Background — often, there’s a legacy Prod codebase…)

#1) regex
#2) SQL execute immediate
) perl
) reflection
) rmi
* embedded db
* multiple-statement SQL batch to implement complex logic
* byte code engineering
* DB constraints to replace app validation
* MOM as concurrency tool
* DB as concurrency tool
* file system as concurrency tool
* jmx
* annotation

Also-rans
* javascript, if this fits into the infrastructure
* xslt
* aop

I don’t fancy dependency-injection, ORM, ajax.

WallSt IT project timeline is no joke

When I first joined GS, I was in awe of my managers’ /commitment/ to project deadlines. Before GS, I always felt deadlines are usually arbitrary and deserve little respect. I used to feel deep in my guts that developers ultimately hold an upper hand over managers *when* (not IF) deadlines become difficult. Basically, I feel the regular developers are hard working but aren’t saints, so we are all to varying degrees lazy and undisciplined, and tends to leave too much work to the end. On the other hand, managers can never be 100% on top of every Dirty detail, the quick and Dirty, and the Dirt swept under the carpet, so they are always surprised at end of the time frame.

After a few years in GS, I feel on Wall St, a lot is at stake when the timeline pressure heats up. Managers are visibly concerned about missing deadlines. Developers often work long hours. Major source of developer stress (and burnout).

If you feel elsewhere developers miss timelines 80% of the time, then it’s 20% in GS.

The hallmark of Wall St IT teams — pays more and demands more productivity (quicker delivery). Keyword is productivity.

Everyone can stick to a timeline if it’s long enough, but GS typically gives you one-tenth to half the timeline given elsewhere. It’s not uncommon to see a 10-month project squeezed into 1-month.

A lot of your career is at stake too. If you manage user/manager expectations well, then you could win big wins and move up.

————–

That was my view while in GS. After many years outside GS, I feel Barclays and BAML try to emulate GS with up to 50% effectiveness. Outside GS, the other Wall St IT managers can also give demanding deadlines, but the consequences of miss is not so severe. If you miss one without some scapegoat, then you can get any amount of pressure including but not limited to

• no pressure at all
• fake pressure
• asked to work OT
• sidelined to less critical roles
• given some coach to work with you on the same project
• official warning
• performance improvement process. (I hate this, though it is designed to protect the employee.)

GS workload — squeeze-in

Here's another reason GS developers get _2_to_3_ times the work done per man-month compared to other companies on Wall Street.

Say at beginning of Jan I start a 4-week project in GS. Need to release by end of month. Then scope expands by 30% — fine we can absorb it in the same timeframe. Then some unrelated production issue pops up. No one else knows it better, so I let it squeeze into my workload. It is taking many hours, but manager won't extend the deadline. Then other mini-projects squeeze into my workload. I end up spending only half my time on the main project. However, deadline stays the same. I'm now expected to double up my effort to swallow all the new stuff squeezed in.

You may think it's unreasonable, but the hard, cold fact is, the other team members could cope, so must I too. Sometimes they look just like sponge with unlimited absorption.

Mind you, the initial 4-week estimate is not fake — It's a reasonable estimate by all accounts. But GS managers know they can squeeze in more stuff later, so they do.

GTD on-time: GS time booking

A developer position is created by business, and funded by business. Why wall street pays sky high for developers? Because business generates high profit and business wants more IT guys to help them shorten “time to market”.

In my GS system, code quality is low priority[1]. But many GS people say productivity is higher than other banks. I believe “productivity” is measured by the number of many-hours actually spent and _booked_ to the project. But how is GS time booking different from other banks?

For a developer, the number of man-hour in a year is a fixed 1920 man-hours. I typically book into 20 – 40 projects each year. I feel at least 6 levels of managers above me (up to MD and partner) look at my time booking data to manage their budget and expenditure. Unlike other banks, GS line managers look bad in front of very senior managers if developers overbook time. Developer is a scarce resource in GS and many levels of managers watch their burn-rate. Other banks are less burn-rate-conscious.

Suppose a project is budgeted 40 man-hours but is expanding to 60 due to scope increase, then other projects must shift to the next year. Business don’t like their project postponed, so they push IT managers, hard. IT managers want to please the business, so they push developers. Other banks are less determined.

In GS, there’s real resistance to postponing projects to next year. Most employees and consultants endeavor to deliver in time even though requirement often change. In other banks, IT guys say “we aren’t machines or supermen”, but in GS many developers are expected to be 80% more productive than other banks. This is achievable if a developer knows her system inside out.

If I say “My project is booking 50% more time because requirements changed drastically”. Manager would come down to scrutinize and ask “Every project has changing requirements, but they don’t exceed budget this much. Why are you slower?” I feel other banks don’t say this.

I feel GS is more productive because business can really push IT. Both business and IT are motivated by promotion and bonus

[1] “Finish your work on time, with reasonable quality”

##shallow technical xp is uaually enough for GTD

(See also post on theory^how-to explaining threading and eclipse.)

Here’s a list of sexy technical skills hiring managers look for, but when you spend a few months in it you realize there’s no real *accumulation* or depth in it. 1 month’s systematic learning is enough then you need to “meet and greet” the creatures in it. That’s how you become productive with each tool.
– spring, hibernate
– RV
– JMS
– FIX
– gemfire
– perl, python because most places don’t use advanced features
– web service

? threading requires some theoretical and analytical skill.
? jvm tuning may require accumulation but needed in only low-latency systems

In contrast,
+ DB tuning requires accumulation
+ c++ requires accumulation
+ c#
+ swing, wpf
—-
+ unix has depth but no job requires it
+ reflection, but no job spec mention it
+ eclipse has, but few job specs mention that

code tracing ^ performance tuning

In terms of practical value-add, keeping the job, workload management, time for family, keep up with colleagues and avoid falling into bottom quartile…. #1 challenge i see so far is legacy code tracing. How about latency and throughput?

market feed
eq/FX,
google, facebook
clearance

I think most performance optimization work is devoid of domain knowledge. Exceptions include
* high frequency trading
* CEP

##[10]result-oriented lead developer skills

(… that are never tested in interviews)

Many buy-side shops need a lead developer not to manage people but get things done.

In many of my past teams, Site-specific local system knowledge is 95% of the required knowledge. Generic knowledge pales in comparison including portable GTD and zbs.

[t] diagnosis
[t] Unravel and explain (high AND low-level) data flow and business logic. Requests come from business and other teams.
[t] how-to-achieve (H2A) a requirement — knowledge about existing system is far more relevant than generic techniques. Highly valued by employers. Budget is often allocated to requirements.

I feel the Lab49 consultants and the productive GS developers, and Sundip Jangi are good at all these areas.

[t=tracing large amounts of code is required]

release test failures and code quality

(blogging again) XR, You mentioned some colleagues often release (to QA) code that can’t complete an end-to-end test .. (Did I got you wrong?)

That brings to mind the common observation “95% of your code and your time is handling the unexpected”. That’s nice advertisement and not reality — a lot of my GS colleagues aggressively cut corners on that effort (95% -> 50%), because they “know” what unexpected cases won’t happen. They frequently assume (correctly) a lot of things about the input. When some unexpected things happen, our manager can always blame the upstream or the user so we aren’t at fault. Such a culture rewards cutting-corners.

I used to handle every weird input combination, including nulls in all the weird places. I took the inspiration from library writers (spring, or tomcat…) since they can’t assume anything about their input.

I used to pride myself on my code “quality” before the rude awakening – such quality is unwanted luxury in my GS team. I admit that we are paid to serve the business. Business don’t want to pay for extra quality. Extra quality is like the loads of expensive features sold to a customer who just needs the basic product.

As we discussed, re-factoring is quality improvement at a price and at a risk. Business (and manager) may not want to pay that price, and don’t want to take unnecessary risk. I often do it anyway, in my own time.

##java GTD knowledge2deepen on daily basis {SocGen

(blogging again)

Even though we can pass most java interviews (in the east coast:)), I find myself learning bits of practical java how-to on a *daily* basis.

* IDE
* ant and JVM launch script
* reflection
* thread pool
* concurrency constructs
* serialization
* callback listeners
* MOM
* generics
* JMX
* swing

They present everyday challenges to a intermediate developer like me, so better knowledge does improve problem solving, code t r a c i n g and design. These challenges illustrate one reason why I feel less proficient in java than in Perl. I can solve most *practical* Perl problems. Java is bigger and present more “problems”. As I spend more years in java, I hope to pick up more know-how. Therefore java remains my primary study subject, along with c++.

In the list above, I have deliberately omitted vendor products as they aren’t core java IMO. But obviously there’s a lot to learn in them.

* spring
* hibernate
* gemfire
* fitnesse

##java value-add` GTD/KPI skill when u join a trading team

What skills make you a more resourceful and *productive* developer to a fast-paced trading system? These usually have little to do with arch design. Almost never asked in interviews, ironically.

See also my post on 2 types of tools.

1) IDE remote debugging
** breakpoint on exception
** conditional breakpoint
** compile with local vars
2) IDE refactor — rename, search

) reflection, dynamic proxy
) profiling
) jconsole, JMX
) requesting Runtime.gc()
) generics for algorithm abstraction but this tends to be too complex
) code gen? cache triggers

Threading? nope it seldom adds too much biz value

stronger]Perl than java though I spent more years]java (theory^GTD)

Q: what elements make me Feel strong in c# or c++ or perl…?
A21: passing a large number of non-trivial interviews
A36: discussion with veterans. Can follow. Can contribute.
A78: help veterans solve problems
A96: independently solve non-trivial problems (colleagues couldn’t help you solve)
A01: hands-on skill with the tough and core parts of the language, like threading, generics, Reflection …
A10: tools, tools and tools

Q: what elements make me Look strong in c# or c++ or perl…?
A63: track record – a wide range of non-trivial mainstream projects, often with substantial (or impressive) volume and performance
A81: certification or the type of knowledge therein
A56: insight into strange errors that other veterans don’t dig into

For c#, I have A36, A01, A96
For java, I can improve A78, A10
——-
In any programming language there’s a body of abstract theory (including design principles, language features…) on one hand, and practical how-to on the other hand. I’m a theory guy meaning I understand theory faster. Practical on-the-ground problem solving skill is ….mediocre.

Now, Why am I stronger in Perl than java even though I spent more years writing java? Here’s my new answer — I understand java fundamentals better than Perl fundamentals; but I know more Perl how-to than java how-to. Most practical Perl problems I can resolve faster than others. (Well, not many people spent that many years using Perl.) I have worked with many Perl developers and many of them tell me I know a lot of little tricks.

Here’s one example of a killer java skill — Multithreading is the most complex part of java (followed by generics and reflection, IMO). Complex in the JVM ie “theory”. In practice, app developers stick to code idioms and proven constructs. These “best practices” shield programmers from the underlying complexities. If you know some of these “best practices”, then you don’t need a deep understanding of the JVM internals. What I just said is all within my comfort zone. However, when threads meet swing, messaging, distributed cache, app servers…, we are challenged. Not much theory. Just a lot of code on the ground. You need resourceful, quick problem solvers. Good understanding of java fundamentals can help to some extent.

Another example – eclipse IDE. Never tested in interviews. No theory. But can really make a programmer so much faster or slower.

##where implementation complexities lie: DB, java …

You could be familiar with all the generic tech below, but when you take over a production codebase as a new “owner”, sooner or later you have to wade through megabytes of code and learn to trace the flow. To me, that’s the essence of complexity.

For each DB column, how does data flow from end to end? It takes unknowable effort to document just one column in one table. in a DB-centric world (like GS), these are often the first step in understanding (or architecting) a system, either a tiny sub-system or a constellation of systems
DB join logic — often directly reflects real-world relationship
DB constraints — they affect system behavior
DB view definitions
stored proc, triggers
html — can implement categories, multiple-choices, page flow, data clump…
config files that control java/script/proc
everything in JIL
autosys dependencies
javascript
scripts
java/c++/c# — are the “main” implementation language in a typical financial apps, where most of the code logic exist.

Tracing code flow means we must trace through all of the above. Ironically, interviews can’t easily test how fast someone can “figure out” how data flows through a system. I feel this could be the hardest thing to find out during an interview.

This is yet another reason to stay in java (or c++/c#) and avoid spend too much time scripting. Over time you might become better at tracing code flow through java.

I used to think I can figure out the flow by myself. Now i know in a typical financial system this is impractical. You must ask the experienced developers.

[09] insightful article: managing tight deadlines

http://fh.rolia.net/f0/c1050/hit/post/6237273.html (removed?) + my comments

“A job usually involves many phases: investigation, brainstorming, design, implementation, testing and documentation. A quality job requires effort and time in every phase. However, when push comes to shove, many of the phases can be skipped. The problems will show up much later. By that time, nobody would care who’s to blame. And companies are more than willing to budget for these problems, in the form of increased support, more bug fixes, or even a brand-new system. You just have to be WILLING and ABLE to produce imperfect software.”

“The second important thing to managing work load is that you have to be the master of your domain, not your boss. This means you don’t tell your boss everything. And you make a lot of decisions yourself. Otherwise, you lose control.” — My GS team lead knows too much about my project. I tell him everything about my project.

“It starts from estimates. You know better than anyone else how long each piece will take. A hard piece to others might be easy for you. But a simple task might end up taking a lot of your time. Don’t tell your boss that you’ve worked on something before and can borrow a lot of code from previous projects.”

“The same applies in the middle of your project. A seemingly complicated piece could turn out to be smooth sailing. Yet a small issue could bog you down for many hours. Again, don’t tell your boss you finished something in an hour which was budgeted for half a day. But do tell him that a bug from another team cost you many hours unexpectedly.”

“What do you do when you see something wrong in the requirement? Or something wrong with other people’s work which you depend on? If you’re pressed for time, act as if you didn’t see them. Act like a fool. You may be punished for missing your own deadline, but you’re unlikely to be punished for not spotting other people’s mistakes.” — By not reporting the issues early, project will suffer but you will not, but avoid making your boss look bad — try to give him a scapegoat. Project will suffer — project will need more time, and the reason is other people’s mistakes. You expand the impact of their mistakes to get more time for yourself. Conversely, when your mistake affects them, they might do the same.

“What do you do when deadline approaches and you discovered a big hole in your work? Again, act like a fool. Act as if you didn’t see them. You hand in your work. And a week later, people would find issues. But that’s normal. Nobody’s perfect. You and your boss get punished for missing deadline; but neither(???) of you would be held responsible for (non-critical?) bugs. Rather you will be given new budget to fix things, probably after a relaxing break in the sun.”

“Every decision you make affects your schedule. Be flexible. Be creative. Be able to accept imperfections. Be a liar if need be. The important thing is to look good, not to be good. Image is everything. And you can cut a lot of corners without affecting your image.”

— GS Slogan says “Tell your manager the bad news early”. If it’s your mistake, then decide if he will find out you were hiding it. Some managers periodically ask “Any problem?” Often he can’t be sure you were knowingly hiding problems — you could act like a fool.

— There are different bugs and different levels of impact. Manager may say some functionality is important or some time line is important, but question them in your head.  It takes a lot of observations to figure out which ones are really important to your manager’s bonus. Many delays and missing features are manageable or expected.

— My GS team peers know what bugs are tolerable to the manager. If manager must explain it to users and other teams, then you know it’s a visible bug. This knowledge comes from experience. However, initially you need to do some quality work to win trust.

— In fact, the best GS (and other) team members often contribute non-trivial bugs, partly because they are more productive and change lots and lots of code quickly.

##[09] productivity skills in some financial firms #Le2xr

XR,

(to be published on blog, no need to reply)

I wrote about the same topic in a 2007 email to you — http://bigblog.tanbin.com/2007/12/interview-topics-are-irrelevant-now_10.html, and you replied — http://bigblog.tanbin.com/2007/12/fwd-interview-topics-are-irrelevant-now.html.

My GS experience shows that I could be somewhat slow initially[1], when getting up to speed on an unfamiliar system. Now i’m feeling a similar “teething pain”.

Now, what technical knowledge do highly *productive* developers tend to have in common?

* IDE — big area i am lacking, though i used IDE’s extensively since 2006.
** ant integration and external build tool
** cvs/svn integration
** content assist
** setting classpath
** using multiple versions of jre system library
** remote debugging? few know it but it’s powerful

* java generics
* junit — how to run, debug, write
* cvs/svn — synchronization, tortoise, branching, tagging
* ant, maven, ivy
* sql
* unix — grep, find, vi …

In GS and my new team here, I have met a few experienced java developers who know Eclipse a lot more than I do, just like I know Perl and Unix more than others. I learned so much Perl by reading and using advanced features — things like the O’Reilly cookbook tricks and tips.

[1] initial 2-4 months

interview topics are irrelevant in project

XR,

You asked me the challenges of Wall Street projects. I feel in the
first 2-6 months on this job, “thoughput” is top priority. quickly
complete every assignment. Most important tech skills are

*1) tracing code flow, often complex flows involving sql, braching,
looping, scheduled job (cron etc), table-driven business logic…

*2) practical testing of your own code. This requires
disciplined/creative thinking and deep experience. Without good test
coverage, we risk breaking things and lose people’s trust. However,
there’s never infinite time/resource to support testing. We can
sometimes become too slow with our testing.

* syntax of perl, sql, java. The more you memorize, the less time
spent searching online.
* searching for various tech solutions online
* modify other people’s code , with almost 0 documentation on the
(non-trivial) business logic
* reading tons of scattered and /patchy/ documentation about many
related external systems.
* troubleshoot mysterious java compilation problems
* proficiency using a unix shell
* proficiency using cvs to checkout/checkin code

label – GTD_skill in findev

GTD means “survival”, not exceeding.