2 pitfalls on my accu path #portable,MD..

Say you stay in one team for 5 years, hoping to accumulate expertise and insight

  1. Trap: local system knowledge, not portable
    • eg: Qz
    • Tibrv wrappers in 95G is not so bad
  2. Trap: poor market depth
    • eg: vol fitter
  3. Trap: limited standardization across companies.
    • eg: cryptocurrency
    • eg: OMS?
    • eg: mkt data dissemination cf raw mkt data parsing
Advertisements

top 2 portable skills { each job

These are portable, longevity skills either for IV or GTD.

+number (1-3) = subjective valuation as of 2018
[grey color = portable financial domain skill, not tech skill ]

  • NIE +1 — 1) hacking open source php
  • GS +3 higher value than all other jobs —
    • 1) java GTD 2) SQL 3) database tuning 4) Database batch processing architecture
  • Citi +2 — 1) bond math 2) market-maker quote pricing system “architecture”
  • Barc +1 — 1) option math 2) analytics library integration
  • 95G +2 high value over mere 5 months —
    • 1) java wait/notify 2) store-proc for trade booking 3) FIX connectivity 4) Tibrv (basics only) and MOM system architecture
  • OC +1 — 1) c# GTD 2) basic MSVS 3) quote distribution architecture
  • Stirt +0 — 1) curve building basics. This job paid reasonably well, so I won’t prefer it for a low-paying top “contexx”
  • Mac +2 — 1) serious use of standard (portable) python 2) devops including git
  • RTS +2 — 1) c++ instrumentation 2) raw mkt data feed 3) order book replication 4) bash/perl for personal automation 5) socket (basics only)
  • mvea? — 1) lowLatency, high volume eq trading architecture 2) c++ (not C) GTD in one of the biggest c++ trading platforms.

##orgro lens:which past accu proved long-term # !! quant

(There’s a recoll on this accumulation lens concept…. )

This post is Not focused on IV or GTD. More like zbs.

Holy grail is orgro, thin->thick->thin…, but most of my endeavors fell short. I have no choice but keep shifting focus. A focus on apache+mysql+php+javascript would have left me with rather few options.

  • —-hall of famers
  • 1) [T] data structure theory + implementation in java, STL, c# for IV — unneeded in projects
  • 2) [CRT] core java knowledge including java OO has seen rather low churn,
    • comparable to c++
    • much better than j2EE and c#
  • 3) [T] threading? Yes insight and essential techniques. Only for interviews. C# is adding to the churn.
  • 4) [j] java/c++/c# instrumentation using various tools. Essential for real projects and indirectly helps interviews
  • [C] core C++ knowledge
  • [C] GTD knowledge in perl/python/sh scripting
  • [j] google-style algo quiz — Only for high-end tech interviews. Unneeded in any project
  • [R] SQL? yes but not a tier one skill like c++ or c#
  • coding IV — improved a lot at RTS
  • ————————also-ran :
  • devops
  • [C] personal productivity scripts
  • [T] probability IV
  • [C] regex – needed for many coding interviews and real projects
  • [C] low level C skills@RTS {static; array; cStr; reinterpret_cast;  enum; typedef; namespace; memcpy}
  • [!T] bond math? Not really my chosen direction, so no serious investment
  • [!T] option math?
  • SQL tuning? not much demand in the trading interviews, but better in other interviews
  • [R] Unix — power-user GTD skills.. instrumentation, automation? frequently used but only occasionally quizzed
  • [R] Excel + VBA? Not my chosen direction

–strengths
C= churn rate is comfortable
D = has depth, can accumulate
R= robust demand
T= thin->thick->thin achieved
j|J = relevant|important to job hunting

## UPSTREAM tech domains: defined by eg

It pays to specialize in a domain that helps you break into similar domains. (Unsuccessful with my quant adventure so far.)

Note “wrappers” API are never upstream. Very common in big banks including Qz.

  • #1 eg: [L] socket —- C++ is the alpha wolf, leader of the wolf pack
  • [D] market data —- Nyse/Nasdaq (binary) feed parer seems to be the leader, over the slower FIX feeds
  • [D] execution algo —- sell side equities desk seems to be the leader in terms of volume and variety of algos
  • data analytics language —- python looks like the leader
  • eg: [D] alpha —- equities trading is the leader
  • eg: threading —- java is the leader of the pack. In c++ threading is too hard, so once I master some important details in java threading, it helps me build zbs in c/c#, but only to some extent
  • eg: regex/string —- Perl is the leader
  • eg: [D] consolidated firm-wide risk —- SecDB is the leader. Applicable to buy-side? I think so.
  • lambda —- C# is the leader. Scripting languages follow a different pattern!
  • list/stream —- Linq is the leader
  • drv pricing theory —- Black-Scholes —- feels like a leader or upstream, but not /convincing/
  • [L] heap/stack —- C++ is the leader. Java and c# are cleaner, simpler and hides the complexities.
  • defensive coding —- (and crash management) C++ is the leader, because c++ hits more errors, like Japan on earthquake management.
  • eg: collections —- C++ is the leader of the pack. There’s a lot of low level details you gain in STL that help you understand the java/c#/python collections
  • eg: [L] latency —- C++ is the leader of the pack in low-level optimizations .. In-line, cache-efficiency, footprint..
  • eg: [L] pbref^val —- C++ is the leader. C# has limited complexity and java has no complxity
  • [L = lowLevel]
  • [D= well-defined application domain]
  • ? automated pricing —- no upstream
  • ? Fixed income trading systems —- no upstream. IRS^Treasury^credit bonds (trading platforms) have limited features in common.
  • [D] VaR/risk analytics —- FI derivatives in big banks seem to be the leader, but I feel one trading desk in one firm may not respect another system

%%c++keep crash` I keep grow`as hacker #zbs#Ashish

My current c++ project has high data volume, some business logic, some socket programming challenges, … and frequent crashes.

The truly enriching part are the crashes. Three months ago I was afraid of c++, largely because I was afraid of any crash.

Going back to 2015, I was also afraid of c++ build errors in VisualStudio and Makefiles, esp. those related to linkers and One-Definition-Rule, but I overcame most of that fear in 2015-2016. In contrast, crashes are harder to fix because 70% of the crashes come with no usable clue. If there’s a core file I may not be able to locate it. If I locate it, it may not have symbols. If it has symbols the crash site is usually in some classes unrelated to any classes that I wrote. I have since learned many lessons how to handle these crashes:
* I have a mental list like "10 common crash patterns"
* I have learned to focus on the 20% of codebase that are most convoluted, most important, most tricky and contribute most to debugging difficulties. I then invest my time strategically to rewrite (parts of) that 20% and dramatically simplify them. I managed to get familiar and confident with that 20%.

* I have learned to pick the most useful things to log, so they show a *pattern*. The crashes usually deviate from the patterns and are now easier to spot.

* I have developed my binary data dumper to show me the raw market data received, which often "cause" crashes.

* I have learned to use more assertions and a hell lot of other validations to confirm my program is not in some unexpected *state*.
* I figured out memset(), memcpy(), raw arrays are the most crash-prone constructs so I try to avoid them or at least build assertions around them.
* I also figured signed integers can become negative and don’t make sense in my case so I now use unsigned int exclusively.
* I also gained quite a bit of debugger (gdb) hands-on experience

Most of these lessons I picked up in debugging program crashes, so these crashes are the most enriching experience. I believe other c++ programs (including my previous jobs) don’t crash so often. I used to (and still do) curse the fragile framework I’m using, but now I also recognize these crashes are accelerating my growth as a c++ developer.

focus+engagement2dive into a tech topic#Ashish

(Blogging. No need to reply)

Learning any of the non-trivial parts of c++ (or python) requires focus and engagement. I call it the “laser”. For example, I was trying to understand all the rules about placing definitions vs declarations in header files. There are not just “3 simple rules”. There are perhaps 20 rules, with exceptions. To please the compiler and linker you have various strategies.

OK this is not the most typical example of what I want to illustrate. Suffice to say that, faced with this complexity (or ambiguity, or “chaos”) many developers at my age simply throw up their hands. People at my age are bombarded with kids’ schooling, kids’ enrichment, baby-sitting, home repair [1], personal investment [2], home improvement… It’s hard to find a block of 3 hours to dive in/zoom in on some c++ topic.

As a result, we stop digging after learning the basics. We learn only what’s needed for the project.

Sometimes, without the “laser”, you can’t break through the stone wall. You can’t really feel you have gained any insight on that topic. You can’t connect the dots. You can’t “read a book from thin to thick, then thick to thin again”. You can’t gain traction even though you are making a real effort. Based on my experience, on most of the those tough topics the focus and engagement is a must.

I’m at my best when I have my “laser” on. Gaining that insight is what I’m good at. I relied on my “laser” to gain insights and compete on the job market for years.

Now I have the time and bandwidth, I need to capitalize on it.

[1] old wood houses give more problems than, say, condos with a management fee

[2] some spend hours every day

meaningful endeavor(Now)towards20Y-career build`#dnlg;zbs;IV

Q: what efforts go towards 20Y-career building?
A: instrumentation in c++(java is a bit more churn); socket programming; STL internals; JVM internals; shell scripting; pthreads; FIX; linux internal?

This question is slightly vague, but it’s subtly different from

Q2: what efforts create social value over 20Y?
Q3: what efforts are cumulative over 20Y?

I feel the key question in this write-up is about building a deep and broad “foundation”, upon which we can start to plan (possible) social value, retirement planning, parenting/family building etc.

  • 😦 teaching and research —- Somehow I tend to feel research creates long-term value but I suspect most research efforts are not valuable!
  • 😦 open-source software —- I tend to feel OSS projects create social value, but most of them are not successful and have no lasting influence
  • 😦 Deepening your domain knowledge such as (German’s) security lending —- It does increase your value-add to some employers and help you move up to management, but I don’t call it “foundation”.
    • MSFM and other formal training —- I feel this is less effective, more ivory-tower, but it does build a firm theoretical foundation.
  • fitness and health .. more important than I feel (as shown by my action)
  • zbs such as instrumentation skills? Yes zbs helps your KPI, and also indirectly helps you with interviews. It’s portable skill, unlike local system knowledge. Can the value last 20Y? Depends on the technology churn. c++ and java are safer.
  • IV muscle building? My favorite career building effort. 20Y? 10Y yes.
  • English/Chinese writing and vocab

skill: deepen^diversify^stack up

Since 2010, I have carefully evaluated and executed 3 broad strategies:

  1. deepen – for zbs + IV
  2. diversify or branch-out. Breaking into new markets
  3. stack-up – cautiously
  • eg: Deepened my java/SQL/c++/py knowledge for IV and GTD. See post on QQ vs ZZ.
  • eg: diversified to c++, c#, quant, swing…
  • eg: diversify? west coast.
  • eg: diversify? data science
  • eg: diversify? research + teach
  • eg: stack-up to learn spring, hibernate, noSQL, GWT.

Stack-up — These skills are unlikely to unlock new markets. Lower leverage.

GTD stress/survival on the job? None of these help directly, but based on my observation GTD skill seldom advance my career as a contractor. It could create a bit of spare time, but it’s a challenge to make use of the spare time.

I feel German’s strategy goes beyond tech skills. He treats tech skills as a tool, used to implement his business ideas to create business value. I feel his target role is a mix of architect/BA but more like “product visionary”. It’s somewhat like stack-up.

in-depth^cursory study@advanced program`topics

I have seen both, and tried both.

AA) A minority of developers spend the time studying and understanding the important details of threading, generics, templates … before writing code.

BB) Majority of developers don’t have the time, vision or patience, and only learn the minimum to get the (sloppy) job done. They copy-paste sample code and rely on testing, but the test cases are limited to realistic UAT test cases and will not cover edge cases that have yet to occur. For concurrency designs, such tests are not good enough.

AA is often seen as unnecessary except:
* Interviewers often go in-depth
* In a product vendor rather than an in-house application team
* If requirement is unusual (eg: B2BTradingEngine), then BB won’t even pass basic developer self-tests. Sample code may not be available — see my post http://wp.me/p74oew-1oI. So developers must bite the bullet and take AA approach. This is my sweet spot. See
https://bintanvictor.wordpress.com/2016/10/31/strength-gtd-with-non-trivial-researchinnovation/ https://bintanvictor.wordpress.com/2016/03/30/theoretical-complexity-strength/
https://bintanvictor.wordpress.com/2016/03/03/tech-specializations-no-such-luxury-in-singapore/

On Wall St projects, I learned to abandon AA and adopt BB.

On Wall St interviews, I still adopt AA.

On West Coast, I think AA is more needed.

each year grow towards a principal developer

A lead developer of any system (not just a component) needs a few years experience [1] building a similar system. Each year, I hope to accumulate relevant know-how [2]. I’m not a people manager or project manager but I can be the principal developer.

Pause ….

However, I’m not accumulating “everything relevant” because … A lot of the know-how is never quizzed in tech interviews. Here are a few examples
* build/release/test/cont-int
* version control
* peer review/QA/release process
* wiki, jira
* back-up
* IDE
* system admin scripting
* job schedulers

As a wall st contractor, I used to work in kinda greenhouse environment with all of the infrastructure taken care of, so I just focused on “get-code-done” and get productive with it. It’s an easier life.

Non-lead interviews often focus on implementation details. Benny Zhao once /identified/ just 4 or 5 clusters like 1) algo 2) language details 3) OO design. This skill set is insufficient for a principal developer.

[1] but look at Saurabh, Sundip, Kirupa, Yang … they came in with only something slightly similar
[2] not useful to focus on coding year in year out.