##skillist to keep brain active+healthy

— generally I prefer low-churn (but not lethargic) domains to keep my brain reasonably loaded:
[as] classic data structure+algorithms — anti-aging
[s] C/C++,
SQL,
c++ build using tool chain, and shooting using instrumentation
c++ TMP
[s] memory-mgmt
[s] socket,
[s] mkt data,
[s] bond math,
basic http?
[a=favor accu ]
[s=favor slow-changing domains]
— Avoid churn
  • jxee,
  • c#
  • scripting,
  • GUI
— Avoid white hot domains popular with young bright guys … too competitive, but if you can cope with the competition, then it could keep your brain young.
  • quant
  • cloud? but ask Zhao Bin
  • big data
  • machine learning
Advertisements

##conquests since GS #j,c++..

Until I left GS, I didn’t know how it feels to “conquer” a sizable, lucrative tech skill. Such a tech skill represents a specific job market with supply and demand.

  • perl? not sizable not lucrative 😦 Until 2011, Perl was my only core competency 😦 How lucky am I now !
  • coreJava QQ (as defined on WallSt) .. was my first conquest. After this conquest I haven been trying to replicate my success story, while many peers stayed in one firm in order to move up.
  • SQL? Many interview topics not explored, but now SQL is no longer a sizable job market:(
  • MOM? Not sizable
  • sockets? not so sizable, not yet conquered
  • bond math … was a small conquest
  • c++ QQ .. was my 2nd conquest, as experienced in 2017 onward
  • CIV .. was my 3rd conquest. Growing bigger, though I only rate myself B among west coast candidates.

c++ecosystem[def]questions are tough #DeepakCM

C++ interviewers may demand <del>c++ecosystem knowledge</del> but java also has its own ecosystem like add-on packages.

As I told my friend and fellow c++ developer Deepak CM,

  1. c++ecosystem QQ questions can be more obscure and tougher than core c++ questions
    • tool chain — compiler, linker, debugger, preprocessor
    • IPC, socket, pthreads and other C-level system libraries
    • kernel interface — signals, interrupts, timers, device drivers, virtual memory+ system programming in general # see the blog catetory
    • processor cache tuning
    • (at a higher level) boost, design patterns, CORBA, xml
    • cross-language integration with python, R, pyp, Fortran + other languages
  2. java ecosystem QQ questions are easier than core java questions. In other words, toughest java QQ questions are core java.
    • java ecosystem questions are often come-n-go, high-churn

Low level topics are tough

  1. c++ ecosystem questions are mostly in C and very low-level
  2. java ecosystem questions are usually high-level
    • JVM internals, GC … are low-level and core java

 

##%%c++(n java)memoryMgmt trec : IV

Q: Your resume says “memory mgmt” for c++, so what did you do personally, not as a team?
A: I will talk about the features in my system. Most of them are not written by me, honestly

  • per-class: restricting allocation to heap or stack
  • per-class: customize operator new for my class
  • per-class: disable operator new for my class and provide allocateFromRingBuffer() API
  • per-class: move support
  • static thread_locals
  • ring buffer to eliminate runtime allocation
  • custom smart pointers
  • memory leak detector tools like valgrind+massif .. breakdown heap/non-heap footprint@c++app #massif
  • RAII
  • pre-allocate DTOs@SOD #HFT #RTS
  • customize new_handler() — I didn’t write this

Q: java

  • GC tuning
  • memory sizing
  • JDK tools like jhat
  • memory profiler tools
  • string intern
  • investigating memory leaks, all related to GC

##coreJava QQ topics: stand-out from crowd

Q: what core java QQ topics (not necessarily skills) to make you stand out from the crowd?

  1. (Best example) low-level core threading features — higher market value than ecosystem features like concurrency utilities, which are so numerous that most candidates are not expected to know in-depth
  2. jGC
  3. latency tuning — including JIT
  4. collections internals

— second tier QQ topics

  • reflection, AOP, bytecode engineering? never popular
  • advanced generics? out of fashion
  • class loader internals? out of fashion
  • JNI? never popular
  • jvm internals? never popular and never needed on any project

##[18]portable skills like coreJava/c++

I have found core java and core c++ outstanding domains for my personality. I feel there are not too many similar domains with

  1. feature: standardized knowledge highly portable across teams
  2. feature: requires more than such (simple) knowledge that anyone can pick up in a crash course
  3. feature: low churn high stability

… so we can hope to accumulate. Here are some comparable domains:

  • pure algo 🙂 yes standardized, non-trivial questions. I can accumulate. Better than brain teasers
  • socket and sys programming:) Yes standardized questions, including malloc, sharedMem
  • SQL 🙂 good despite shrinking demand.
    • Can we say the same (“shrinking”) about ANSI-C? Well, lots of c++ questions are C.
  • bond math 🙂 not so deep but good
  • JGC + jvm tuning? 😦 churn
  • python ? 😦 Deep questions are rare and non-standardized, like concurrency, meta-programming..
  • quant-dev domain 😦 questions are standard, but too few jobs
  • algo trading? 😦 i don’t think they ask any standard question

## 2 portable skills gained{each job

These are portable, longevity skills either for IV or GTD.

+1 +2 +3 .. = subjective valuation as of 2018
[grey color = portable financial domain skill, not tech skill ]

  • NIE +1 — 1) hacking open source php
  • GS +3 higher value than all other jobs —
    • 1) java GTD 2) SQL 3) database tuning 4) Database batch processing architecture
  • Citi +1~2 — 1) bond math 2) market-maker quote pricing system “architecture”
  • 95G +2 high value over mere 5 months —
    • 1) java wait/notify 2) store-proc for trade booking 3) FIX connectivity 4) Tibrv (basics only) and MOM system architecture
  • Barc +1 — 1) option math 2) analytics library integration
  • OC +1 — 1) c# GTD 2) basic MSVS 3) quote distribution architecture
  • Stirt +0 — 1) curve building basics. This job paid reasonably well, so I won’t prefer it for a low-paying top “contexx”
  • Mac +1 — 1) serious use of standard (portable) python 2) devops including git
  • RTS +2 — 1) c++ instrumentation 2) raw mkt data feed 3) order book replication 4) bash/perl for personal automation 5) socket (basics only)
  • mvea +1 — 1) lowLatency, high volume eq trading architecture 2) c++ (not C) GTD in one of the biggest c++ trading platforms including multi-file build, gdb, crash investigation 3) advanced language features including TMP and shared_ptr

## execution algo + other domains / skills

OMS — is probably a core module in execution system. Equity and FX systems tend to need OMS but many trading desks presumably need no OMS, very simple OMS or off-the-shelf OMS. A buy-side trading desk may simply use the OMS of the broker. The same could be said of “execution systems” like VWAP algos.

Therefore, I feel the importance of OMS/EMS is over-stated.

SOR — is more niche, less “generic” a skill, but as a module is more visible to the business.

FIX connectivity — is a more generic tech skill and shows resilience to churn.

mkt data — is more “separate” from the other skills.

compare%%GTD to single-employer java veterans

Me vs a java guy having only a single long-term java project, who has more zbs (mostly nlg) and GTD power, including

  • performance tuning in thread pool, DB interface, serialization
  • hot-swap and remote debugging
  • JMX
  • tuning java+DB integration

When it comes to QQ and coding test scores, the difference is more visible than it is with GTD/zbs.

Conclusion — over 10 years, your portable GTD power grows too slow if you stick with one (or very few) system.

Am I advocating job hopping? Yes if you want to remain an individual contributor not aiming to move up.

 

[18] IV (!! GTD)body-build`logbook by skill

I’d like to keep the table columns simple.

muscle delta when duration intensity
2D coding Q 3->5 2018 2D 3
perm/combo 5->8 2017 4D 3
string/array 6->7 2
backtracking 1->6 2018 2D 3
tail recursion 0->5 2017 x hours 1
big O analysis 7->8 2018 1
sorting algo nlg 7->8 2018 x hours 1
advanced recursion 3->5 2018 x Days 3
whiteboard best practice 6->7 2018 2 days 3
shared mem 2->4 2018 2 H 2 coding experiment
c++ for coding Q 3->8 2017 10-20 D 2 bbg
socket 3->6 17-18 x days 2 thanks to IV, not project
rvr/mv/forward 2->5 16-18 x days 2
noexcept 0->5 2018 1 H 1 Trex IV
other c++11 2->4 1
py basics 4 coding test 3->5 2
FIX?
STL?

c++QQ/zbs Expertise: I got some

As stated repeatedly, c++ is the most complicated and biggest language used in industry, at least in terms of syntax (tooManyVariations) and QQ topics. Well, I have impressed many expert interviewers on my core-c++ language insight.

That means I must have some expertise in c++ QQ topics. For my c++ zbs growth, see separate blog posts.

Note socket, shared mem … are c++ ecosystem, like OS libraries.

Deepak, Shanyou, Dilip .. are not necessarily stronger. They know some c++ sub-domains better, and I know some c++ sub-domains better, in both QQ and zbs.

–Now some of the topics to motivate myself to study

  • malloc and relatives … internals
  • enable_if
  • email discussion with CSY on temp obj
  • UDP functions

q[java ecosystem]==jxee+tools+..

This classification helps me organize my java learning, but let’s not spend too much time on this imprecise concept —

So-called “java ecosystem” is anything outside the “core java” stack and include jxee plus ..

  • GC, JIT
  • JNI
  • swing, AWT
  • ajax integration
  • protobuf/json
  • tools: eclipse, Maven, CI tools,
  • tools: JDK bundled tools like jhat, visualvm

 

##some c++QQ topics I took up since Apr17

Combine with https://bintanvictor.wordpress.com/wp-admin/post.php?post=24064&action=edit ? no need… Don’t spend too much time! I keep this list only as motivation and reward, but I don’t want a long list. I want heavy-hitters only, non-exhaustive.

Note “Take-up/Conquest” mean … I now feel as competent as my interviewers on that topic.

Note QQ means … hard topics unneeded in any project.

  • RAII+swap(), the killer combination, in the sports sense of “kill”
  • ADL
  • [3] sfinae
  • [3] crtp
  • covariant return type and virtual ctor i.e. clone()
  • static_assert? Non-trivial for those uninitialized with TMP
  • [3] alternative to virtual functions
  • singleton thread-safety
  • [3] heap memory mgmt strategies
  • [3d] make_shared benefits and limitations (Scott Meyers)
  • [3] shared_ptr concurrency
  • [d] Common mistakes involving shared_ptr
  • weak_ptr vs raw ptr
  • unique_ptr ownership transfer, reset^release
  • factory returning smart pointers
  • [d] emplace employing perfect forwarding
  • [d] naturally occurring temp objects vs const ref and RVO
  • [3] just when std::move() is needed .. see email to CSY
  • std::move usage (outside big4)
  • [d] rval objects vs rvr
  • [3] placement new
  • [3] MI using Abstract Base Classes
  • [3] TCP congestion control
  • TCP OOS control
  • reinterpret_cast
  • non-blocking socket functions
  • — topics to be conquered
  • [3] TCP resend
  • TCP buffer sizing
  • std::forward()

[3=top-33 favorite topics among ibank interviewers]
[d=reached some depth of understanding]

some Technicalities for QnA IV

We often call these “obscure details”. At the same level, these are a small subset of a large amount of details, so we can’t possibly remember them all 😦

Surprisingly, interviewers show certain patterns when picking which technicality to ask. Perhaps these “special” items aren’t at the same level as the thousands of other obscure details??

These topics are typical of QQ i.e. tough topics for IV only, not tough topics for GTD.

  • archetypical: which socket syscalls are blocking and when
  • $LD_LIBRARY_PATH
  • hash table theoretical details? too theoretical to be part of this discussion
  • select() syscall details
  • vptr, vtable

 

##some python constructs to understand

These are the features I feel likely to turn up in production source code or interviews, so you need to at least recognize what they mean but need not know how to use exactly. (A subset of these you need to Master for writing code but let’s not worry about it.)

List of operators, keywords and expressions are important for this purpose

Most built-in Methods are self-explanatory.

##wall-blaster handy techniques to impelement features

Challenge — to implement non-trivial logic quickly, cleanly, with assurance and peace of mind.

Default (stupid) solution is a java/c++/c# implementation – painful. The challenge feels like a stone wall to drill through. The following technologies are good dynamites. (Additional Background — often, there’s a legacy Prod codebase…)

#1) regex
#2) SQL execute immediate
) perl
) reflection
) rmi
* embedded db
* multiple-statement SQL batch to implement complex logic
* byte code engineering
* DB constraints to replace app validation
* MOM as concurrency tool
* DB as concurrency tool
* file system as concurrency tool
* jmx
* annotation

Also-rans
* javascript, if this fits into the infrastructure
* xslt
* aop

I don’t fancy dependency-injection, ORM, ajax.

##1 week in ML — stay marketable and employable in financial IT

(another blog post)

Problem — in any bank, a contractor or employee at any age could lose job due to many reasons – economy, new boss…
Problem — too many young developers join the job market every year, often from overseas
Problem — we aren’t getting younger 🙂
Problem — waves of new technology

If anyone doesn’t feel the need to worry about it, i feel that’s unwise. One of my strategies to survive is focus on old and core technical topics –

* threading. Wall street always likes threading, even if their system doesn’t use threading
* data structures – java collections, STL, trees, pointers, basic arrays and basic strings (both outdated but extremely relevant)
* SQL
* OO – static vs non-static, HASA vs ISA, abstract, constructors/destructors, proxy
* DB table + data model design
* DB tuning

Other areas I like
– memory mgmt
– messaging
– unix
– sockets

These technologies didn’t change for the past 10 years, except some new GC algorithms.

##front office technical know-how

–Most imp to hiring and productivity–

swing, wpf — more complex than all other tools
pricing math
pnl rollup, MTD, unrealized.. is imp to traders
fire fighting
c++ …rv
mastery over db query 

FIX
general low latency techniques
gemfire — imp to both risk and real time pre-trade. Let’s xx2set up
trade booking, cancel, MOD
java OOM — simple techniques
position mgmt is important to traders
trade blotter basics

##extend lead over non-trading developers

(context — IV, not performance) When you compete for trading jobs in SG, you must show convincing value-add otherwise why should they pick you rather than some regular java guy in SG with 15 years xp?

— ranked in terms of entry barrier

  1. * c++
  2. * WPF? i think in a growing number of cases, a java guy can help out on the front end
  3. * socket, TCP/IP, FIX
  4. * analytics
  5. * low latency
  6. * distributed cache
  7. * swing — few java guys do swing but swing is fading
  8. * db tuning
  9. * threading,
  10. * java memory management
  11. * MOM — RV, MQ
  12. * biz jargons — they buy it
  13. * serialization
  14. * CEP??

##java GTD knowledge2deepen on daily basis {SocGen

(blogging again)

Even though we can pass most java interviews (in the east coast:)), I find myself learning bits of practical java how-to on a *daily* basis.

* IDE
* ant and JVM launch script
* reflection
* thread pool
* concurrency constructs
* serialization
* callback listeners
* MOM
* generics
* JMX
* swing

They present everyday challenges to a intermediate developer like me, so better knowledge does improve problem solving, code t r a c i n g and design. These challenges illustrate one reason why I feel less proficient in java than in Perl. I can solve most *practical* Perl problems. Java is bigger and present more “problems”. As I spend more years in java, I hope to pick up more know-how. Therefore java remains my primary study subject, along with c++.

In the list above, I have deliberately omitted vendor products as they aren’t core java IMO. But obviously there’s a lot to learn in them.

* spring
* hibernate
* gemfire
* fitnesse

##java value-add` GTD/KPI skill when u join a trading team

What skills make you a more resourceful and *productive* developer to a fast-paced trading system? These usually have little to do with arch design. Almost never asked in interviews, ironically.

See also my post on 2 types of tools.

1) IDE remote debugging
** breakpoint on exception
** conditional breakpoint
** compile with local vars
2) IDE refactor — rename, search

) reflection, dynamic proxy
) profiling
) jconsole, JMX
) requesting Runtime.gc()
) generics for algorithm abstraction but this tends to be too complex
) code gen? cache triggers

Threading? nope it seldom adds too much biz value

##what zbs skills differentiate java trading developers

I said before that core java trading system developers need a few special skills

1) multi-threading
2) messaging
3) data structures and algorithms
?) distributed caching — trading system often avoid hitting database, so they keep large amounts of data in memory
?) java tuning
?) distributed computing using RMI, web services, ejb etc
?) Spring DI beyond MVC
?) serialization

#1 and 2 are the most broadly recognized requirements.

Now I feel most of these skills are built on top of fundamental java skills. Messaging system uses threading, serialization, data structures. Caching uses threading, data structures, serialization …

I don’t know a well-defined list of “fundamental java skills” but interviewers often zoom into threading, collections and OO. Among these, threading presents the highest complexity, but most applications stick to well-tested fool proof patterns.

I feel java generic collections (in a multi-threaded environment) is perhaps the 2nd in complexity. (I realized this after reading c++. C++ generic collections are quite a bit messier than java.) Most java developers I know never need to go beyond scratching the surface of generics.

In conclusion, to pass interviews, over-prepare on threading and collections, and prepare for high-level questions on messaging. To be a good trading programmer, become really good at threading and serialization… among other things

##java survival skills in a trading system@@

* java/javac command line options
* null pointer prevention
* familiar with ide, esp. (remote) debugger
* familiar with logging tools/techniques to simplify logging to the extreme
** add logging without impacting existing code.
** tracing proxies?
* lots of asserts
* jconsole* get compiler to help prevent potential issues
* memory profiler?
* thread dump analysis
* sound concurrent architecture (high-level) to(low-level) idioms
* spring

Other survival skills/toolkit
* cvs familiarity
* lsof, tcpdump
* core dump analysis
* system memory investigation? never seen such a need
* unix scripting? probably
* vi, find, grep…

##[10] hard CORE c++java skills across industries

— c++
void ptr
double pointer
ptr to ref; ref to ptr
ptr to method
new/delete global overload to intercept new/delete operations
allocators
debugger
scoped typedef, namespace
vptr/vtbl
static initialization sequence

—java
Note: MOM, spring, hibernate, data grid … aren’t core java and out of scope.

reflections
threads
generic

— Below are needed only in rare situations, IMHO
memory management — leak detectors, garbage collection, weak (soft) references, memory profilers,

jdk and cglib proxy
JMX
custom class loaders
byte code engineering
JVMTI — (eg jprobe) powerful tools to analyze production systems.
real time java

##some java skills no longer]demand

(another blog post)

Any powerful technology requires learning. Whatever learning effort I put into EJB, Corba, struts, JSP, JNI, J2ME, Swing, AWT, rule engines .. are becoming less and less valuable to employers. The march of java is merciless. How merciless? Even the creator of java – Sun microsystem – has to go open source on both java and Solaris just to survive, and sell itself to Oracle.

I am now very careful not to invest myself too heavily into anything including popular must-know stuff like Spring, hibernate, distributed cache, EJB3, AOP, CEP, JSF, GWT, protobuf, web services, design patterns, RMI …

I think most of the above are jxee, not coreJava.

Instead, I seek stability in older core java technologies like threading, collections, serialization, messaging, reflection, …

 

(C++ is even older and more stable, but this post is all about java.)

##[09] productivity skills in some financial firms #Le2xr

XR,

(to be published on blog, no need to reply)

I wrote about the same topic in a 2007 email to you — http://bigblog.tanbin.com/2007/12/interview-topics-are-irrelevant-now_10.html, and you replied — http://bigblog.tanbin.com/2007/12/fwd-interview-topics-are-irrelevant-now.html.

My GS experience shows that I could be somewhat slow initially[1], when getting up to speed on an unfamiliar system. Now i’m feeling a similar “teething pain”.

Now, what technical knowledge do highly *productive* developers tend to have in common?

* IDE — big area i am lacking, though i used IDE’s extensively since 2006.
** ant integration and external build tool
** cvs/svn integration
** content assist
** setting classpath
** using multiple versions of jre system library
** remote debugging? few know it but it’s powerful

* java generics
* junit — how to run, debug, write
* cvs/svn — synchronization, tortoise, branching, tagging
* ant, maven, ivy
* sql
* unix — grep, find, vi …

In GS and my new team here, I have met a few experienced java developers who know Eclipse a lot more than I do, just like I know Perl and Unix more than others. I learned so much Perl by reading and using advanced features — things like the O’Reilly cookbook tricks and tips.

[1] initial 2-4 months

skillist^specialize^accu^churn^10yDirection^domainBet

  • Skillist — the leaf-level tech skills. This label is most relevant on old posts without label.
  • 5y_domainBet — helps me choose which industry sector to invest my time and accumulate.
    • Not necessarily a list.
    • 5Y horizon
  • 10y_dir — longer broader view
  • specialize — I’m a specialist type of professional, not a generalist or manager. These posts help me position myself, not necessarily restricting to a particular domain.
  • accu — less specific than “specialize”
  • churn
  • t_skillist and t_feature tags rarely but could be on one post