## strategic TSN among%%abandoned

Q: name the most strategic trySomethingNew domains that I have since abandoned, given up, quit. How about the new plan to take on coding drills as a lifelong hobby?

Don’t spend too much time, because the answers are nothing new even though this is a decent question.

  1. — ranked by surprise
  2. algo trading? actually very few roles spread across a number of firms
  3. c#
  4. drv pricing quant
  5. real time risk as in GS, Qz and Athena
  6. RDBMS tuning
  7. MOM, async, message-driven design knowhow
  8. distributed cache like Coherence and Gemfire
  9. Solaris sys admin and DBA
  10. perl, SQL, Unix/Linux power-user knowledge? No longer a top 10 focus

—-

  • python? Not yet abandoned
  • web dev for dotcom? I did this for years then abandoned it. Many of the tech skills are still relevant like sessions, java thread safety, long-running jobs

market-depth^elite domains #jxee

I used to dismiss “commodity” skills like market data, risk system, JXEE… I used to prefer high-end specializations like algo-trading, quant-dev, derivative pricers. In reality, average salary is only slightly different and a commodity job can often outpay a specialist job.

As I get older, it makes sense to prefer market depth rather than “elite”(high-end niche) domains. A job market with depth (eg jxee, market-data, risk systems) offers a large number of positions. The typical salary of top 10% vs the median are not very different — small gaps. In contrast, the elite domains feature bigger gaps. As I grow older, I may need to reconsider the specialist vs generalist-manager choices.

Reminders about this preference (See also the spreadsheet):

  1. stagnation in my orgradient
  2. may or may not use my specialist skills in math, concurrency, algorithms, or SQL …
  3. robust demand
  4. low churn — a critical criteria whenever I mention “market depth”. I don’t like the market depth of javascript and web java.
  5. salary probabilities(distro): mgr^NBA#marketDepth etc

–case study: Algo trading domain

The skillset overlap between HFT vs other algo systems (sell-side, OTC, RFQ, automated pricing/execution..) is questionable. So is “accumulation” across the boundary.  There seems to be a formidable “dragon gate” — 鲤鱼跳龙门.

Within c++ based HFT, accumulation is conceivable. Job pool is so small that I worry about market depth. My friend Shanyou agreed that most of the technical requirement is latency. C/C++ latency techniques are different from java.

However, HFT developers seldom need to optimize latency

Outside HFT, the level of sophistication and latency-sensitivity varies. Given the vague definition, there are many (mostly java) jobs related to algo trading i.e. better market depth. Demand is more robust. Less elitist.

criticalMass[def]against churn ] tech IV

See also body-building impact{c++jobs#^self-xx

IV knowledge Critical mass (eg: in core java self-study) is one of the most effective strategies against technology churn in tech interviews. Once I accumulate the critical mass, I don’t need a full time job to sustain it.

I have reached critical mass with core java IV, core c++ IV, swing IV (no churn) and probably c# IV.

The acid test is job interviews over a number of years.

Q: how strongly is it (i.e. critical mass) related to accumulation?
A: not much AFTER you accumulate the critical mass. With core java I did it through enough interviews and reading.

Q: how strongly is it related to leverage?
A: not much though Critical mass enhances leverage.

Q: why some domains offer no critical mass?
A: some (jxee) interviews topics have limited depth
A: some (TMP, py) interview topics have No pattern I could identify from interview questions.

 

strategic value of MOM]tech evolution

What’s the long-term value of MOM technology? “Value” to my career and to the /verticals/ I’m following such as finance and internet. JMS, Tibrv (and derivatives) are the two primary MOM technologies for my study.

  • Nowadays JMS (tibrv to a lesser extent) seldom features in job interviews and job specs, but the same can be said about servlet, xml, Apache, java app servers .. I think MOM is falling out of fashion but not a short-lived fad technology. MOM will remain relevant for decades. I saw this longevity deciding to invest my time.
  • Will socket technology follow the trend?
  • [r] Key obstacle to MOM adoption is perceived latency “penalty”. I feel this penalty is really tolerable in most cases.
  • — strengths
  • [r] compares favorably in terms of scalability, efficiency, reliability, platform-neutrality.
  • encourages modular design and sometimes decentralized architecture. Often leads to elegant simplification in my experience.
  • [r] flexible and versatile tool for the architect
  • [rx] There has been extensive lab research and industrial usage to iron out a host of theoretical and practical issues. What we have today in MOM is a well-tuned, time-honored, scalable, highly configurable, versatile, industrial strength solution
  • works in MSA
  • [rx] plays well with other tech
  • [rx] There are commercial and open-source implementations
  • [r] There are enterprise as well as tiny implementations
  • — specific features and capabilities
  • [r] can aid business logic implementation using content filtering (doable in rvd+JMS broker) and routing
  • can implement point-to-point request/response paradigm
  • [r] transaction support
  • can distribute workload as in 95G
  • [r] can operate in-memory or backed by disk
  • can run from firmware
  • can use centralized hub/spoke or peer-to-peer (decentralized)
  • easy to monitor in real time. Tibrv is subject-based, so you can easily run a listener on the same topic
  • [x=comparable to xml]
  • [r=comparable to RDBMS]

## UPSTREAM tech domains: defined by eg

It pays to specialize in a domain that helps you break into similar domains. (Unsuccessful with my quant adventure so far.)

Note “wrappers” API are never upstream. Very common in big banks including Qz.

  • #1 eg: [L] socket —- C++ is the alpha wolf, leader of the wolf pack
  • [D] market data —- Nyse/Nasdaq (binary) feed parer seems to be the leader, over the slower FIX feeds
  • [D] execution algo —- sell side equities desk seems to be the leader in terms of volume and variety of algos
  • data analytics language —- python looks like the leader
  • eg: [D] alpha —- equities trading is the leader
  • eg: threading —- java is the leader of the pack. In c++ threading is too hard, so once I master some important details in java threading, it helps me build zbs in c/c#, but only to some extent
  • eg: regex/string —- Perl is the leader
  • eg: [D] consolidated firm-wide risk —- SecDB is the leader. Applicable to buy-side? I think so.
  • lambda —- C# is the leader. Scripting languages follow a different pattern!
  • list/stream —- Linq is the leader
  • drv pricing theory —- Black-Scholes —- feels like a leader or upstream, but not /convincing/
  • [L] heap/stack —- C++ is the leader. Java and c# are cleaner, simpler and hides the complexities.
  • defensive coding —- (and crash management) C++ is the leader, because c++ hits more errors, like Japan on earthquake management.
  • eg: collections —- C++ is the leader of the pack. There’s a lot of low level details you gain in STL that help you understand the java/c#/python collections
  • eg: [L] latency —- C++ is the leader of the pack in low-level optimizations .. In-line, cache-efficiency, footprint..
  • eg: [L] pbref^val —- C++ is the leader. C# has limited complexity and java has no complxity
  • [L = lowLevel]
  • [D= well-defined application domain]
  • ? automated pricing —- no upstream
  • ? Fixed income trading systems —- no upstream. IRS^Treasury^credit bonds (trading platforms) have limited features in common.
  • [D] VaR/risk analytics —- FI derivatives in big banks seem to be the leader, but I feel one trading desk in one firm may not respect another system

pick java if you aspire 2be arch #py,c#

If you want to be architect, you need to pick some domains.

Compared to python.. c#.. cpp, Java appears to be the #1 best language overall for most enterprise applications.

  • Python performance limitations seem to require proprietary extensions. I rarely see pure python server that’s heavy-duty.
  • c#is less proven less mature. More importantly it doesn’t work well with the #1 platform — linux.
  • cpp is my 2nd pick. Some concerns:
    • much harder to find talents
    • Fewer open-source packages
    • java is one of the cleanest languages. cpp is a blue-collar language, rough around the edges and far more complex.

## any unexpected success@tsn@@

## past vindicative specializations is more comprehensive, but in this blogpost I want to start on a clean slate and answer the question —

Q: I have explored far and wide… for new domains, new challenges, so are there any unexpected successes of try-something-new?

skill discovered proven over %%strength 1-5 val 4 GTD val4IV
unix shell + unix admin 2000 Catcha 2Y 3 4 1
! perl 2000 Catcha 3Y 4 #many years ago 2 1
LAMP+js 2000 1Y 2 0 0
! python #popularity no surprise 1Y 2 #xp@Mac 2 3
! socket #and tools 2017 never 1 #lowLevel 3 if relevant 2
!! threading concept+java impl 2010 4Y 5 #theoretical 0 5
! x-lang collections 2010 5Y 4 #lowLevel+theoretical 1 5
! x-lang OO 2010 NA 4 #lowLevel 0 4
white board coding [1] 2011 2Y 2 @WallSt 0 3
c++instrumentation/build tools
! bond math 2010 Citi 1Y 2 1 if relevant 2
option math 2011 Barc 1Y 3 2 if relevant 1
fin jargon 2010 4Y #mvea 3 #know what’s relevant 2 2
finSys arch #abstract 2010 2Y 2 3 3

[1] Am more comfortable on whiteboard than other candidates.

Explanation marks means Surprise

## past vindicative specializations

see also — For the “domains nlg” (first 5 rows), marketable_domain_xp spreadsheet is more comprehensive but not necessarily more updated.

Quant and other Unsuccessful diversifications aren’t the focus here, but are listed below the table.

 scales 0-5 mkt value given my direction robust
demand
entry barrier accu achieved %%expertise
among peers@WS
tsn: determination
+sacrifice
I was delighted wage ROI val4IV
mktData #socket  4 growing not everyone has xp  2~3 2 #few worked@socket-level 2  5 some 2
bondMath  2 robust math is not natural
to most dev
 2  3 1 #spare time++  3 some 2
orderBook, OMS, FIX #Eq/FX  3-4 robust medium 1 #cod`IV 1 3 #CVA sacrificed  3 none
forex #sg 2 robust medium 1 1 0 #spare time 1 none
c++  5 ok not
growing
higher than I thought  5 #critical mass  3 5 #huge sacrifice  2 minimal
coding drill  5 Growing high 4 #XR disagrees  3 2 #spare time  5 none 4
python  4~3 growing low  2  3 #unknown 1  4 none
bash scripting + unix #devops  2 robust medium  4  3 0  3 none 1
threading xLang 5 growing high  4 #critical mass  5 0 4 some 5
(abstract?) xLang collection internals 5 unexpected strength the deeper the harder  4 #critical mass  4 0 4 some 5
RDBMS 3 shrinking low  4 #critical mass  4 0 #spare time 1 #under-whelming none
[00-06] web dev 1 sustained high growth low  5 #critical mass  3 0 3 none

–algo practice for IV
^ good amount of accumulation
^ my confidence is boosted esp. in c++
^ I’m rising to the challenge of coding test growing popular

–Python
^ boosts my IV confidence
▼no critical mass yet
▼low traction
▼don’t know my expertise relative to peers

–option domain knowledge including BS:
▼ much lower demand than bond math

— C# and XAML
^ Microsoft is a major force
▼not aligned to my current direction

–MSVS:
^ was my Achilles heel, now slowly gaining confidence

–C++ GTD and IV
^ I made real progress in 1) GTD and 2) IV but not during Mac days — wrong job nature
^ deepens my java understanding
▼ those high paying domains (HFT, quant) are too hard to break into

–MOM architecture, products…
▼falling out of favor
▼ not as widespread as I perceived. Probably used in finance and legacy systems

–javascript, php, mysql
▼not aligned to my direction

–gemfire
▼ market too fragmented

–EJB, Spring, Hibernate
▼falling out of favor

##[18] questionable tsn bets: past+future

  • boost beyond shared_ptr
  • functional programming
  • JMS?
  • weblogic
  • sendmail, makefile,
    • In contrast better bets at that time include apache, freeBSD, dns
  • option math?
  • quant dev
  • —— above are questionable bets
  • —— a subset of vindicated bets i.e. paying off above minimum expectation. See also ## past vindicative specializations
  • sockets!
  • bash + scripting
  • bond math
  • py
  • FIX
  • c++
    • c++ multi-file build, gdb, valgrind
    • c++11 — recently it started to pay off
    • pthreads — recently it started to pay off
    • template details — recently it started to pay off
  • —— next 10Y

[17] widely in-use, no longer quizzed #spring,SOAP.

I see a pattern — a new technology is getting adopted and quizzed in-depth at interviews. After 5 years, it is still a favorite, perhaps dominant solution, but 1) the know-how has become common knowledge and candidates are assumed to know it and 2) usage is now standardized and simplified, so the “bar” is lower, and candidates without the knowledge can easily pick it up.

No more in-depth questions needed. Therefore, time previously invested here is wasted, since only superficial knowledge is required now.

  1. Eg: spring/hibernate
  2. Eg: java servlets and JSP — From 1999 to 2008 these topics were heavily quizzed. Still widely in use but often invisible.
  3. Eg: Apache web server — In 2000 I was asked a lot on Apache details. Apache is still very popular. See https://w3techs.com/technologies/overview/web_server/all
  4. Eg: php — still widely used, but I feel not asked a lot. See https://w3techs.com/technologies/history_overview/programming_language/ms/y
  5. Eg: xml parsing — I used to get in-depth questions about DOM or SAX etc. Now I believe xml is still widely used
  6. Eg: web services, SOA, SOAP — Still very much in use
  7. Eg: HTTP protocol details like GET/POST, status codes
  8. Eg: Maven and Ant

Most of my examples are in the high-churn domains like Internet, mobile. I believe the same will happen to interview questions on big data, javascript, Android, iOS , blockchain, ..

The opposite list — some essential technologies underlying multiple technology waves were never heavily quizzed, but, after the waves subsided, remain rather relevant to many niche interviews.

  • TCP/UDP
  • SQL query — joins, subquery, case, ..
  • SQL and DB tuning
  • Unix automation — It can take years to become reasonably competent with all of bash, piping, subshells, trap, shell functions, string operators, file manipulation, and top 30 Unix commands
  • Unix system administration
  • Pthreads, a POSIX standard C library
  • http client programming
  • regular expression
  • Inter-Process-Communication
  • Java servlet session management
  • Java serialization
  • Java reflection

— You write —
There are still many projects using Spring. My current project is also using Spring, but it’s modified by internal team to create an internal framework. When people discuss in meeting, they say “Spring” to refer to this framework. But there are many pitfalls when I use it. To name a few:

  1. a) restful service is easy to implement in spring, ust add related annotations, but it doesn’t work, and after I spent a few days of research, I gave up and choose to use a internally created annotation.
  2. b) some configurations doesn’t work, parameters couldn’t be passed in. I still don’t know what’s the reason. The internal framework code is not accessible for other teams developers, so I don’t think it worth to spent more time to try to figure out.

For this project using Spring, the interview only mentioned this project is using Spring, but didn’t ask any questions about Spring.

For last year, I went through 5 interviews, 2 mentioned the projects are using Spring, and only one client asked some Spring questions.

I recall 5 years ago, 8/10 will ask spring and hibernate questions. Now, still a few clients asked Spring questions, but none asked Hibernate questions.

 

mkt-data tech skills: portable/shared@@

Raw mkt data tech skill is better than soft mkt data even though it’s further away from “the money”:

  • standard — Exchange mkt data format won’t change a lot. Feels like an industry standard
  • the future — most OTC products are moving to electronic trading and will have market data to process
  • more necessary than many modules in a trading system. However ….. I guess only a few systems need to deal with raw market data. Most down stream systems only deal with the soft market data.

Q1: If you compare 5 typical market data gateway dev [1] jobs, can you identify a few key tech skills shared by at least half the jobs, but not a widely used “generic” skill like math, hash table, polymorphism etc?

Q2: if there is at least one, how important is it to a given job? One of the important required skills, or a make-or-break survival skill?

My view — I feel there is not a shared core skill set. I venture to say there’s not a single answer to Q1.

In contrast, look at quant developers. They all need skills in c++/excel, BlackScholes, bond math, swaps, …

In contrast, also look at dedicated database developers. They all need non-trivial SQL, schema design. Many need stored procs. Tuning is needed if large tables

Now look at market data gateway for OPRA. Two firms’ job requirements will share some common tech skills like throughput (TPS) optimization, fast storage.

If latency and TPS requirements aren’t stringent, then I feel the portable skill set is an empty set.

[1] There are also many positions whose primary duty is market data but not raw market data, not large volume, not latency sensitive. The skill set is even more different. Some don’t need development skill on market data — they only configure some components.

value-creation sectors – turn tech2highest profit

Notes on Raymond Teo discussion:  west coast shops … creating big value through high user base, high data, …..innovation is high risk — “risk premium”. In contrast, Singapore IT projects have a fixed “value” i.e. the project budget. ($1m is quite big.) Directly impacts IT salary.

Those “upstream” sectors often needs a smaller number of elite techies.The “upstream” don’t really exist in Singapore. Therefore, only a small elite of  techies in Singapore would end up in the financial IT domain. The vast majority of techies live in a totally different world and earn S$60k (up to $100k) a year as non-managers.

As I stated elsewhere, an mediocre techie easily earn a Wall St salary on Wall St, but the same guy would earn $60k in Singapore.
—–
Database/OperatingSys vendors enjoy highest margins (in silicon Valley) but no all Oracle clients get the same ROI buying oracle. Let’s compare the major client-industries that are using technologies. In which sectors does technology create the most profit? Note consumers are the largest user population of technology, but consumers don’t make profit and are not an “industry”, and (crucially) won’t pay us a salary. Our focus here is the major commercial sectors/industries using technology to make profit.

Telecom equipment makers make heavy use of technology and have good margin. Like the DB/OS creators, a small number of elite engineers create industry standards…. Anyway, here are a few sectors.

#1a) HFT or algo trading, either sell side or buy side
1a) other trading engines
2) risk engine, but is the calc really useful, relevant and accurate to be reliable?
2) settlement system can be just as critical and valuable to an i-bank

1b) search engine?
1b) facebook, linkedin, Amazon …

1c) DB/OS/language, as described above, is not a client industry, but DB/OS profit margin is higher than all other domains.
?) Atlassian

4) Telecom equipment maker, as described above
5) commercial banking, corporate/retail banking, insurance. Lower margin than trading, but higher than non-financial service industries. ecommerce like amazon, ebay…? I feel thousands of online services offer useful services but have low margin. B2B better than B2C.

#eor

telecom operators? I know a lot of operators (US or S’pore) don’t pay very high, partly because technology is at equipment makers. health care? logistics? no no no

[17]U.S.hunt priorities #400w

To optimize for income, I would leverage on my 1) analytics 2) threading 3) SQL (+ possibly algo) expertise. Importance of income:

  • health insurance — will need for kids for sure
  • first few gigs might be low rate, given the dry season
  • home purchase in 5Y — confirmed requirement. If I want short commute + Chinese community, then price will be high
  • might go without income longer, like a few months
  • initial set-up cost — 5k-10k, but at this level adjustment on wife’s part will be tough. Even a 20k relief fund is drop in a bucket. Need 100k.
  • tail risk — I might come back to SGP sooner (like end of 2017) and take a low-pay job
  • tail risk — if GC process goes astray, I may go back to SG without a GC, so the X years I spent here I need to earn enough.

May not have much of a choice, given the slow c2c market right now.

My 2017 priorities:

  1. avoid churn? so avoid web, javascript, in-memory DB, spring/hibernate and most of the new technologies I hear about the west coast
  2. muscle-building contexx? so avoid java/sql? Better to deepen my c++ as a 2nd front. Willing to take lower salary? LG2 can postpone this to 2018/2019. See post on re-enter c++
  3. my c++ learning is reaching a critical mass
  4. —- unsorted lower priorities —-
  5. income to keep me feeling secure and self-respecting. I got this from the offers!
  6. flexibility to work from Singapore. Can spend more time with wife, parents and kids.
  7. location for upcoming home-purchase
  8. unlock new markets — west coast, c++, data science ..
  9. commute?
  10. chance to impress manager
  11. leisure time to exercise, improve c++, call home, learn driving etc

My ideal (yet realistic) 1st project:

  • slightly below (like 80%) my capacity. Chance to impress manager due to my specialty knowledge. “These managers could make things happen”.
  • has spare capacity to check out the potential homes
  • salary? LG2 like $65/hr
  • c++ or java
  • possibly west coast but the high rate is usually for FTE.

Alternatively, a temp contract to try c++ again, but only after I clear some high-end java interviews to build my confidence in earning capacity.

  • low salary like 120k
  • hands-on c++ (not C) on a large codebase to build mileage
  • NY or west coast or anywhere else

upstream, knowledge-intensive, high-margin sectors for S’pore

Upstream/strategic, knowledge-intensive, high-margin, high-barrier sectors for Singapore

Singapore is traditionally not a high-tech economy…. To compete, Singapore must seek greener pastures and keep a steady foothold therein. This is what every advanced economy does.

Role models? Switzerland, Hongkong, Taiwan, Silicon Valley, Boston

Entry barriers due to large capital, specialized know-how, talent shortage… Singapore EDB likes a sector with such barriers.

RnD – upstream, knowledge-intensive and entry barrier.

— Now the sectors —
$ finance — #1 most special (“sacred”) sector. One of the pillars of the economy but finance is first among equals [1]. Unlike other pillars, finance is part of the life support of this nation. Singapore gov needs long term strategies to protect (its currency and) financial health, even survival, of the nation. It grows a deep foreign reserve and invests it long term.

Not sure if finance is the most high-margin sector. Commercial banking is lower margin; AssetMgmt, security trading and investment banking create few jobs.

[1] if you shortlist the pillar industries of { oil, logistics}, where Singapore competes successfully on the global arena.

$ oil — perhaps the #2 sacred sector.
$ medical
($ life science? not sure what this includes)
$ electronics
$$ semiconductor fabrication – at the higher end of electronic sector
$$ IC design, esp. microprocessor design – still higher margin

$ telecom equipment and telecom operator — 2 big sectors by revenue
$ enterprise IT solutions, including software production and distribution
$ consumer IT product creation. Not a big sector, but look at Creative Lab
$ aviation — at the higher end of the logistic sector. Servicing, component design, research, airport ..
$ higher education and training

————
Now a small sample of the opposite list. Many traditional sectors don’t meet all the criteria in the title but do support a lot of jobs for Singaporeans —
– logistics?? Singapore’s traditional bread-and–butter economic contributor (commerce is another), but margin is deteriorating.
— distribution, warehousing
– commerce
— traditional retail – i.e. consumer shopping
– construction
– tourism and hospitality, casino
– entertainment – online/electronic gaming, music and film
– marketing and advertising, media, publishing, exhibition, conferencing

Many of these above sectors are domestic. They don’t directly contribute to the Singapore Team on the international front — contrast electronics, aviation, medical, tourism… Every advanced economy must show competitiveness and high value-add on a number of major global markets.

marathon – strengthen your(GTD+)lead in the pack#sg high-end

Y Lin pointed out those high-end jobs tend require specialized skills. I now feel concurrency is an example, in addition to —

latency – c++ is the default for extreme latency
latency – sockets
latency – market data
latency – java?
dnlg – math — better entry barrier, closer to business.
dnlg – jargons in FX, Rates…
dnlg – arch
FIX
MOM


real projects are completed in lowly tools like sql, java, C, scripts, but interviewers need to see much much more than those.

skillist^specialize^accu^churn^10yDirection^domainBet

  • Skillist — the leaf-level tech skills. This label is most relevant on old posts without label.
  • 5y_domainBet — helps me choose which industry sector to invest my time and accumulate.
    • Not necessarily a list.
    • 5Y horizon
  • 10y_dir — longer broader view
  • specialize — I’m a specialist type of professional, not a generalist or manager. These posts help me position myself, not necessarily restricting to a particular domain.
  • accu — less specific than “specialize”
  • churn
  • t_skillist and t_feature tags rarely but could be on one post