A paradox to share with you. For years I often wonder
Q1: why does wall street (a shorthand for “world financial centers”) pay something like double the salary compared to main street, for a senior programmer? (Disclaimer — Wall St is not the only sector; premier tech firms also pay well above main street.) On many teams including my current team, I see people doing main street programming work, using main street technical skills without any financial domain knowledge. Why pay us so much higher than main street?
I feel this is a non-trivial paradox. A deeper understanding helps us manage our career, manage our competitiveness, manage our job security, manage our workload, manage our value-add.
In this letter I use “ibank” as a shorthand for all the employers in the investment-banking and security trading business, sometimes including buy-side. Yes I’m deliberately vague and imprecise.
Factor 1: ibanks have higher margin than main street, even after 2008, and crucially a small percentage of employees are responsible for the profit. Many ibank jobs are, to be honest, not critical to the money-making machinery, but many developer jobs are. To put things into perspective, traders, fund managers, deal-makers, advisors/salesmen … are in the profit center and even more instrumental to the money-making machinery. The closer to the money, the higher is one’s value to the ibank. Highest paid wall street (non-manager) tech roles are probably the high-frequency low-latency experts because they are closest to the money. Their code has a most direct impact on the competitiveness of our trading engine in the arms race. Similarly, our pricing system calculates the prices sent directly to the market and have a direct impact on our competitiveness.
Therefore, many senior programmers are considered instrumental/important for the high margin of an ibank.
Factor 2: these tasks are often fairly complex and easy to mess up, and therefore calls for expertise, backed by track record. (To be fair, these are not the most complex.) For comparison again look at the most important roles in ibank profit centers. There’s usually a lot of decision-making, a huge flood of information to digest. An ibank can hire fresh graduates to create trading applications, but they are likely to fail to meet some of the requirements. In and outside Wall St, software dev is knowledge-intensive, methodical, intricate, even fragile , unlike many other jobs in finance.
 sensitive to human mistakes. That’s why we say 99.9% of software have bugs.
Therefore ibanks generally recognize in-house app dev as high-value (sometimes instrumental/enabling) and complex. I have heard many times “we can’t introduce this new instrument to the market without the IT system”.
Factor 3: Just a small number of experienced coders are needed. In main street, we see armies of developers for a given project. (Lee Chong Wah may disagree.) On Wall St, team size is cut to a third or a quarter. In so many ibanks I have worked, I always see a lot of senior developers and few junior developers. I feel OCBC (and other traditional banks) see developers as the same “armies”.
Therefore ibanks have high profit, and want to use part of the profit to attract a small number of elite developers. This is a “perfect storm” for higher salary. Now, in reality, a team of 3 to 7 senior developers would have just 1 key developer, so the other guys are not that critical. Perhaps one of them is doing nothing special at all. Low-value, low-importance, mundane code, but still paid well above main street. (I might be one of them now.)
Factor 4: During recruitment, ibanks typically have an adequate budget for a senior developer. If market rate is 140k, and their budget is 90k then they eventually hit the reality that strong candidates (with track record) won’t be interested, so they end up scraping the bottom of the barrel. Mostly tough and unsuccessful.
So managers go get a sufficient budget, screen, select and offer to this fairly strong candidate, but for various reasons he doesn’t get to play that key role. He may even end up with some truly unimportant modules, somewhat over qualified and overpaid. Managers can’t just drop him, but can leave his contract (if contractor) to expire without renewing. If not a contractor, then bonus is in question. If the guy is actually good but the mundane job must be done in the team, then maybe unfair to mistreat him….
That’s my answer to the opening question Q1. But there’s another question, because ibank managers are not stupid….
Q2: For the non-critical roles, why don’t they hire strong main street developers and pay them main street salary, to do whatever main street work there is in the project? No track record needed.
I feel many enlightened and bold ibank managers actually do that, but there’s a risk of uncertainty in following that route. Instead, most managers much prefer candidates with proven track record in a similar project. Also, recruiters are reluctant to submit profiles without relevant experience. Therefore, main street developers are largely excluded from the talent pool.