indexing inside a value object – in-memory OODB

In Quartz OODB (sandra), indexing among a large # of deals is indispensable — Avichal… Otherwise I believe you have to read entire haystack looking for needles.

This post is mostly about indexing INSIDE an object. For a dictionary-based value-object[1] in OODB, It’s good to assign a small integer “fieldId” to each field.

I was told that many dict-based objects (protobuf etc) use the same technique — assign a fieldId to each key-value pair. So this.getAge() would do an array lookup rather than a hash table lookup.

Note java/c++ objects don’t need this, because compiler knows the exact “offset” of this.age field relative to the “this” pointer, typically the lowest address of the “real estate”.

SecDB value objects are probably hashmap-based (according to some veteran), so for instance it’s easy to add a new “ref2” field to existing Position objects, without recompiling java or c++ apps. This flexibility is anti-java, anti-SQL and probably slow, so fieldId might help but not sure if it’s adopted.

To simulate this in java, your object would get a pointer to a Map field.

[1] not the key objects in a pair-object. Remember most in-memory DB use hashmaps.


Boss is less reliable a protection than your own survival skills

On a perm job, i can tell mgr my family’s need for job security and seek monthly feedback to avoid layoff. But If a new mgr takes over or profit declines, you can’t control much of anything. This pattern is ruthlessly, mercilessly universal.

Your knowledge and track record might be less portable than something like a citizenship, GC, a degree.

Boss and employer are unreliable but investing-in-yourself may affect your performance on the job.

likability vs technical caliber

Q: Out of 10 colleagues, suppose “Beth” is the 2rd most likable and “Yen” the 2nd least likable. How serious is the gap in their likability? Is Yen  really that unpopular?

Perhaps Not if you think carefully. If you rank the 10 people in your own team, the 2nd least likable person (the “Yen” in your team) is probably not bad. Out of 10 colleagues, it’s rare to find a single person clearly “nasty”. You may find the 6th through the 10th are all reasonable teammates and you have a hard time ranking them, (but consider bonus/layoff season). If you look at the last ranking colleague, perhaps he has a strong personality/self-centeredness, or she’s always too busy to help you when you need her, or she forgets/ignores your requests, or he is a bit aloof and cocky and not that warm, or she’s loud, or he’s too nice with the opposite sex, or she’s working too hard and makes you look lazy, or he has less humor than others …

(By the way, for focus and clarity, i will disregard the important fact that any measurement used in such ranking is extremely vague. No objective measurement, except surveys.)

Another ambiguity with likability is, the colleague you rank in the 2nd half may be my favorite type of personality. Clearly, we should both remove personal “en1yuan2” — someone who helped me a lot i may not like, but someone who made me a lot of trouble i may find quite amicable and fun to be around.

Another ambiguity is cultural boundary. The “nice” personality in one culture is usually considered polite and welcome (but probably not charming/endearing) in another, but some 2nd-half kind of professional guy in China is quite possibly viewed with admiration and attraction by some Americans, for example. Note there are infinite types of 2nd-half personalities!

Another ambiguity — I may agree with you this guy is nice and polite, but in a secret ballot I just won’t put him in the first half of likability. Nevertheless I may elect someone not so nice because she accepts ME better and makes ME comfortable. Likability is something personal to ME, just like clothing.

In a twist beyond explanation, I may never invite the nicest colleague for lunch (perhaps he’s too perfect and way above me), but I do share lunch with a lot of colleagues less than “perfectly likable” — I call it rapport.

Likability becomes serious at review/promotion/bonus time but i think only the manager’s (+ a few other managers) view counts.
It becomes serious when — you try to transfer internally.
It becomes serious when — you are seen as a go-to person, a helpful teammate — reputation. People would mention your name when they need or received (non-technical) help.
It becomes serious when — you need a massive help from (multiple) someone.
It becomes serious when — you organize an event.
It becomes serious when — you ask people for a character reference.
If a colleague confides in you, that’s a sign of your likability. Does it buy you anything? You bet.

Now consider technical caliber of engineers. The effectiveness, personal contribution and value-add of the 2nd decile vs 2nd last decile is somewhat more significant and visible. The difference can be objectively assessed. A mediocre engineer vs an insightful or fast-learning or quality-conscious or clear-thinking or systematic engineer — real difference. It shows in the work.

That partly explains why the very nice people I know often make slow progress professionally, but “mad scientists” often move up on the strength of technical caliber. My view is biased. I know many who believe it’s more important who you know than what you know. That’s a question for the next blog.

How about business analysts, accountants, professional traders, researchers and salespeople?

field hiding by subclass, learning notes

(c# has the trickiest hiding rules; c++ has  more than java.)

First, remember the onion model described in other posts. To rephrase one more time, each instance field occupies memory “cells” in a layer of the onion. If class B has an instance field vf, and subclass C has a same-name instance field vf, then a C “onion” has storage for both fields.

myRef.vf resolves to one of the 2 fields at compile time depending on declared type of myRef. P40 [[java precisely]]

On P25 [[java precisely]], B has a static field sf, and so does C. The C onion in memory has NO storage for either field, but it has a pointer to the class object, which has storage for the field.

Q: does the C class object have storage for both static fields?
%%A: probably no. C class object just needs a pointer to B class object.

Question: “myRef.sf” refers to …? P40 [[java precisely]]
A: At compile time, if the C onion is declared as type B, then the field is B.sf
A: At compile time, if the C onion is declared as type C, then the field is C.sf

I feel the static and non-static cases are similar — both compile-time binding. This is the simple rule for field binding. For method call binding, static and non-static behave differently. Remember that runtime binding is very strict and requires non-static and non-field, among other conditions. Failing any condition means compile-time binding.

Wall St IT project timeline is no joke

When I first joined GS, I was in awe of my managers’ /commitment/ to project deadlines. Before GS, I always felt deadlines are usually arbitrary and deserve little respect. I used to feel deep in my guts that developers ultimately hold an upper hand over managers *when* (not IF) deadlines become difficult. Basically, I feel the regular developers are hard working but aren’t saints, so we are all to varying degrees lazy and undisciplined, and tends to leave too much work to the end. On the other hand, managers can never be 100% on top of every Dirty detail, the quick and Dirty, and the Dirt swept under the carpet, so they are always surprised at end of the time frame.

After a few years in GS, I feel on Wall St, a lot is at stake when the timeline pressure heats up. Managers are visibly concerned about missing deadlines. Developers often work long hours. Major source of developer stress (and burnout).

If you feel elsewhere developers miss timelines 80% of the time, then it’s 20% in GS.

The hallmark of Wall St IT teams — pays more and demands more productivity (quicker delivery). Keyword is productivity.

Everyone can stick to a timeline if it’s long enough, but GS typically gives you one-tenth to half the timeline given elsewhere. It’s not uncommon to see a 10-month project squeezed into 1-month.

A lot of your career is at stake too. If you manage user/manager expectations well, then you could win big wins and move up.


That was my view while in GS. After many years outside GS, I feel Barclays and BAML try to emulate GS with up to 50% effectiveness. Outside GS, the other Wall St IT managers can also give demanding deadlines, but the consequences of miss is not so severe. If you miss one without some scapegoat, then you can get any amount of pressure including but not limited to

  • no pressure at all
  • fake pressure
  • asked to work OT
  • sidelined to less critical roles
  • given some coach to work with you on the same project
  • official warning
  • performance improvement process. (I hate this, though it is designed to protect the employee.)

real time risk – option mkt-maker

Core component is the valuation engine. Each vol surface gets rebuilt every 10 minutes. Positions are re-valued every minute on average.

I believe option position valuations change more often because spot moves more often than implied volatility.

(Similarly, in Sprite of Baml Stirt, the FX spot moves more often than fwd points. It also moves more often than interest rate market data, including IR futures prices.)

I was told it’s hard for any eq vol market maker to survive without a real time pricer. When client calls for a quote, it’s problematic to rely on a 2-hour stale vol surface.

Re-pricing in 2 areas —
1) position mark-to-market (most important)
2) outgoing quote/RFQ pricing

I feel these 2 areas are related. If you are a market maker in an index (stoxx for eg), then your own position will influence your outgoing quotes. I guess the Reason is — position means market-risk, and position means hedging. Everyone needs to hedge, esp. a market maker.

For volatility instruments, delta-hedging must be adjusted constantly.

all c# value types are effectively structs@@

This is probably imprecise language, but good enough for a beginner.

There are only 2 sub-types of value types — structs and enums. Note simple types (like int) are struct type aliases.

Now, enums are backed by integers (short, long etc), so it has to share a lot of basic features with integers. I feel ALL enum features are very similar to int features. I feel if enum shows any “deviation” from struct, it’s likely superficial. Therefore, essentially all value types are structs

I guess all value objects are struct instances that won’t reveal their addresses (but structs can be passed by  “ref”). The way to pass these objects around is copying/boxing.