indexing inside a value object – in-memory OODB

In Quartz OODB (sandra), indexing among a large # of deals is indispensable — Avichal… Otherwise I believe you have to read entire haystack looking for needles.

This post is mostly about indexing INSIDE an object. For a dictionary-based value-object[1] in OODB, It’s good to assign a small integer “fieldId” to each field.

I was told that many dict-based objects (protobuf etc) use the same technique — assign a fieldId to each key-value pair. So this.getAge() would do an array lookup rather than a hash table lookup.

Note java/c++ objects don’t need this, because compiler knows the exact “offset” of this.age field relative to the “this” pointer, typically the lowest address of the “real estate”.

SecDB value objects are probably hashmap-based (according to some veteran), so for instance it’s easy to add a new “ref2” field to existing Position objects, without recompiling java or c++ apps. This flexibility is anti-java, anti-SQL and probably slow, so fieldId might help but not sure if it’s adopted.

To simulate this in java, your object would get a pointer to a Map field.

[1] not the key objects in a pair-object. Remember most in-memory DB use hashmaps.

Advertisements

Boss is less reliable a protection than your own survival skills

On a perm job, i can tell mgr my family’s need for job security and seek monthly feedback to avoid layoff. But If a new mgr takes over or profit declines, you can’t control much of anything. This pattern is ruthlessly, mercilessly universal.

Your knowledge and track record might be less portable than something like a citizenship, GC, a degree.

Boss and employer are unreliable but investing-in-yourself may affect your performance on the job.

likability vs technical caliber

Q: Out of 10 colleagues in your team, suppose “Beth” is the 2rd most likable and “Yen” the 2nd least likable. How serious is the gap in their likability? Is Yen really that unpopular?

Perhaps Not. If you rank the 10 people in your own team, the 2nd least likable person (the “Yen” in my illustration) is probably not bad. Out of 10 colleagues, it’s rare to find a single person clearly nasty, or unpleasant. You may find the 6th through the 10th are all reasonable teammates and you have a hard time ranking them (but how about at a time of bonus/layoff ? ) If you look at the last ranked colleague (, perhaps he has a strong personality/self-centeredness, or she’s always too busy to help you when you need her, or she forgets/ignores your requests, or he is a bit aloof and cocky and not that warm, or she’s too loud, or he’s too nice with the opposite sex, or she’s working too hard and makes you look lazy, or he has less humor than others …

(By the way, for focus and clarity, i will disregard the important fact that any measurement used in such ranking is extremely vague. )

Another ambiguity with likability is, the colleague you rank in the 2nd half may be my favorite types of personality. Clearly, we should both remove personal “恩怨” — someone who helped me a lot i may not like, but someone who made me a lot of trouble i may find quite amicable and fun to be around.

Another ambiguity is cultural boundary. The “nice” personality in one culture is usually considered polite and welcome (probably not charming/endearing) in another, but some 2nd-half kind of professional guy in India is quite possibly viewed with admiration and attractiveness by some Europeans, for example. Note there are infinite types of 2nd-half personalities!

Another ambiguity — I may agree with you this guy is nice and polite, but in a secret ballot I just won’t put him in the first half of likability. Nevertheless I may elect someone not so nice because she accepts ME better and makes ME comfortable. Likability is something personal to ME, just like clothing.

In a twist beyond explanation, I may never invite the nicest colleague for lunch (perhaps he’s too perfect and way above me), but I do share lunch with a lot of colleagues less than “perfectly likable” — I call it rapport.

Likability becomes serious at review/promotion/bonus/layoff time but i think only the managers’ view counts.
It becomes serious when — you try to transfer internally.
It becomes serious when — you are seen as a go-to person, a helpful teammate — reputation. People would mention your name when they need or received (non-technical) help.
It becomes serious when — you need a massive help from someone.
It becomes serious when — you organize an event.
It becomes serious when — you ask people for a character reference.
If a colleague confides in you, that’s a sign of your likability. Does it buy you anything?

Now consider technical caliber of engineers. The effectiveness, personal contribution and value-add of the 2nd decile vs 2nd last decile is more significant and visible. The difference can be objectively assessed. A mediocre engineer vs an insightful or fast-learning or quality-conscious or clear-thinking or systematic engineer — real difference. It shows in the work.

That partly explains why the very nice people I know often make slow progress professionally, but “mad scientists” often move up on their strength of technical caliber. My view is biased. Unlike me, many colleagues believe it’s more important who you know than what you know.

How about business analysts, accountants, professional traders, researchers and salespeople?

field hiding by subclass, learning notes

(c# has the trickiest hiding rules; c++ has  more than java.)

First, remember the onion model described in other posts. To rephrase one more time, each instance field occupies memory “cells” in a layer of the onion. If class B has an instance field vf, and subclass C has a same-name instance field vf, then a C “onion” has storage for both fields.

myRef.vf resolves to one of the 2 fields at compile time depending on declared type of myRef. P40 [[java precisely]]

On P25 [[java precisely]], B has a static field sf, and so does C. The C onion in memory has NO storage for either field, but it has a pointer to the class object, which has storage for the field.

Q: does the C class object have storage for both static fields?
%%A: probably no. C class object just needs a pointer to B class object.

Question: “myRef.sf” refers to …? P40 [[java precisely]]
A: At compile time, if the C onion is declared as type B, then the field is B.sf
A: At compile time, if the C onion is declared as type C, then the field is C.sf

I feel the static and non-static cases are similar — both compile-time binding. This is the simple rule for field binding. For method call binding, static and non-static behave differently. Remember that runtime binding is very strict and requires non-static and non-field, among other conditions. Failing any condition means compile-time binding.

Wall St IT project timeline is no joke

When I first joined GS, I was in awe of my managers’ /commitment/ to project deadlines. Before GS, I always felt deadlines are usually arbitrary and deserve little respect. I used to feel deep in my guts that developers ultimately hold an upper hand over managers *when* (not IF) deadlines become difficult. Basically, I feel the regular developers are hard working but aren’t saints, so we are all to varying degrees lazy and undisciplined, and tends to leave too much work to the end. On the other hand, managers can never be 100% on top of every Dirty detail, the quick and Dirty, and the Dirt swept under the carpet, so they are always surprised at end of the time frame.

After a few years in GS, I feel on Wall St, a lot is at stake when the timeline pressure heats up. Managers are visibly concerned about missing deadlines. Developers often work long hours. Major source of developer stress (and burnout).

If you feel elsewhere developers miss timelines 80% of the time, then it’s 20% in GS.

The hallmark of Wall St IT teams — pays more and demands more productivity (quicker delivery). Keyword is productivity.

Everyone can stick to a timeline if it’s long enough, but GS typically gives you one-tenth to half the timeline given elsewhere. It’s not uncommon to see a 10-month project squeezed into 1-month.

A lot of your career is at stake too. If you manage user/manager expectations well, then you could win big wins and move up.

————–

That was my view while in GS. After many years outside GS, I feel Barclays and BAML try to emulate GS with up to 50% effectiveness. Outside GS, the other Wall St IT managers can also give demanding deadlines, but the consequences of miss is not so severe. If you miss one without some scapegoat, then you can get any amount of pressure including but not limited to

  • no pressure at all
  • fake pressure
  • asked to work OT
  • sidelined to less critical roles
  • given some coach to work with you on the same project
  • official warning
  • performance improvement process. (I hate this, though it is designed to protect the employee.)

real time risk – option mkt-maker

Core component is the valuation engine. Each vol surface gets rebuilt every 10 minutes. Positions are re-valued every minute on average.

I believe option position valuations change more often because spot moves more often than implied volatility.

(Similarly, in Sprite of Baml Stirt, the FX spot moves more often than fwd points. It also moves more often than interest rate market data, including IR futures prices.)

I was told it’s hard for any eq vol market maker to survive without a real time pricer. When client calls for a quote, it’s problematic to rely on a 2-hour stale vol surface.

Re-pricing in 2 areas —
1) position mark-to-market (most important)
2) outgoing quote/RFQ pricing

I feel these 2 areas are related. If you are a market maker in an index (stoxx for eg), then your own position will influence your outgoing quotes. I guess the Reason is — position means market-risk, and position means hedging. Everyone needs to hedge, esp. a market maker.

For volatility instruments, delta-hedging must be adjusted constantly.

all c# value types are effectively structs@@

This is probably imprecise language, but good enough for a beginner.

There are only 2 sub-types of value types — structs and enums. Note simple types (like int) are struct type aliases.

Now, enums are backed by integers (short, long etc), so it has to share a lot of basic features with integers. I feel ALL enum features are very similar to int features. I feel if enum shows any “deviation” from struct, it’s likely superficial. Therefore, essentially all value types are structs

I guess all value objects are struct instances that won’t reveal their addresses (but structs can be passed by  “ref”). The way to pass these objects around is copying/boxing.

C-style programing]java: can be legit

A friend told me it doesn’t make sense to write C-style programs in java, but I believe some of the fastest high volume trading engines
– use very few objects (perhaps some Object.java instances or array objects), but lots of primitives
– minimize garbage collector to almost never
– use no java collections, but arrays (albeit objects)
– use many dedicated threads but not always thread pools
– use almost no locks among the threads

So why java instead of C? I feel in such domains, C is indeed the language of choice, but I guess java offers

* platform independence — (but do we need it, when we use JIT compiling?). Truly compile-once, run anywhere.
* GC — (occasional use), since C memory management can be tricky.
* threading — the superior threading constructs in java
* off-the-shelf modules — many are available to java only, though less common in low-latency.
* pointers — java eliminates all (problematic) access to raw pointers
* platform-specific coding — java prevents programmers from resorting to platform-specific features. For any powerful/resourceful developer, if there exists a shortcut, it will be exploited someday — human nature
* without pointers and complex type declarations, programming is more readable, easier and less error prone
* the option to occasionally use some of the many java features, like the ones below. Note in a superfast engine, not all parts are equally latency-sensitive, so those “easy” parts can (and probably should) be written in a higher-level language. C won’t give you that option.
** RMI
** collections
** NIO

avoid convenience wrappers;xx low-level APIs #RV,socket

See also high^low-level expertise

Many examples of the same observation — convenient high-level wrappers insulate/shield a developer from the (all-important) underlying API. Consequently, in a hardcore interview he can’t demonstrate his experience and mastery in that particular “real” technology — he only knows how to use the convenience wrappers.

  • example — servlet — in one of my GS trading desks, all web apps were written atop an in-house framework “Panels”, which was invented after servlet but before JSP, so it offers many features similar or in addition to JSP. After years of programming in Panels, you would remember nothing at all about the underlying servlet API, because that is successfully encapsulated. Only the Panels API authors work against the underlying servlet API. Excellent example of encapsulation, standardization, layering and division of labor.

Notice the paradox — guys who wrote those wrappers are considered (by colleagues and job interviewers) as stronger knowledge experts than the users of the wrappers. If you think about it, you (as everyone else) would agree that knowledge of the underlying API is more valuable than GTD skill using the wrappers.

  • example — Gemfire — one of my trading engines was heavy on Gemfire. The first things we created was a bunch of wrapper classes to standardize how to access gemfire. Everyone must use the wrappers.

If you are NOT the developer of those wrappers, then you won’t know much about gemfire API even after 12 months of development. Interviewers will see that you have close to zero Gemfire experience — unfair!

  • example — thread-pool — One of my ML colleagues told me (proudly) that he once wrote his own thread pool. I asked why not use the JDK and he said his home-made solution is simple and flexible — infinitely customizable. That experience definitely gave him confidence to talk about thread pools in job interviews.

If you ask me when a custom thread pool is needed, I can only hypothesize that the JDK pools are general-purpose, not optimized for the fastest systems. Fastest systems are often FPGA or hardware based but too costly. As a middle ground, I’d say you can create your customized threading tools including a thread pool.

  • example — Boost::thread — In C++ threading interviews, I often say I used a popular threading toolkit — Boost::thread. However, interviewers repeatedly asked about how boost::thread authors implement certain features. If you use boost in your project, you won’t bother to find out how those tools are internally implemented. That’s the whole point of using boost. Boost is a cross-platform toolkit (more than a wrapper) and relies on things like win32 threads or pthreads. Many interviewers asked about pthreads. If you use a high-level threading toolkit, you avoid the low-level pthreads details..
  • example — Hibernate — If you rely on hibernate and HQL to generate native SQL queries and manage transaction and identities, then you lose touch with the intricate multitable join, index-selection, transaction and identity column issues. Won’t do you good for interviews.
  • G5 example — RV — one of my tibco RV projects used a simple and nice wrapper package, so all our java classes don’t need to deal with tibrv objects (like transport, subject, event, rvd….). However, interviewers will definitely ask about those tibrv objects and how they interact — non-trivial.
  • example — JMS — in GS, the JMS infrastructure (Tibco EMS) was wrapped under an industrial-strength GS firmwide “service”. No applications are allowed to access EMS directly. As a result, I feel many (experienced) developers don’t know JMS api details. For example, they don’t use onMessage() much and never browse a queue without removing messages from it. We were so busy just to get things done, put out the fires and go home before midnight, so who has time to read what he never needs to know (unless he plans to quit this company)?
  • example — spring jms — I struggled against this tool in a project. I feel it is a thin wrapper over JMS but still obscures a lot of important JMS features. I remember stepping through the code to track down the call that started the underlying JMS server and client sessions, and i found it hidden deeply and not well-documented. If you use spring jms you would still need some JMS api but not in its entirety.
  • example — xml parsing — After many projects, I still prefer SAX and DOM parsers rather than the high-level wrappers. Interviewers love DOM/SAX.
  • G3 example — wait/notify — i used wait/notify many times, but very few developers do the same because most would choose a high-level concurrency toolkit. (As far as I know, all threading toolkits must call wait/notify internally.) As a result, they don’t know wait/notify very well. Strangely, wait/notify are not complicated as the other APIs mentioned earlier but are tricky to 90% of developers.
  • G7 example — FIX — one of my FIX projects uses a rather elaborate wrapper package to hide the complexity of FIX protocol. Interviewers will ask about FIX, but I don’t know any from that project because all FIX details were hidden from me.
  • example — raw arrays — many c++ authors say you should use vector and String (powerful, standard wrappers) to replace the low-level raw arrays. However, in the real world, many legacy apps still use raw arrays. If you avoid raw arrays, you have no skill dealing with them in interviews or projects. Raw arrays integrate tightly with pointers and can be tricky. Half of C complexities stem from pointers.
  • G5 example — DB tuning — when Oracle 10 came out, my Singapore (Strategem) DBA friend told me the toughest DBA job is now easier because oracle 10 features excellent self-tuning. I still doubt that because query tuning is always a developer’s job and the server software has limited insight. Even instance tuning can be very complicated so DBA hands-on knowledge is probably irreplaceable. If you naively believe in Oracle marketing and rely on self-tuning, you will soon find yourself falling behind other DBAs who look hard into tuning issues.
  • G3 example — sockets — i feel almost every place where sockets are used, they are used via wrappers. Boost, ACE, java, c#, python, perl all offer wrappers over sockets. RTS is the exception. However, socket API is critical and a favorite interview topic. We had better go beyond those wrappers and experiment with underlying sockets.

I recently had to describe my network programming experience. I talked about using perl telnet module, which is a wrapper over wrapper over sockets. I feel interviewer was looking for tcp/udp programming experience. But for that system requirement, perl telnet module was the right choice. That right choice is poor choice for knowledge-gain.

The Quoine interviewer Mark was the only hiring manager to appreciate wrappers more than underlying socket knowledge. He is the exception that proves the rule.

y UDP uses recvfrom() !! read()

http://www.cs.cmu.edu/afs/cs/academic/class/15441-f01/www/lectures/lecture03.ppt shows that UDP server must use recvfrom() and not read() because only recvfrom() returns (by reference param) the client’s address.

In contrast, TCP establishes a connection/session/virtual-circuit so the thread calling read() already knows the other side’s address, so recvfrom(oppositeAddr,…..) and sendto(oppositeAddr,…) aren’t required — The logic is all in the names!

multiple intermediate data storage]real time trading servers

For easy prod support, get your first stage of processing to save
intermediate output to cache, DB or MOM, and 2nd stage to pick up from
there. You can have many stages (ie pipes) and pipe connectors.
This might help your job security if other developers can’t easily
figure out all of your techniques saving, accessing, investigating (in
prod), filtering, monitoring the intermediate data. Remember gemfire
doesn’t have a working data browser?
This helps testing. Remember Mithun’s DBank cash management project.
This helps prod monitoring.
This helps everyone understand the business as they can see the
intermediate data in blood and flesh. You can get interesting
statistics.
Recall Reo has limited logging so we don’t know why some events don’t
happen upon a user action or market update.

IV at MS-IRD (java) + Imagine c++ #done

Q: implement getAngleBetweenHourAndMinuteHands(byte hr, byte min)

Q: implement getFirstNPrimeNumbers(int howMany)

Q1: Given a list of 99 random integers (uniqueness not guaranteed), write a function to find any pair that adds up to 1000. The function takes 2 arguments – a target int and a list of int.
%%A: I blogged about this. Build a hashset to hold all 99 numbers in 1st pass. In 2nd pass, for each number, compute the “complement” and look for it in the hashset.

Q1a: what if the list is pre-sorted?
%%A: no change. Same performance. The pre-sort doesn’t help.

Q2: how do you price an IRS? I will assume an existing vanilla IRS
%%A: the existing contract’s fixed rate is compared to the prevailing swap rate. The difference represents a periodic cash flow that’s fixed! Just discount that to PV.

Q2a: what if you have a discount curve?

——c++
Q: Given a singly linked list, write a recursive function to reverse the list

Class Slist{
Node * head;
Node * tail;
}
A: https://bintanvictor.wordpress.com/2017/06/26/reverse-a-linked-list-without-recursion/

Q (ez coding question): for an array of characters (null-terminated string), write a (recursive or iterative) function to reverse print it.

Q: given a simple class with an int field, implement pre-increment operator and post-increment operator
A: I have never done this in work, but the syntax is easy to find on-line.

2+1 live sources for troubleshooting "my trade" in PROD

DB, log, and cache — then link what you discover to source code. These are only 3 places we can find crucial telltale signs. It is therefore extremely important to exploit these sources to the max, esp. DB.

(Background — if a trading system is misbehaving intermittently and a particular trade is messed up, how do you find out what’s going on?)

If I can add one other trick, I hope JMX remote operation can turn on object graph dump of any object in the prod JVM.

I feel MOM is not that easy to use as a persistent data store. Data gets purged quickly. In contrast, DB often has an audit/history table.

Credit Suisse IV questions (automated bond trading)

Q: What’s your ideal dev environment, in terms of release, code sharing, code review, test-driven,

Q: What design patterns did you use to improve flexibility and testability?

Q: what’s your testing strategy (i think this is a very important question and challenge.)

Q: what are the pros and cons of implementing Runnable vs extending Thread?
%A: Runnable is either better or as good as Thread. I don’t see any advantage in extending Thread.

Q: volatile bool vs AtomicBool?
%A: AtomicBool is bigger but offers atomic operations unavailable in volatile.

Q: choose iterative vs recursive
%A: speed very similar. Same O(n)
%A: stack over flow
%A: concurrency – parallelism

java^c# generic: struct, boxing, erasure

At the heart of all the jargon and debates lies one core issue — field layout (FL). Essentially [1], field layout is what truly defines a type, be it class or struct.

For example, the java LinkedList’s Node class has field layout to hold a payload and a “next-node” — all pointers. Since untyped Node came to the scene first, Java generics resorted to type erasure (TE) for backward compatibility. As a result, Node of any payload is the same Node type at runtime. Consequently, LinkedList of any payload boils down to the same type. one-size-fit-all.

C# needs no TE. However, in a twist of clever design, Node of any reference payload has a field layout just like java. So LinkedList of any reference type boils down to the same type. Reduces code bloat compared to c++ template.

vvv Java Node of int has to be boxed/unboxed because Node payload (like any reference type) must be a subtype of Object.java. Given a node, the compiler is /hardwired/ to treat the “payload” field therein as a pointer to the Heap (where boxed Integer lives).

^^^ C# Node of int is a distinct type with a distinct field layout.[2] The int value is a struct Instance embedded in Node object’s real estate. No boxing/unboxing.
** Struct payloads are all treated this way.

This is yet another low-level technical issue affecting only high-performance, high volume market data where megabytes of packet copying is expensive. Non-trading developers tend to consider this a niche that’s “too” specialized.

See P156 [[c#precisely]]

[2] Incidentally, a Node of float has the same size as a Node of int, but compiler interprets the payload field as int vs float. Therefore different field layouts.
[1] This is practical thinking. I am no language lawyer. Specifically,
– A type can have only 1 field layout.
– If 2 types have identical field layout, then we can make one an alias of the other like a typedef. It’s wasteful to create too many duplicate types.

Yang KPI:learn’huge codebase-goto person, connect user-expectations+codebase

This is an annotated conversation with a team mate.

> “Y (our team lead) has good system knowledge” like how things *should* work.

If Y asks a developer a specific question about how something actually works in the current implementation, he often points out “discrepancies” thanks to his system knowledge. In that case, either the developer is wrong or there’s a bug — Y’s system knowledge is quite sound. I (Bin) feel Y is sharp, quick and can filter and absorb the gist into his big picture.

I feel current implementation is probably fairly close to the “expecation” because it has been used (therefore verified) for years.

I feel Y approaches a complex system from top down, focusing on the how-things-should-work, or so-called “system knowledge” which is non-trivial. That’s one reason Y is more interested in the SQL queries. SQL is useful because DB serves as “checkpoint”.

Part of his learning is weekly user meeting on (high-level or detailed) requirements, bugs … — boils down to user EXPECTATIONS. Team members don’t get that insight. I feel that is more than biz knowledge — such insight connects user expectation with implementation. Much of the insight is summarized in jira and you can tell how much insight there is.

Another part of his learning is knowledge transfer session from his predecessor.

I feel Y asks better questions than I did. “system knowledge” questions. Roland told me he has difficulty getting answers but Y is *persistent* and asked lots of sharp questions until he got all his high/low level doubts cleared. My focus tended to be at a lower level.

Y became a go-to person quickly.

< By system knowledge i think you also mean the PATTERNS and SUMMARIES of logic in codebase. After you study codebase bottom up you inevitably see such patterns.
> “yes”

< how did you pick up in the beginning? There's so much code to get through.
> “only 2 ways to learn — support or development. I worked on development of pgrid and GCA. i followed emails between users and support guys and went into code to verify.”
I (Bin) think there’s also a lot of help from the Infosystem colleagues.

I didn’t follow all the email chains and take those as learning opportunities. Email overload.

Y gave me a plausible learning opportunity — document the haircut rules. But there’s too much logic. I studied it a few times but didn’t connect to the log messages or user issues. I didn’t learn enough to make me a go-to person. Now i know support is the way to go — i get to know what’s important among the hundreds of complex rules and also how-things-should-work.

c++ nested class experiment

#include
class Outer{
   int m_o;
   public:
   class Inner   {
      public:
        int m_i;
        Inner(){m_i=999;}
        Inner(Outer & m_outer, int x){
           m_outer.m_o = x; //against c++98
        }
   };
   Outer(int y)   {
       cout <m_inner.m_i<<" <—- this field is no-arg constructed.n";
       m_inner.m_i = y;
   }
   void Display()   {
       using namespace std;
       cout << m_o << endl
            << m_inner.m_i << endl;
   }
   Inner m_inner;
};
int main(){
   Outer    objOut(10); //inner == 10
   Outer::Inner objIn(objOut, 5); // objIn discarded, but objOut modified to 5
   objOut.Display();
   cin.get();
}

Set (data structure) support across languages#+py

java collections? hash set + sorted set (tree set, skip list…)
STL? tree set + sorted multiset
STL extensions? hash set

—- Languages Below offer reduced support for Set. Emulate with dict —-
c# ? HashSet is not a first-class citizen like Map and List
perl? no set
php? no set. Array and Associative array are the only builtin data structures
python? Set is not a first-class bultin like dict and tuple. There are fewer operators on Sets. However set() is a built-in free standing function, just like tuple() and dict()

—————
The reason STL supports set as a distinct container from map? Short answer is Efficiency. C++ is a low level, high-efficiency, application-building tool. At that level, set offers real advantage over map.

java RMI in trading systems

Now I feel rmi is rather easy, battle-tested, proven, mature,
well-researched, … compared to many alternative solutions. Here's
RMI usage in a trading system circa 2011 —

Nobody calls Neo server via RMI.  The only way you can talk to Neo is
via JMS/Protobuf.  So even if you have 100 instances of Neo servers,
JMS distributes the messages across them.

Neo does make _outbound_ RMI calls to PricingControl, Arb/prop/model
trading engine, and various other systems.

front office developers for fx options trading desk

Question: Guess how many developers are needed in the fx option trading desk at one of world's biggest investment banks? Note this desk handles all fx option trading for the entire bank.
Answer: about 20 world wide including consultants.

The other trading desk in the FX space is the FX cash desk. Not sure how many developers.

If there are 20 similar trading desks in the bank, then total number of _desk_specific_ developer head count is going to be around 500, at most 1000. I think this is a very small percentage (5%?) of total IT head count.

In 2007 GS had about 6000 IT headcount. This bank is probably double. I'd say at least 10,000 IT head count. I would estimate less than 5% of them are directly funded by trading desks.

Now I know why front office trading is a rare opportunity.

Risk developers are classified into 2 types
– some are dedicated to a desk
– some are firm-wide

financial app testing – biz scenario^implementation error

Within the domain of _business_logic_ testing, I feel there are really 2 very different targets/focuses – Serious Scenario (SS) vs Implementation Imperfections (II). This dichotomy cuts through every discussion in financial application testing.

* II is the focus of junit, mock objects, fitnesse, test coverage, Test-driven-development, Assert and most of the techie discussions
* SS is the real meaning of quality assurance (QA) and user acceptance (UA) test on Wall St. In contrast II doesn’t provide assurance — QA or UA.

SS is about real, serious scenario, not the meaningless fake scenarios of II.

When we find out bugs have been released to production, in reality we invariably trace root cause to incomplete SS, and seldom to II. Managers, users, BA, … are concerned with SS only, never II. SS requires business knowledge. I discussed with a developer (Mithun) in a large Deutsche bank application. He pointed out their technique of verifying intermediate data in a workflow SS test. He said II is simply too much effort and little value.

NPE (null pointer exception) is a good example of II tester’s mindset. Good scenario testing can provide good assurance that NPE doesn’t happen in any acceptable scenarios. If a scenario is not within scope and not in the requirement, then in that scenario system behavior is “undefined”. Test coverage is important in II, but if some execution path (NPE-ridden) is never tested in our SS, then that path isn’t important and can be safely left untested in many financial apps. I’m speaking from practical experience.

Regression testing should be done in II testing and (more importantly) SS testing.

SS is almost always integrated testing, so mock objects won’t help.

MS FX trading IV

Q: what if someone successfully de-serializes a singleton? What’s the value of that static field?

%A: null, but if it de-serializes into the same class loader as the original singleton, then the field will be non-null

%A: whether it’s null or not, this new instance is usable. Whoever has a handle on this new instance can use it without calling getInstance().

Q: How do you prevent cloning of a singleton?
A: override Object.clone() to throw.

Q: a base class has a static method and field. Can subclass declare those same members?
%A: yes hiding.

Q: what’s hiding vs overriding?
%A: compile-time vs runtime binding

Q: If I have a base type variable pointing to a derived object, and I call a static method defined in both, which version runs?
%A: base version, due to compile-time binding.

Q: what are the generations in a GC? Does an object jump from Eden to another generation or does it move incrementally?

Q: what objects are in each generation esp. the permanent generation?

high frequency ^ quant developer

I feel these are the 2 most advanced developer roles.

Low-latency (high frequency) doesn’t always pay higher than average. Not sure about quant dev.

Quant dev involves more math and more domain knowledge than other Wall St IT jobs, but I feel no CFA needed. However, I feel low latency offers very low exposure to domain knowledge.

Low latency is technically tougher than quant by a wide margin. Quant uses limited threading or efficient data structures.

putty — save host-specific config

A host-specific config is saved in a so-called saved-session. To access the list of saved-sessions, go to, paradoxically, _New_Session.._

To see or modify a particular saved-session, you must first Load it. You may need to hit Load several times. Once you are in the “editor” mode on a saved-session,

  • * Navigate to a particular tree branch, like Window
  • * change a particular value there
  • * There’s no Save button. You must navigate back to the original Session page, and hit Save (repeatedly:). This Save button
    ** saves all the settings in the editor
    ** saves them under the session name.
  • * If a parameter has an unsavable value, system will _silently_ drop the change and rollback.

http://tartarus.org/~simon/putty-snapshots/htmldoc/Chapter4.html#config-saving seems to be a good FAQ for this.

How about the config file for each connection? Apparently putty uses registry…

–DefaultSettings as a template: save it if you repeatedly use this template.

launch -> select DefaultSettings -> LOAD -> change any setting -> come back and (click to) highlight DefaultSettings -> SAVE

references are seldom reseated – java^c++

A reference or pointer takes just 32 bits. Cheap to copy.

For java refs and c++ pointers, The standard, default operation is a bitwise copy. That's what you get at function boundaries. Whenever you see a reference variable “reseated”, raise a red flag — something unusual is going on.

A java best practice is to declare all local reference variables as final (will raise a real red flag:), unless justified otherwise.

10 coding tips of high-frequency exchange connectivity

[c] use enum/char rather than vptr #Miami
[c] hand-crafted hash table, which is often more optimized for our specific requirement #Barcap
[c] Fixed-width, padded char-array without null terminator #Miami
[c] favor Set to Map
[c] hand-crafted reference counted data structure #Miami
[c] monolithic network-friendly wire format STRUCT for high volume data transfer. Nothing fancy.
[c] java finalizers slowing down garbage collection
[c] favor arrays to object graphs #UBS
[c] avoid boxing and java colllections. Favor primitives. #CS
specialized operator-new. See [[eff c++]]
unions

—-
Note — all of these are considered optimizations on memory. That’s where C++ outsmarts java.

[c=according to insiders]

10 (random) arch features of HFT

When talking to low-latency shops, i realize the focus shifts from pricing, trade booking, position mgmt … to market data, message formatting and sockets – rather low-level stuff. A high-frequency trading engine has many special features at architectural and impl levels, but here i will focus on some important architectural features that make a difference. By the way, my current system happens to show many of these features.

1) message-driven, often using RV or derivatives. Most trading signals come in as market data, tick data, benchmark shifts, position adjustments (by other traders of own own bank). Among these, I feel market data poses the biggest challenge from the latency perspective.
2) huge (reluctantly distributed – see other post) cache to minimize database access
) judicious use of async and sync IPC, if one-big-machine is undesirable.
3) optimized socket layer, often in C rather than c++. No object-orientation needed here:)
) server collocation
) large number of small orders to enable fine-grained timing/cancel and avoid disrupting market
) market data gateway instantiates a large number of small objects
) smart order router, since an order can often execute on multiple liquidity venues

Beyond the key features, I guess there’s often a requirement to immediately change a parameter in the runtime rather than updating a database and waiting for the change to be noticed by the runtime. I feel messaging is one option, and RMI/JMX is another.

sybae ltrim() and rtrim() can change a empty string to a null string

I do get some null cusips from this query —

select cusip=ltrim(rtrim(p.cusip)), price = case
when traderMarkInd = 'Y' then traderMarkPrice
when autoMarkEnabled = 'Y' then autoMarkPrice
end, p.account, i.position, maturityDt=99991231
from TblMuniMarkPrice p left join TblMuniInventory2 i on p.cusip = i.cusip
and p.account = i.account
where 1=1
and p.cusip is not null
and ltrim(rtrim(p.cusip)) is null

static mailer using /bin/mailx

public class StaticMailer {
private static final String tmp1 = System.getProperty("mailx");
private static final String MAILX = (tmp1 == null ? "/bin/mailx" : tmp1);

static public char send(String to, String subject, String body) {
String[] command = new String[] { MAILX, "-s " + subject, to };
return privateSend(command, body);
}

static private char privateSend(String[] command, String body) {
System.out.println(Arrays.asList(command));
try {
Process pr = Runtime.getRuntime().exec(command);
OutputStream stdin = pr.getOutputStream();
stdin.write((body + "n").getBytes());
stdin.flush();
stdin.close();
BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream()));
String line;
while ((line = input.readLine()) != null) {
System.out.println(line);
}
pr.waitFor();
} catch (Exception e) {
e.printStackTrace();
return 'e';
}
return '0';
}
}

algo trading – 2 types

Algorithmic Trading vs. Trade Origination

Here’s the key question you need to ask:

  • Is a human making the trading decisions and simply having a computer help with the execution, or is the machine handling the execution and making the trading decisions?

The first category – where the computer only assists with the execution – is called algorithmic trading.

The second category – where the computer actually makes decisions – can be called trade origination, although that is not a canonical name

low-level differences between HASA^ISA, in terms of function pointers

Some interviewer (MS or CS) asked me about the differences.

1) Subclass instance has the _same_address_ as base object, that’s why you can cast the ptr (or reference) up and down the inheritance hierarchy. See post on AOB ie address of basement.

2) Also, all inherited methods are available in the “collection” of function pointers of the derived object. In other words, derived object advertises all those inherited “features” or “services”, if you think of a family of interchangeable components. Derived object can _stand_in_ for the base TYPE.

In OO languages, a pure interface is basically a collection of function pointers. Has-a doesn’t expose/advertise the internal object’s function pointers, so the wrapper can’t “stand in” for that type.

gemfire seminar take-away

In-memory index is a SkipListMap – a java5 implementation of SortedMap interface.

Market data? Can carry Nyse/Nasdaq feed but not bigger feeds like OPRA. All Tick data from nyse? fine, but probably not historically.

— Buzzwords of gemfire —
schema-less, just like secDB
Object Query Language
continuous-query
higher reliability than RDBMS? What a claim!
durable subscriptions
conflation
many-to-many eg trades to settlement-netting
Session state mgmt – for http sessions across a cluster

comparing Libor rate and treasury yield@@

(See also TED ie the spread for a 3-month maturity.)

You can compare 12-month USD Libor rate (i.e. Eurodollar deposit interest rate)  with a zero treasury’s yield of the same maturity.

Note Libor covers short maturities, overnight to 12 months, whereas T covers 3 months – 30 years. Even when there’s a match on maturity, usually you can’t directly compare Libor rates with a treasury yields, because treasury generates coupon payments.

What a trader/quant likes in a developer

clear, quick communications. software Developers tend to be less sociable.

Most developers don't sit side by side with traders, and aren't trained to present a problem or structure an answer.

More than taking a requirement and implementing it. Recognize the intent, propose better solutions, in terms of speed, scalability, and the most “logical construct” …

Quants – responsible for princing — where there's more math
developers – implementation

+ve carry in treasury market

In many markets, cost of carry is a cost to a holder of a security, because …. frequently it’s bought under borrowed money. A typical short term interest is 200%/yr, around 0.5bps/day.
In the Treasury market (perhaps in agency and muni too), each day you earn accrued interest, which is typically 500bps/yr or about 1.5bps/day. Positive carry.

The earlier a buyer settles the trade, the earlier she starts earning accrued interest[1]. As a result, the earlier a buyer settles, the higher the price to pay.

This reminds me “start investing in 401k early, so you start earning returns”

http://bondtutor.com/btchp5/topic3/topic3.htm has numeric examples

http://glossary.reuters.com/index.php?title=Cost_of_Carry points out that in the comm market cost of carry means storage cost.

Credit Suisse tick data C++IV

Mostly QQ!

Q: auto_ptr vs scoped_ptr? See my other blog post

Q: when did you use a binary functor?
%%A: provide that class to a template to concretize it

Q: if I have an empty class, what’s the size of an instance?
AA: 1 byte

Q: offsetof() macro in C?

Q: ref vs ptr

Q: when is copy ctor implicitly invoked?

Q: given a vector of ints, how do I add 1 to each of them, in-place?
%A: for_each(), transform()

Q: what are functors?

Q: did you use priority queue?

Q: in network programming, how Did you use select?
%%A: My system used it but I didn’t study it in-depth

Q1 (actually easy): Anything wrong with a vector of references
%%A: won’t compile

Q1b: can I put references into containers?
%A: never seen it.
%A: I know it’s sometimes ok to put pointers in containers, or const pointers. Since references are very much like pointers, it might be ok
A: compiler breaks saying “pointer to reference is illegal”. I think it’s because iterator of vector is a regular pointer to the underlying_type. If underlying_type is an int ref, then the iterator would be a pointer to an int ref.
A: In STL source code there’s

typedef T* pointer;

If T is int&, this code won’t compile.

financial domain XP ^ domain Knowledge

Domain _experience_ is what hiring managers look for. If there’s a high-paying job in some rarified field (say analytics, algo trading, low-latency..) then domain experience on the resume is the crucial factor. Many candidates can have strong tech background, and strong domain knowledge, but only those with “relevant experience” will get short-listed, or those with good recruiter connections.

The domain experience shows in terms of 1) jargon 2) architecture, but actually is very local and not portable at all.

Generic domain knowledge is very “thin”. You can pick it up by reading or over a few months on the job. Mostly jargon terms, and very little math. Hiring managers ask questions to verify such knowledge but it’s not hard to pass.

eSpeed/BrokerTec inter-dealer Treasury market workup

(In the offer case, “Lift-up” is similar, but “hit-down” is easier in English.)

The bids on the eSpeed order book is like a stack of cards. Top of the stack is one or multiple bids at the same bid price. For example 2 bids at 99.8. Needless to say, the trader who came earlier is first_among_equals, but remember both bidders are on the same price level, like 2 physical cards spread out horizontally, on top of the card stack [1].

When you  h i t  them, you potentially “lower” the market – “hit down”. 2 scenarios/outcomes, inspired by P117 [[Complete guide to capital markets]] —

Scenario/Outcome 1 – you lower the market – During the workup, you and “secondary” hitters make up a total  o r d e r  size exceeding the total bid quantity at 99.8, consisting of 2 original bids + secondary bids. Afterwards, all the “cards” at 99.8 are removed, exposing the cards at 99.7, which is the new top-of-stack.

Scenario/Outcome 2 – best bid remains at original level – similar to Scenario 1 but not “exceeding”. After workup, top bidder is seldom the original top  bidders. Original 1st bidder has the first choice to “join” the workup [2]. If she doesn’t  a b s o r b  all the sell orders, then she’s removed at the 99.8 bid level.

Looking at both scenarios, you see that “hit-down” can never move the market up.

[1] In semi-technical terms, it’s like a linked list anchored at top slot of the card “stack”.

[2] This is often an automated trading system

dark side of left shift operator – java

1) First, remember left-shift only operates on integer types.
2) Second, remember most if not all integer types use 2’s complement, where leading bit represents sign.

Now the dark side – Multiplication by 2 can move the original number between positive and negative universes.

In this context, Left shift is simpler to understand than multiplication. Left shift by 1 is equivalent to (xxx *2). Confirmed on Sun.

Q: how about multiplication by 4 or 3?
%%A: I feel multiplication by even numbers tend to give the same flipping problem.

Sun says –
If an integer multiplication overflows, then the result is the low-order bits (possibly with a leading ‘1’) of the mathematical product as represented in some sufficiently large two’s-complement format. As a result, if overflow occurs, then the sign of the result may not be the same as the sign of the mathematical product of the two operand values.

Amazon cod`IV(full source) – bitwise operator

–Method 1, submitted and rejected:
/**
 * Q: program to test whether an input long integer’s binary representation is
 * the same left-to-right or right-to-left.
 */
public class ShiftTest {
    public static void main(String[] args) {
        long input = 0xff00ff0180ff00ffL;
        char[] bitsForDisplay = populateArray(input);
        dumpArray(bitsForDisplay);
        for (short start = 0, end = 63; start < end; start++, end–) {
            if (bitsForDisplay[start] != bitsForDisplay[end]) {
                System.out.format(“nBad – Mismatch between Positions #%d and #%d. (1-based)”,
                    ++start, ++end);
                return;
            }
        }
        System.out
                .println(“nGood – Binary representation reads identical from left or right”);
    }

    private static void dumpArray(char[] array64) {
        int subscript = 0;
        for (char element : array64) {
            System.out.print(element);
            if (++subscript % 8 == 0)
                System.out.print(‘ ‘);
        }
    }

    private static char[] populateArray(long input) {
        char[] bitsForDisplay = new char[64];
        for (int subscript = 0; subscript < 64; subscript++, input <<= 1) {
            // if the shifted num is negative, then leading bit must be 1
            bitsForDisplay[subscript] = ((input < 0) ? '1' : '0');
        }
        return bitsForDisplay;
    }
}

–A similar method without converting to array
Maintain two masks. with
– 1st bit set only
– last bit set only

Apply each mask on the input — the AND result should be both 0 or both positive.

Now shift both masks and repeat.

countDownLatch in server shutdown – real example

In Neo, we use a count down latch to block System.exit() until a number of clean-up tasks are completed on several event-handlers. Event handler could run on any thread, but not the shutdown thread. Each clean-up would trigger upon an event, do the clean-up then call countDown() exactly once.

If a clean-up should run twice, then it should call countDown() exactly twice.

ledger vs trade booking (aka product processor)

In any trading desk, there’s a product processor specifically for a subset of asset classes. For example, OTC eq options probably uses a different product processor than listed eq options. Arguably the Most essential function of Product processor is trade booking. I-ticket is an example.

Ledger is the authoritative books-and-records for a subset of trading accounts. It’s not current — There’s end-of-day recon between trade booking and ledger. Before the recon, ledger could be out of date, so the trade booking has the most updated positions.

There’s a ledger for treasuries, for eq (TMS), for muni (TMS) etc. They all roll up to GL.

In GS private wealth, the heart of the system is the ledger not for trading accounts but for client accounts – client sub-ledger. To ensure it gets updated correctly, there’s nightly recon with product processors (ie trade booking). There’s a dedicated operations team to resolve all breaks within about 7 hours.

benign race condition (homegrown ConcurrentMutableInteger

Does the getInt() below need “synchronized”? I am not expert but I would go out on a limb and say no. It either gets the value before or the value after the increment, never a value inconsistent with reality. As Bill Lewis said in his pthread book about semaphore getCount(), whenever some thread wants to /use/ the getInt() return value, it should bear in mind the value could be stale, since other threads could change it any time.

The race between increment() and getInt() is probably benign. However, if increment() isn’t synchronized, then you could get lost-updates — inconsistent with reality. Note volatile field won’t help.

// simpler than AtomicInteger, but not lockfree.
class ConcurrentMutableInteger implements Serializable{
 private int i = 0;
 public synchronized void increment() {this.i++;}
 public int getInt() {return this.i; }
}

XR’s memory efficiency in large in-memory search (DB diff via xml

A real world Problem: 2 DB tables have very similar content. The owner decided to periodically export each table into a 200MB xml file. Each row becomes an element in xml. Each field an attribute. Business Users want to compare the rows on-demand via a servlet. Need to show per-field differences.

Performance? A few seconds to compare the xml files and show result.

Due to various overhead while reading and write files, -Xmx1000m is required.

Insight — duplicate object is the chief performance issue. Key technique is an object registry (like SSN in US), where each recurring object (strings, Floats too!) is assigned an 32-bit Oid. Dates are converted to dates then long integer (memory efficient).  To compare 2 corresponding fields, just compare their Oid. String comparison is much slower than int comparison.

Use an AtomicInteger as Oid auto-increment generator — thread-safe.

Solution:
– load each XML into memory, but never to instantiate 2 identical objects. Use the Oid.
** construct an {Object,Oid} hashmap?
– now build object graphs, using the Oid’s. Each row is represented by an int array
– 1m rows become 1m int arrays, where 0th array element is always Column 0, 1st element always Column 1 …
– Since Column 3 is primary key, we build a hashmap (primary key Oid, int array) using 1st xml file
– now we go through the 1m rows from 2nd xml file. For each row, we construct its int array, use the primary key’s Oid to locate the corresponding int array from the hashmap, and compare the 2 int arrays.

Note xml contains many partial rows — some of the 50 fields might be null. However, every field has a name. therefore our int-array is always 50 but with nulls represented by -1.

Table has 50 columns, 400,000 rows. Each field is char(20) to char(50).

message requester to wait5min for response #wait()

Imagine a typical request/reply messaging system. I think in JMS it’s usually based on temp queues, reply-to and correlation-Id — See other blog post. In contrast, RV has no broker. It’s decentralized into multiple peer rv daemons. No difference in this case —

Suppose a message broker holds a lot of queues. One of the queues is for a request message, from requester system to a pricing system. Another queue is for pricing system to return the new price to the requester.

Now, pricing system is slow. Requester should wait for no more than 5 minutes. If the new price comes back through the reply-queue 301 sec later, requester will ignore this stale price since it’s too risky to place an order on a stale price in a fast market. How do you implement this?

My design — Requester main thread can use condVar wait(5*60*000). Another thread in requester JVM can block forever in onMsg(), and notify main thread when something received.

Timer is the 2nd usage of condVar as Stoustrup described.

(I actually implemented this in a few trading engines.)

firm ledger vs client ledger

A broker-dealer’s ledgers roll up to GL. I did GL posting for the GS private wealth business. I post profits (and losses) earned in PWM’s account and IBD’s account and FICC’s account etc, but these accounts belong to the broker/dealer , so they logically roll up into a GL hierarchy.

 

Client sub-ledger doesn’t roll up to GL. Assets in those sub ledger accounts  belong to clients not the firm. If there’s any roll-up, it’s among the multiple accounts of a single client.

Spring self-quiz 2

Q1: spring bean scopes? prototype/singleton/request/session…
A1: http://static.springsource.org/spring/docs/2.5.x/reference/beans.html#beans-factory-scopes

Q3(common): why use spring?
%A3: JMS, MVC, tx, Spring security, AOP,  hibernate,

Q4: how do u use spring tx management
A4: xml or annotation

Q: what's so special about DI, if we can easily use method/constructor injection?

Q: auto wiring and @Inject …?:

Q: singleton bean?

various meanings of "forward"

Note any time people say “forward”, it’s relative to “spot” or current. Background – There’s always some _fluctuating_, measurable variable. Think of it as temperature. It has an observable current level, and people try to estimate the level in x months, and people trade on that estimate.

forward price of an underlying security — is the simplest. Relevant in the Black-Scholes

“forward price” in Treasury — is probably a misnomer IMO. P132 [[complete guide]] clarifies the 2 meanings of “forward-price”

forward rate — see blog on FRA. It is like a CD rate for a 3 month term starting x months from today.

forward contract — see blog. It’s a contract to delivery some copper, or currency, or a treasury.

collateral in various trading systems

I know a few distinct systems relying on collateral (re)valuation.

– margin trading. The assets bought is held under the broker's name. Used as collateral. Broker has the right to liquidate the position if a margin call fails.
** forwards
** OTC options
– CDS
– IRS ?
– repo
– regular car loan
– regular mortgage
– many corporate loans

In all cases, collateral needs to be re-valued periodically – mark-to-market.

I feel custodian bank doesn't hold your assets as collateral. They have no legal right to liquidate your assets.

reoffer engine: basic questions u r expected to know

Q: why wpf and swing, rather than browser-based?
A: real time updates of every position
A: traders need to switch among the grids without reloading from server. We have a local cache. Separate jvm from swing, and data-binding for wpf.

Q: what if 2 updates come from different sources and both trigger repricing. How does the classic system schedule them?
A: some cache triggers are single-threaded, some are multi-threaded. There's an incrementing version number.

Q: why is PNL updated in real time? isn't EndOfDay enough?
A: nice to have.
A: We only persist EOD pnl data

Q: how does libor or other benchmarks affect pricing?
A: Some positions are priced with a spread on benchmarks

Q: how many traders?
A: about 120

Q: what do the middle office do with your app?
A: MSRB
A: faxp
A: they might look at PnL on our Swing app

Q: how many middle office people?
A: Fewer than traders

Q: what other users use our app?
A: Trading floor support
A: FA

Q: how does our system calculate or use credit risk data?
A: we don't calculate anything using credit risk data
A: each bond has its own yield curve. Typically a spread on Libor. If you are an AAA bond, then your curve might be Libor itself. If you are BBB, then Libor + some spread. The spread depends on your credit rating.

Q: I thought prop traders manage their reoffer pricing using their own algorithm
A: prices maybe
A: marks are sometimes calculated in ARB and fed into Autoreo. Then Autoreo uses those marks for reoffer pricing.

Q: how does our trader hedge?
A: swaps, swaptions, caps/floors, CDS, treasuries, futures, stocks,

Q: What if we send out a wrong (very low) offer price and it's lifted by a buyer? How do we back out?
A: TMC can cancel it for us (so-called long-term “relationship”) , but in most cases, you can't simply cancel a trade simply because you want to change your mind.

Q: in classic, how does one cache trigger update cache in another JVM?
A: async by JMS subscription, and synchronously by RMI. Each cache JVM has a cache port.

Q: how does the swing app get notified when server side gets a new price, benchmark, pin, spread, new trade etc?
A: local cache subscribes to JMS

Q: bid/ask spread indicates liquidity?

a simple cache server staying alive

Main server threads simply keep running, usually blocked in socket.accept() or rmi server mode. This thread below is responsible to shut down JVM at 23:00. As a redundant measure, there’s also an Autosys stop job to shut down the JVM.

public void run () {
      if (hour < 1 || hour > 23) {            return;      }
      while (true) {
            try {
                  Thread.sleep (10 * 60 * 1000);
                  // set up hour from current system system
                  if (hour < 1 || hour > 23) {              System.exit (0);                  }
            } catch (Exception ex) {} // do nothing
      }
}

split 30 threads into 3 JVM processes@@

Q: if I want to support up to 30 concurrent tasks, which is better

X: 3 processes, 10 threads each
Y: 1 process holding 30 threads

It depends primarily on data-sharing. If there's a pattern of clustering among the threads, where most traffic is within 10 threads, then it's possible to run those 10 in one JVM.

If threads share data randomly and heavily, then X may require IPC overhead. You can minimize it with a database or file-based task allocation. IPC adds dev effort, maintenance effort, complexity.

In practice, X is more fault-tolerant. You can configure the processes differently.

ML muni trading engine IV

(Now I think this is an RV environment, so some of my assumptions below aren’t valid.)

Suppose a MOM holds a lot of queues. One of the queues is for a request message, from requester system to a pricing system. Another queue is for pricing system to return the new price to the requester.

Now, pricing system is slow. Requester should wait for no more than 5 sec. If the new price goes into the reply-queue 6 sec later, requester should ignore it. How do you implement this?

A: requester sends in one thread, then wait(5). Upon return, it checks if anything received on the onMsg() thread. If nothing received, then it assumes timeout.

java equals()hashCode() 2b overriden together@@

Update — It’s instructive to examine this contract along with the equals()/compareTo() consistency — see other blog posts.

The _only_ contract is equals() shows equal =} Hash must collide. The _only_ problematic scenario is “equals-without-collision”. All other scenarios are permissible under the java law.

The dire consequence in one sentence — “a Set contaminated with dupe objects“. Ignore maps at this stage.

  • Suppose myClass.java is used as HashSet keys.
  • Suppose I override equals() and end up having myClass instances A and B tested equal.
  • Suppose we didn’t bother to override hashCode() and end up with different hash values ! Oh my Goodness….
  • Suppose A is already in the hashset. B comes along.
  • ==> jvm sees different hash codes and keeps both in separate chains (separate chaining) in the hash table!

Q: That’s unexpected, but What’s the consequence?
A: rule violation.  A Set is supposed to keep only one of these 2 “equivalent”[2] objects.

[2] STL parlance?

The other mistake is similar. Suppose we did override hashCode() and get different hash values on A and B so jvm correctly keeps both in the hash table, but we didn’t bother to override equals() so the inherited equals() mistakes objects A and B to be equal.

Q: In what case can we safely break this rule and override equals() without hashCode()?
A: i think if you use any hash-free container. They never call hashCode(), so a broken hashCode() doesn’t matter.

spring ApplicationContext (MVC++) is like a container or server

AppCx often controls some thread pool(s), esp. in trading systems. AppCx (subclass of BeanFactory) has many “early” features of a container or server —

– It is automatically started at JVM start-up
– it instantiates many objects, calling special methods on each
* It has reference to those objects and therefore serve as a crucial lookup registry or service locator
– it often remains “alive” forever as if a daemon thread

SmartLifeCycle hooks into the start-up. So does InitializationBean and init-method.

Incidentally, there are similar start-up hooks in JMS template. http://www.jarvana.com/jarvana/view/org/springframework/spring/1.2.9/spring-1.2.9-javadoc.jar!/org/springframework/jms/core/JmsTemplate.html#execute(org.springframework.jms.core.SessionCallback,%20boolean) says “…allowing the JMS Connection to be started on the fly.” — pure magic. For a topic consumer, this is the crucial but hidden start of subscription. The unsubscribe is equally hidden — when the session is closed.

2-envelope (exchange) paradox

Paradox: In a game show there are two closed envelopes containing money. One contains twice as much as the other (both have even amounts). You [randomly] choose one envelope (denoted EnvelopeA) and then the host asks you if you wish to change and prefer the other envelope. Should you change? You can take a look and know what your envelope contains.

Paradoxical answer: Say that your envelope contains $20, so the other should have either $10 or $40. Since each alternative is equally probable, the expected value of switching is 1/2 $10 + 1/2 $40 which equals $25. Since this is more than your envelope contains, this suggests that you should switch. If someone else was playing the game and had chosen the second envelope, the same arguments as above would suggest that that person should switch to your envelope to have a better expected value.

——– My analysis ——-
Fundamentally, the whole thing depends on the probability distribution of X, where X is a secret number printed invisibly inside both envelopes and denotes the smaller of the 2 amounts. When the host presents you the 2 envelopes, X is already printed. X is “generated” at beginning of each game.

D1) Here’s one simple illustration of a distribution — Suppose a computer generates an even natural number X, and the game host simply puts X dollars into one envelope and 2X into the other. Clearly X follows a deterministic distribution implemented in the computer. X >= 2, and since a computer has finite memory it has an upper limit on X. We should also remember any game host can’t have a trillion dollars.

D2) Another distribution — the game host simply think of a number in a split second. Suppose she isn’t profit driven. Again this X won’t be too large like 2 million.

D3) Another distribution — ask a dog to fetch a number from the audience, each contributing 100 even integers.

In all the scenarios, as in any practical context there’s invariably a constraint on how large the integer can be. We will assume the game host announced X <= 1000.

Why we talk about “distribution” of X — because we treat X as a sample drawn from a population of eligible numbers, which is fundamental to D1/D2/D3 and all other X-generation schemes. Some people object, saying that population is unlimited, but I feel any discussion in any remotely practical context will inevitably come up against some real world constraints on X. A simple yet practical assumption is that X has a discrete uniform distribution [2, 1000]. As soon as we nail down the distribution of X, the paradox unravels, but I don’t have time to elaborate fully. Just a few pointers.

If we don’t open our first Envelop (A), then we must be indifferent, Regardless of distribution. If we do open, then long answer. Suppose you see $20.

If we have no idea how X is generated, we can safely assume it has an upper limit and a lower limit of 2. Still it’s insufficient information to make a decision. Switching might be better or worse or flat.

If the number generator is known to be uniform [2,1000] switch is profitable(?). If we saw $2000 (2 times the max X) we don’t switch. Any other amount we see in EnvelopeA, we switch (?). So most players would open first envelope and switch!

When we see $20 and still feel indifferent, we implicitly assume X distribution follows P(X=10) == 2*P(X=20). Not a uniform distribution but definitely conceivable.

———-
Let’s keep things even simpler. Say we know X has some unknown distribution but only 3 eligible values [2,4,8] and we see $4 in A. Why would some players feel indifferent after seeing $4? If an insider reveals the X-generation is heavily biased towards low amounts, then we obviously know B is highly likely to be $2 not $8 so don’t switch.

Extreme case — P(X=8) is near zero. P(X=2) is 10 times higher than P(4), so most players would see 2 or 4 when they open EnvelopeA, and very few see a $4 as the “small brother of $8”. After you see enough patterns, you realize A=4 is most likely to be the bigger brother of a $2.

Therefore the player indifference is based on no rational thoughts. Indifference is unwise (i.e. ignoring key signals) when we see a lot of pattern about the X-generation. You can probably apply conditional probability treating the pattern as prior knowledge.

message retention +! durable subscriber

Q: what conditions are necessary for keep-forever?
A (obvious): message has no expiration date
A (obvious) : if a durable sub has not sent an ACK.

From now on, we focus on non-durable, which is more widespread in trading engines.

A: If an NDS (non-durable-subscriber) subscribes, then is abruptly lost, right after a message is published? In this case, will the broker consider the message subscribed but undelivered?

A: if an NDS subscribes, but is too slow

A: if an NDS gets into an infinite loop so can’t disconnect

A: if an NDS gets into a deadlock so can’t disconnect

A: if an NDS freezes up but doesn’t disconnect??

NDS unsubscribe happens at TopicSubscriber.close(). P77 [[JMS]]. But does it happen during a crash????