essential FIX messages

Focus on just 2 app messages – new order single + a partial fill. Logon/Logout/heartbeat… LG2 — important session messages.

Start with just the 10 most common fields in each message. Later, you need to know 20 but no more.

For the most important fields – tag 35, exectype, ordstatus, focus on the most important enum values.

This is the only way to get a grip on this slippery fish

25-13:34:27:613440 |250—-> 16 BC_ITGC_14_CMAS_104 1080861988 |

8=FIX.4.2^A9=0162^A35=D^A34=00250^A52=20120725-17:34:27.610^A49=BARCAP014A^A56=ITGC^A115=BARCAP1^A11=1040031^A22=5^A38=100^A40=1^A44=0.01^A47=A^A48=RIM.TO^A54=1^A55=RIM^A59=0^A100=ITGC^A376=-863982^A10=124^A

|

— A NewOrderSingle followed by 2 exec reports —

25-13:34:27:802535 |<—-252 13 BC_ITGC_14_CMAS_104 1079105241 |

8=FIX.4.2^A9=220^A35=8^A128=BARCAP1^A34=252^A49=ITGC^A56=BARCAP014A^A52=20120725-17:34:27^A37=869-1^A17=10^A150=0^A20=0^A39=0^A55=RIM^A54=1^A38=100^A32=0^A31=0.000000^A151=100^A14=0^A6=0.000000^A11=1040031^A40=1^A22=5^A48=RIM.TO^A60=20120725-17:34:27^A59=0^A47=A^A10=002^A

|

25-13:34:27:990564 |<—-253 13 BC_ITGC_14_CMAS_104 1079105241 |

8=FIX.4.2^A9=230^A35=8^A128=BARCAP1^A34=253^A49=ITGC^A56=BARCAP014A^A52=20120725-17:34:27^A37=869-1^A17=20^A150=2^A20=0^A39=2^A55=RIM^A54=1^A38=100^A32=100^A31=7.010000^A151=0^A14=100^A6=7.010000^A11=1040031^A40=1^A22=5^A48=RIM.TO^A60=20120725-17:34:27^A59=0^A47=A^A30=MSCN^A10=076^A

|

Advertisements

Fwd: design a finite state machine for a FIX engine

We need a cache of all the pending orders we created. All from-exchange FIX messages must match one of the pending orders. Once matched, we examine its current state.

 

If the message is belated, then we must reject it (informing all parties involved).

 

If the message is delivered out of sequence and too early, then we must keep it on hand. (a 2nd cache)

 

If the message is for immediate consumption, then consume.

 

———-

 

the FSM in FIX is about state transition and should be pertinent to all state machines. Basically, it goes from pending new to new to fill or cancelled. From cancelled, it cannot go back to new or fill etc.

Anthony

 

FIX^MOM untangled – exchange connectivity

The upshot — some subsystem use FIX not MOM; some subsystem use MOM not FIX. Some sites send FIX over MOM (such as RV) but not common.

It’s important to realize FIX is not a type of MOM system. Even though FIX uses messages, I believe they are typically sent over persistent TCP sockets, without any middle-ware. Consider ICE data service.

Say we (sell-side or buy-side) have a direct connectivity to an exchange. Between exchange and our exterior gateway, it’s all sockets and possibly java NIO — no messaging. Data format could be FIX, or it could be the exchange’s proprietary format — here’s why.

The exchange often has an internal client-server protocol — the real protocol and a data flow “choke point” . All client-server request/response relies solely on this protocol. The exchange often builds a FIX translation over this protocol (for obvious reasons….) If we use their FIX protocol, our messages are internally translated from FIX to the exchange proprietary format. Therefore it’s obviously faster to directly pass messages in the exchange proprietary format. The exchange offers this access to selected hedge funds.

As an espeed developer told me, they ship a C-level dll (less often *.so [1]) library (not c++), to be used by the hedge fund’s exterior gateway. The hedge fund application uses the C functions therein to communicate with the exchange. This communication protocol is possibly proprietary. HotspotFX has a similar client-side library in the form of a jar. There’s no MOM here. The hedge fund gateway engine makes synchronous function calls into the C library.

[1] most trader workstations are on win32.

Note the dll or jar is not running in its own process, but rather loaded into the client process just like any dll or jar.

Between this gateway machine and the trading desk interior hosts, the typical communication is dominated by MOM such as RV, 29West or Solace. In some places, the gateway translates/normalizes messages into an in-house standard format.

design considerations – my first homemade FIX client

In a typical sell-side / market maker, the FIX gateway is a FIX client for a liquidity venue. The liquidity venue (typically an exchange, but also ECNs, dark pools) runs a massively multi-threaded, FIX server farm, possibly with custom-made FPGA hardware. I believe the technical requirements for an exchange FIX server is different from a sell-side FIX client.

Therefore, I assume my audience are far more interested in the client code than the server code.

My mock server implementation is a mock-up, with no optimization, no resilience/robust design, no fault-tolerance, no select()-based sockets, no high-availability, no checksum validation.

— design priorities —
Since this is a from-scratch implementation of a large spec, I focused on basic functionality. Debugging and instrumentation is important in the early stage. A lot of extra code was created to help developers verify the correct operation of all those nitty gritty details.

I don’t aim to build a complete engine, but the basic functionality I build, i try to build it on solid ground, so hopefully it doesn’t become throw-away code.

At the heart of the OO design was the abstraction expressed by AbstractClient, AbstractState and ClientSession. These form the backbone of the core object graph. The fields and constructors were chosen with care. Cohesion and low-coupling were my ideals but sometimes hard to achieve.

These core classes help meet most of the fundamental requirements related to object state of a client connection, sequence number, state transition, output stream sharing, connection liveness monitoring… I wouldn’t say these are well-design classes, but they are a reasonable first attempt at modelling the problem using java classes.

Reusable components were created whenever possible, such as the message parser, message formatter.

FIX engine are highly configurable. All config parameters are extracted into a config object, which is obtained from a config factory. One config for Test, one for Production or UAT etc.

— features —
– parsing a raw FIX msg into an array, which is faster to access than a hashmap. By our design, we avoid hash lookup completely.
– Destroyer.java is a home-made solution to the potential forever-blocking of readLine(). On some platforms, socket read is not interruptible and the only way to unfreeze a socket read is to close the socket. We do that in this implementation but only after sending a final logout message.
– Various template methods to guarantee close() and guard against resource leak
– AbstractClient has most of the generic logic, while SingleOrderClient implements very small amount of custom logic.
– FIX msg formatter is generic enough to accept a map of tags and values — any tags and values
– sequence num validation
– state machines on both client and (simplified) server. State design pattern.
– server side — very few features, but ConnectionManager is reasonable encapsulation of new socket creation.

—- client-side features to be implemented —-
– heart beat — more robust. Should never fail.
– timeout waiting for ack
– re-logon if no heartbeat for 60 seconds
– checksum validation
– storing exchange order-ack into database
– Right now, given time constraint, i didn’t use nonblocking IO. All FIX system should use non-blocking IO.
———- Disclaimer ————-
I spent about 4 hours on this project, as suggested by Jonathan. I had to do some basic research on FIX since I have never programmed FIX (except some ECN connectivity modules that probably used FIX under the hood).
————— How to run the server/client —————–
Run ServerMain.java, then ClientMain.java.
To see the heart beat monitoring/intervention (Destroyer) in action, put 500 in getSilenceThreshold().

top 10 fields in NOS ^ exec report

—- top 5 fields common to NOS (NewOrderSingle) and exec report —
#1) 22 securityIDSource. Notable enum values
1 = CUSIP
5 = RIC code
*** 48 SecurityID, Requires tag 22 ==NOS/exec
*** 55 optional human-readable symbol ==NOS/exec

#2) 54 side. Notable enum values
1 = Buy
2 = Sell

#3) 40 ordType. Notable enum values
1 = Market order
2 = Limit order

#4) 59 TimeInForce. Notable enum values
0 = Day (or session)
1 = Good Till Cancel (GTC)
3 = Immediate Or Cancel (IOC)
4 = Fill Or Kill (FOK)

—- other fields common to NOS and exec type
*11 ClOrdID assigned by buy-side
*49 senderCompID
*56 receiving firm ID
*38 qty
*44 price

—- exec report top 5 fields
* 150 execType
* 39 OrdStatus. Notable enum values
0 = New
1 = Partially filled
2 = Filled

* OrderID assigned by sell-side. Not in NOS. Less useful than the ClOrdID

client conn IV#FIX/java

Q: OrdStatus vs ExecType in FIX?
http://fixwiki.fixprotocol.org/fixwiki/ExecutionReport
http://fixprotocol.org/FIXimate3.0/en/FIX.5.0SP2/tag39.html — good to memorize all of these

Each order can generate multiple execution reports. Each report has an exectype describing the report. Once an order status changes form A to B, OrdStatus field changes from A to B, but in the interim the exectype in the interim execution reports can take on various values.

Enum values of exectype tag and ordstatus tag are very similar. For example, a partial fill msg will have exectype=F (fill) and ordstatus=1 (partial fillED).

exectype is the main carrier of the cancellation message and the mod message.

In a confusing special case, The exectype value can be the letter “i”, representing OrderStatus, spelt “OrderStatus”.

Q: what’s a test request in FIX
%%A: invite opposite party to send a test heartbeat.

Q: what’s send Message() vs post Message in COM?
http://wiki.answers.com/Q/What_is_the_difference_between_PostMessage_and_SendMessage

Q: intern()?

Q: are string objects created in eden or ….?

socket, tcp/udp, FIX

I was told both UDP and TCP are used in many ibanks. I feel most java shops generally prefer the simpler TCP.
Specifically, I learnt from many veterans that some ibanks use multicast (UDP) for internal messaging.

FIX is not much used. It is sometimes used as purely message formatting protocol (without any session management) on top of multicast UDP. (In contrast, full-featured FIX communication probably prefer TCP for connection/session control.)

Similar – Object serialization like protobuf.

##essential features of FIX session

Top 5 session-message types– Heartbeat(0), Test Request(1) ResendRequest(2), Logout(5) / Logon(A).[1] This short list highlight the essential functionality of the concept known as “FIX session”. I feel the most important features are sequence number and heart beat.

FIX session concept is different from a database server session. I feel FIX session is more like a TCP connection.

http://fixprotocol.org/documents/742/FIX_Session_Layer_rev1.ppt says A FIX Session can be defined as “a bi-directional stream of ordered messages between two parties within a continuous sequence number series”

Each FIX Engines is expected to keep track of both sequence numbers, including The Incoming Sequence Number expected on inbound messages received from the counterparty

http://javarevisited.blogspot.com/2011/02/fix-protocol-session-or-admin-messages.html is a blog post by a fellow developer in finance IT.

http://aj-sometechnicalitiesoflife.blogspot.com/2010/04/fix-protocol-interview-questions.html is another blogger

[1] In parentheses are MsgType values, which are ALWAYS single-characters.

quickly parse a FIX message into key/value pairS

http://bigblog.tanbin.com/2012/03/design-considerations-my-first-homemade.html is the context.

    /**
     * Parsing to array is faster than parsing to a HashMap
     */
    public static ArrayList quickParse(String fromServer) {
        ArrayList ret = new ArrayList(Collections.nCopies(
            MAX_FIX_TAG, (String) null));
        for (String pair : fromServer.split(SOH)) {
            String[] tmp = pair.split(“=”);
            if (tmp.length != 2)  continue;
            int tag = Integer.parseInt(tmp[0]);
            ret.set(tag, tmp[1]); //treat ArrayList as array — efficient
        }
        return ret;
    }