Real Time Symbol Data is responsible for sending out all security/product reference data in real time, without duplication.
- latency — typically 2ms (not microsec) latency, from receiving to sending out the enriched reference data to downstream.
- persistence — any data worthing sending out need to be saved. In fact, every hour the same system sends a refresh snapshot to downstream.
- performance penalty of disk write — is handled by innoDB. Most database access is in-memory. Disk write is rare. Enough memory to hold 30GB of data. https://bintanvictor.wordpress.com/2017/05/11/exchange-tickers-and-symbols/ shows how many symbols there across all trading venues.
- insert is actually slower than update. But first, system must check if there’s a need to insert or update. If no change, then don’t save the data or send out.
- burst / surge — is the main performance headache. We could have a million symbols/messages flooding in
- relational DB with mostly in-memory storage