月度归档： 2008年8月

Studying biz rule tables used on WallSt

发表于 作者 BinTAN

Background: many rule based systems (on Wall St or elsewhere) have thousands of rules in database tables. The simplest and most used pattern is the lookup table. A lookup table has one (or more) input columns and one (or more) output columns.

Each combination of input fields maps to exactly one row. There should be a unique composite index on the input columns as a group.

The concept is simple, but sometimes needs to be hand-crafted and perfected, because
* these rules are often part of the most important 1% source code in a big code base
* users change rules frequently, at least in some Wall St firms
* we often need to query the tables with complex criteria. Such complex queries are usually implemented in source code, but for quick production support we often need to run them by hand, tweak and rerun. In some cases, we end up learning all the intricacies by heart. If your full time job is a spell-checker you won’t need the dictionary much.
* we often need to explain how rules are disqualified, filtered out, ranked, grouped, applied, overlayed, re-applied…. Many IT veterans would say blow-by-blow knowledge of a code module is needed in extremely rare situations.. Welcome to Wall St these business rules are examined and tweaked on a daily basis by multiple groups of users.

Compare with Informatica lookup transform.

2 ways for http sessions to share a single object

发表于 作者 BinTAN

2 ways for sessions to share a single huge object

Requirement: a huge object to be instantiated before the first session, and kept alive after all sessions end. Object is not immutable.

Solution 1: put that object in an app-scope bean.

Solution 2: static field in a special class. Your huge object will be reachable from the persistent class-object.

Objects used by each method call

发表于 作者 BinTAN

When studying method calls (aka messages) among objects, look for the objects used by each call. For a “call” (a better focus than “method”), the most important objects in practice are, ranked by usage

– member objects in the args — folks often pass in a collection or “composed” object just to pass in member objects.

– member objects in the host

– args

– host object — If the host object is not used by the call, then this method should be static.

– caller object must appear as args, like m1(this,.) — Without “this” as an argument, the call can’t access the caller object. Caller object exists as a “local” variable upstairs the call stack (thread).

## IV halo across domains #specific=better

发表于 作者 BinTAN

See also — relevant55.tech.txt, [[ further improve ]], [[ towards an architect ]]. This halo list is not that rigorous, only a /sideshow/.

If you invest more than enough into …. say, java concurrency, you may grow an invisible little halo in the eyes of peers and interviewers. Other halos: google brank, microsoft brank, GS brank

  1. [a] lock free, CAS, statement reordering, membar, volatile^atomic
  2. conditionVar, spurious wake-up
  3. [a] read/write lock, tryLock, deadlock prevention
  4. [a] tail recursion
  5. sql tun` including EXPLAIN, hints, statistics,
  6. [a] hash table internals, red black tree
  7. [a] O(n) sorts
  8. [a] Morris

C++ specific:

  1. [ah] compile-time template meta-programming
  2. smart pointers used with factory, class field etc
  3. epoll,
  4. [h] reinterpret_cast

Java specific:

  1. jvm tun` including mem, GC, profilers
  2. thread pool usage and internals
  3. callable/future
  4. [h] remote debugging
  5. [h] reflection? not really seen as halo
  6. [h] JMX
  7. [a] concurrent skip list,

also-rans

  • [a] tx isolation levels
  • solaris/linux tun` && capacity plann`@@ Rember the Solaris 10 seminar?
  • serialization

[a=academic, could be considered zbs, never really needed in GTD]
[h=hacky, powerful black magic, rather useful but often frowned upon]

STL creators’ design priority

发表于 作者 BinTAN

Simplicity and ease of use was a low priority for the STL creators. Efficiency is perhaps #1 design priority. Both time cost and space cost. I guess STL was designed to be usable in telecom and other real time small equipment with limited memory and CPU resources.

Whenever people optimize for efficiency, they *specialize* their components for various scenarios. I think that explains the proliferation of iterators. Also, many algorithms overlap, as methods and as free functions.

Template itself was invented for efficiency, at the cost of simplicity. STL exemplifies template usage. Efficiency in terms of code reuse and algorithm abstraction.

The father of STL doesn’t believe in OO. Any OO design in STL? STL was designed just like C (without classes) — for maximum algorithm efficiency and maximum abstraction. Classes and templates were employed as a means to that end.

G5 key tibRV objects for Java developers

发表于 作者 BinTAN

* msg objects — well-defined entity beans
* transport objects

The 3 objects below build on top of msg and transports … These are described in one single chapter [[Events and Queues]] in the java documentation.

* listener objects
* callback objects, implementing onMsg()
* queue objects and dispatchable interface

Less important:
* dispatcher object (subclass of Thread)

—
Once you understand the above, you can see that certified messaging builds on top of them:
* CM transport objects
* CM listener objects
* CM review callback — for reviewing “missed calls”
* CM msg objects

java collections — practical questions

发表于 作者 BinTAN

Q: most popular concurrent data structures?
A: concurrent hash map is #1. #2 is blocking queues. Both use Lock.java interface

Q: drawback of concurrent hash map?
A: size()?

Q: most useful utilities in Collections.java?
A: sort, synchronized….

Q: what’s concurrent modification exception?

Q: what’s fail fast?
A: see javadoc on ConcurrentModificationException

Q: interviewer’s favorites?
A: hash tables is #1, also array list, linked list,

eclipse Factory refactor — keywords

发表于 作者 BinTAN

Background: you apply this operation “introduce factory” on an existing contructor.

* new factory class or same class — new factory method can be created in the same class as the contructor or in a new factory class. Both are standard practices.

* return interface — instead of classes. Eclipse can’t automatically.

* static — method. Usually the factory method is static, to allow everyone to call it.

* private — by default the original contructor gets privatized

y Trigger

发表于 作者 BinTAN
A few benefits of triggers in the mortgage commissions system

* testability — To test the triggers, just run one or a bunch of inserts. Use sqsh, aqua studio, fitnesse or any other tool. If the logic is in Perl/Java, i don’t think it would be so simple.

* Modular — All (and only) the logic related to inserting records are contained in the triggers. This is separate from other logic. Separation of concern. Other developers are shielded from the complexity of saving records.

* complexity — The insert trigger in the MtgAccts table involves more than 5 queries. To implement the same logic using Java or Perl would take 3-5 times more lines of code, by my estimate.

* flexibility — during production support. Suppose we need to load a few records for a production issue. Just write the query by hand and test in QA and then run it in sqsh/aqua or another tool.
* reuse — 70% of the trigger code in MtgAccts table are used in other tables and also used in the spApproveMtg stored proc.

There are some concerns …

* performance — is not a top priority. Each month’s load job takes less than 30 seconds now.

* UDB — There’s some concern over migrating stored proc and triggers to UDB. Trigger support in UDB is probably richer than sybase. Therefore the triggers are probably UDB-friendly.

no throw from destructors please

发表于 作者 BinTAN
http://www.parashift.com/c++-faq-lite/exceptions.html#faq-17.3 is a short and clear explanation. Here are my comments:
No throw please until you catch the original exception. In fact, Most if not all the methods in exception class are declared with an empty throw()

——
The C++ rule is that you must never throw an exception from a destructor that is being called during the “stack unwinding” process of another exception. For example, if someone says throw Foo(), the stack will be unwound so all the stack frames between the throw Foo() and the } catch (Foo e) { will get popped. This is called stack unwinding.
During stack unwinding, all the local objects in all those stack frames are destructed. If one of those destructors throws an exception (say it throws a Bar object), the C++ runtime system is in a no-win situation: should it ignore the Bar and end up in the } catch (Foo e) { where it was originally headed? Should it ignore the Foo and look for a } catch (Bar e) { handler? There is no good answer — either choice loses information.
So the C++ language guarantees that it will call terminate() at this point, and terminate() kills the process. Bang you’re dead.
The easy way to prevent this is never throw an exception from a destructor. But if you really want to be clever, you can say never throw an exception from a destructor while processing another exception. But in this second case, you’re in a difficult situation: the destructor itself needs code to handle both throwing an exception and doing “something else”, and the caller has no guarantees as to what might happen when the destructor detects an error (it might throw an exception, it might do “something else”). So the whole solution is harder to write. So the easy thing to do is always do “something else”. That is, never throw an exception from a destructor.
Of course the word never should be “in quotes” since there is always some situation somewhere where the rule won’t hold. But certainly at least 99% of the time this is a good rule of thumb.

user-dict^idict ] python #speculations

发表于 作者 BinTAN

Never tested in IV and never needed in projects…

There are various terms for the same concept
– registry — generic concept across languages. A class has a registry and an instance has a registry too.
– namespace — has special meaning in python, but some veterans say the various registries are implemented same as a namespace
– idic i.e. internal dict, but exposed for runtime introspection

2 types of dictionaries in python —

A) user dictionary are the well-understood, simple dictionaries you create
B) “idic” meaning “internal dict” is a building block of python objects, (possibly) including user dictionary objects.

A user dict is usually taught as an object with methods and fields. I hope that’s not over-simplification.

A namespace for a module (or class or anything) is implemented as an idic. Accessed as whatever.__dict__. In a twist for consistency and coherence, this idic can be *used* as a regular user dictionary. You can do whatever.__dict__.keys(). You may rightly feel that a __dict__ is implemented the same as a user dictionary, but I doubt it. It could be implemented as a complicated symbol table but exposed as a simple dict.

Even if a __dict__ (i.e. an idic) is implemented the same way as a user dict, I strongly believe beginners better treat them as 2 distinct types unrelated to each other, though the syntax looks similar.

Internal dict is the basis of python objects[3].  I guess a idic is more like a small expandable hash table [2]. Must be small because a *concrete* idic instance exists in EVERY parent class, subclass, and EVERY instance[1] of EVERY class.

[1] It’s not too wrong to say it resembles the C struct.
[2] a perl object IS nothing but a hashtable or associative array. See the camel book. In php, an object is implemented as an associative array (or simply an “array”).
[3] at least user defined objects

forward contract^futures contract

发表于 作者 BinTAN

a futures contract == a standardized forward contract guaranteed by an exchange, just as
a listed option == a standardized version of an OTC option

I feel we should first understand commodity forward contracts first. Commodity futures are the simplest and earliest form of futures. Forward contract is the foundation of futures; futures contracts form a foundation for IRS, swaption and other derivatives.

Note the fundamental entity (or tradable instrument) is the contract. A forward corn contract is nothing but a spot corn contract with a future delivery date. Note price is fixed at the transaction time, otherwise there’s no contract, no agreement, no obligation.

In mid 1997 i took short positions on copper futures market, agreeing to sell (for a future date) at relatively low delivery prices (say $121k) and lost badly when prices rose. I had to cover the short by entering a long position (say 131k). Counter party to every contract is always the Exchange. Effectively, i agreed with Exchange to sell to Exchange at 121k and buy at 131k for the same quantity of copper for the same delivery date. When I entered each position some sum was frozen in my margin account. Since my 2 positions cancelled out and $10k was permanently frozen (actually transferred out). Exchange or the broker allowed me to close my margin account.

This is a forward contract, but traded on the exchange.

http://thismatter.com/money/forex/fx_forwards.htm says

“FX futures are basically standardized forward contracts. Forwards are contracts that are individually negotiated and traded over the counter, whereas futures are standardized contracts trading on organized exchanges. Most forwards are used for hedging exchange risk and end in the actual delivery of the currency, whereas most positions in futures are closed out before the delivery date”

java calling win32 tibrvsend.exe

发表于 作者 BinTAN

            /**
            * –daemon and the arg must be distinct strings
            *
             * quotes aren’t necessary
            */
            String[] strArrayWithoutQuote = new String[] {
                        “C:\\tibrv\\8.3\\bin\\tibrvsend”, “-daemon”,
                        “localhost:7500”, “-service”, “9013”, “-network”,
                        “;239.193.224.50,239.193.224.51;239.193.224.50”,
                        “GENERIC.g.TO_MTS_TRADE_REQUEST”, xml };
            System.out.println(Arrays.asList(strArrayWithoutQuote));
            execAndWait(strArrayWithoutQuote);
      }
      /**
      * http://www.rgagnon.com/javadetails/java-0014.html says — If you need to
      * pass arguments, it’s safer to a String array especially if they contain
      * spaces.
      */
      static private void execAndWait(String[] command)
            try {
                  Runtime runtime = Runtime.getRuntime();
                  Process p = runtime.exec(command);
                  BufferedReader stdInput = new BufferedReader(new InputStreamReader(p
                              .getInputStream()));
                  String s = null;
                  while ((s = stdInput.readLine()) != null) {
                        System.out.println(s);
                  }
                  BufferedReader stdError = new BufferedReader(new InputStreamReader(p
                              .getErrorStream()));
                  while ((s = stdError.readLine()) != null) {
                        System.err.println(s);
                  }
                  p.waitFor(); // advised to do this after streams
            } catch (IOException e) {
                  throw new RuntimeException(e);
            } catch (InterruptedException e) {
                  throw new RuntimeException(e);
            }
      }

interbank broker^ECN, FX or Treasury

发表于 作者 BinTAN

FX Interbank broker (EBS or Reuters) is like eSpeed and brokerTec; FX ECN is like tradeweb — connecting big dealers to instituional clients including corporations and fund managers.

Many if not most ECN’s are anonymous, just like EBS (and some Reuters systems). Some large clients may have a relationship with big dealers so they may probably prefer conversational dealing just like Reuters Dealing.
* FXAll’s original system is a relationship based conversational dealing system between dealers and institutional clients.
* 360T offers a relationship-based, non-anonymous platform.

Reuters Dealing 3000 spot/forward are anonymous.

For tradeweb, liquidity providers are primarily large banks. I think FX ECN may have some hedge funds serving as liquidity providers too.

In FX jargons, “market-maker” and “liquidity provider” (also Dealing Desk) generally refers to dealers with inventory. ECN usually means a platform operator forever without position. “Broker” is used very loosely in the FX business and can include any entity except a retail client.

count(nullable_field) can return 0 – tricky in outer join

发表于 作者 BinTAN

Opening quiz: how do you get count() to return a 0? I believe most novice SQL developers are unfamiliar.

See the vmware questions in pearl blog. Tested in sybase.

select count(field_allowing_nulls) — would not count the /null-cells/. See P 91 [[ introduction to sql ]]. Note you can think of the table as a spreadsheet with named and numbered columns, numbered rows, and numbered cells.

select count(*), count(1234) — would simply count all rows in the entire table without filtering

This null-behavior is esp. important in outer joins.

Generalize -> Just like count(), many aggregate functions like agg_func(filed_allowing_nulls) would filter out null values before any calculation. Best understood with average(), which must discard all null values before computation.

(badly) implicit ctor calls — var declaration

发表于 作者 BinTAN

Q: When do c++ constructors get invoked? Harder to tell than in java.

MyClass a; // calls noarg, behaves exactly like int/float and other builtin types

a = MyClass(); // calls noarg, assignment, and destructor.
// New object created on stack then duplicated in a’s memory location, field by field. New object is then discarded as a STACK var.
// I believe the parentheses are compulsory after the class name.

MyClass arr[9]; // nine constructor calls. no-arg constructor needed! Note the position of [].

MyClass b(a); // calls copier ….. [2]

C c3 = c1; // calls copier, not assignment. same as [2]. Disabled by “explicit” keyword on the copier ctor

C c2;
c2 = c1; // calls assignment, not copier

////// So far, all the variables are nonref stack vars.

MyClass b(); // parsed as a function prototype, not a constructor call

MyClass * ptr; // confirmed no constructor call. The pointer is uninitialized.

MyClass * ptr = 0; // ptr starts off as uninitialized, then it is initialized to 0 i.e. null. You can see these 2 steps by debugging.

y FX and MM are in one desk

发表于 作者 BinTAN

In Citi and ML, FX desk also handles money-market (“local market”) deals. This is largely because that client needs, pricing and operations in both markets (FX forward and MM) have always been closely linked and inter-dependent [1] for hundreds of years.

I believe Money market means short-term lending market. Usually up to 12 months.

[1] Personally I feel it’s more of a one-way dependency, rather than mutual-dependency. Every client with regular FX transactions has periodic, predictable FX needs in the near future. FX forwards always, always involves (short-term usually) interest rates.