strategic value of MOM]tech evolution

What’s the long-term value of MOM technology? “Value” to my career and to the /verticals/ I’m following such as finance and internet. JMS, Tibrv (and derivatives) are the two primary MOM technologies for my study.

  • Nowadays JMS (tibrv to a lesser extent) seldom features in job interviews and job specs, but the same can be said about servlet, xml, Apache, java app servers .. I think MOM is falling out of fashion but not a short-lived fad technology. MOM will remain relevant for decades. I saw this longevity deciding to invest my time.
  • Will socket technology follow the trend?
  • [r] Key obstacle to MOM adoption is perceived latency “penalty”. I feel this penalty is really tolerable in most cases.
  • — strengths
  • [r] compares favorably in terms of scalability, efficiency, reliability, platform-neutrality.
  • encourages modular design and sometimes decentralized architecture. Often leads to elegant simplification in my experience.
  • [r] flexible and versatile tool for the architect
  • [rx] There has been extensive lab research and industrial usage to iron out a host of theoretical and practical issues. What we have today in MOM is a well-tuned, time-honored, scalable, highly configurable, versatile, industrial strength solution
  • works in MSA
  • [rx] plays well with other tech
  • [rx] There are commercial and open-source implementations
  • [r] There are enterprise as well as tiny implementations
  • — specific features and capabilities
  • [r] can aid business logic implementation using content filtering (doable in rvd+JMS broker) and routing
  • can implement point-to-point request/response paradigm
  • [r] transaction support
  • can distribute workload as in 95G
  • [r] can operate in-memory or backed by disk
  • can run from firmware
  • can use centralized hub/spoke or peer-to-peer (decentralized)
  • easy to monitor in real time. Tibrv is subject-based, so you can easily run a listener on the same topic
  • [x=comparable to xml]
  • [r=comparable to RDBMS]

##simplicity@design pushed to the limit

Here I collection simple concepts proven rather versatile, resilient, adaptable. Note in these designs, the complexity can never disappear or reduce. Complexity shifts to somewhere else more manageable.

  • [c] stateless — http
  • [!s] microservices –complexity moves out of a big service into the architecture
  • [c] pure functions — without side effects
  • use the database concept in solving algo problems such as the skyline #Gelber
  • stateless static functions in java — my favorite
  • [c !s] garbage collection — as a concept.Complexity shifts from application into the GC codebase
  • REST
  • in c# and c++, all nested classes are static, unlike in java
  • python for-loop interation over a dir, a file, a string … See my blog post
  • [c] immutable — objects in concurrent systems
  • [c] STM i.e. single-threaded mode, without shared mutable #Nsdq
  • [c] pipe — the pipe concept in unix is a classic
  • JSON
  • [c] hash table as basis of OO — py, javascript, perl..
  • [c] sproc (+trigger) — as a simple concept “data storage as guardian of its data, and a facade hiding the internal complexities”
  • [!s] dependency injection
  • [c !s] EDT — swing EDT and WPF
  • [c] RAII
  • smart pointers as a concept
  • singleton implemented as a static local object, #Scott Meyers
  • [c=celebrated, classic, time-honored, ..]
  • [!s = not so simple in implementation]

 

stateless (micro)services #%%1st take

in 2018, I have heard more and more sites that push the limits of stateless designs. I think this “stateless” trend is innovative and bold. Like any architecture, these architectures have inherent “problems” and limitations, so you need to keep a lookout and deal with them and adjust your solution.

Stateless means simplicity, sometimes “extreme simplicity” (Trexquant)

stateless means easy to stop, restart, backup or recover

Stateless means lightweight. Easy to “provision”, easy to relocate.

Stateless means easy scale-out? Elastic…

Stateless means easy cluster. Http is an example. If a cluster of identical instances are stateless then no “conversation” needs to be maintained.

microservices “MSA” #phrasebook

I feel MSA is more of a architect interview topic, not a developer interview topic. Dev complexity is low by design.

eg: error acct lookup, receiving productId + possibly a clientId, returning an error acct

Now the phrasebook:

  • jxee — As of 2019, I guess jxee has the best support for MSA
  • enterprise — enterprise-bias. Most of the practices used in SOA/MSA come from developers who have created software applications for large enterprise organizations.
  • SOA — is the ancestor and now out of fashion. I think MSA will also fall out of fashion.
  • stateless — stateless microservice is best. Can be highly concurrent and scaled out
  • scalability — hopefully better
  • decentralized — rather than monolithic
  • modularity
  • communication protocol — supposedly lightweight, but more costly than in-process communication
    • http — is commonly used for communication. Presumably not asynchronous
    • messaging — metaphor is often used for communication. I doubt there’s any MOM of message queue.
  • cloud-friendly — cheaper
  • flexible — in the face of changing requirements, though I’m not sure time-to-market will improve
  • simple-facade — (of a big monolithic service) is now replaced by more complex interface, so I suspect this is not always popular.
  • complexity — (various forms) is the public enemy but I don’t know which weapon (REST,SOA,ESB,MOM,Spring) actually works
  • in-process — services can be hosted in a single process, but less common
  • devops — is a driver
    • testability — each service is easy to test, but not integration test
    • loosely coupled — decentralized, autonomous dev teams
    • deployment — is ideally independent for each service, and continuous, but overall system deployment is complicated

c# static classes : java/c++

–c++:

use a (possibly nested) namespace to group related free functions. See google style guide.

c# has static classes. C++ offers something similar — P120 effC++. It’s a struct containing static fields. You are free to create multiple instances of this struct, but there’s just one copy for each field object. Kind of alternative design for a singleton.

This simulates a namespace.

–java:

In [[DougLea]] P86, this foremost OO expert briefly noted that it can be best practice to replace a java singleton with an all-static class

–c# is the most avant-garde on this front

  • C# static class can be stateful but rarely are
  • it can have a private ctor

Single-Threaded^STM #jargon

SingleThreadedMode means each thread operates without regard to other threads, as if there’s no shared mutable data with other threads.

Single-Threaded can mean

  • no other thread is doing this task. We ensure there’s strict control in place. Our thread still needs to synchronize with other threads doing other tasks.
  • there’s only one thread in the process.

RESTful – phrase book

agile – …

coupling – less coupling between client and server, so server side changes are easier. I think SOAP requires client rebuild.

b2b – still dominated by SOAP

resource-oriented services – …

object – each URL logically represents an object, which you can Get (query), POST (create), PUT (update its content) or DELETE

Q: is REST simpler than traditional SOA or SOAP web services?

Q: the concept of “resource” — does it sometimes mean a database record?

java cod`IV #threading #Gelber

My solution was attached on 27 Jan 2011
——-fwd——–
Hi Bin,
Here is the problem I was speaking about:
1. Develop an application in Java that will satisfy the following requirements:

– Read in any number of text files.
– Calculate 5 most commonly used letters
– Calculate 3 most commonly used characters
– Calculate 3 most commonly used numbers
– Calculate 10 most commonly used words
– Concurrently parse all files, however, limit your application to 2 parser threads. If there are more files than available parser threads, you will need to queue files for parsing.

2. Write an application in Java that will pull down the HTML content of any number of specified websites – single file per URL, no depth. Strip out all metadata and generate statistics by showing 2 most commonly used letters, numbers, characters and words.

Regards,
Brendan

[12] y implement state transition by static methods#OMS

(This discussion is relevant to any high-concurrency finite state machine or FSM, but I will use an order management system as example.)

Suppose my FSM implements state transitions in the form of instance methods defined on the Order object, myOrder->A_to_B(Event). Inside such a method, we verify this->currentState == A, validate other Conditions, and take the appropriate Action to process the Event, before updating this->currentState = B.

(The Transition/Event/Condition/Action/Validation concepts are fundamental to FSM in general. See http://www.thetibcoblog.com/2007/06/26/differences-between-a-bre-and-a-rule-driven-cep-engine-part-1)

I once proposed that A_to_B could convert to a static (java/c#/c++) or (c++) global, free-standing function A_to_B(Event, Order). I once said that this “might” be a bit easier to multi-thread. Now I feel in most cases such a simple change (from non-static to static) won’t improve concurrency. Such a change may have bigger benefits (to be elaborated) but concurrency is rarely one of them. However, here’s one possible “rare” context.

Following Functional Programming principle (as seen in FMD), each and every object is immutable, so is Order. Therefore the global function will return a new Order instance but representing the same real world order. Immutability removes the need to synchronize, completely.

Q: But what if 2 threads need to access a given Order?
A: If one of them is a reader, then immutability will probably help.
A: If both are writers, then my counter question is why this scenario is allowed to exist —

I feel in low latency OMS each order should ideally be updated by the same physical thread throughout. Multi-queue — each security/cusip belongs exclusively to a single thread, never hopping from thread X to thread Y — Rule 1.

I feel Rule 1 is an extremely practical rule but in some contexts it is over-restrictive and needs to be relaxed, but still at any moment an order is only updated on a single thread, never concurrently on 2 threads — Rule 1b.

However, I feel all good software rules have exceptions (as explained elsewhere in my blog) so sometimes it’s legitimate to break Rule 1b, though I won’t condone such reckless practice. Well I’m not boss…:)

In such a system, I take the liberty to assume every writer thread must put the updated Order into a shared cache. 2 writers using the instance method A_to_B()….must synchronize on the cache (usually a hash map). The FP design could use CAS. Since the chance of concurrent write is low, CAS retry is is unlikely, so CAS will be faster. Uncontended synchronization is always slower than uncontended CAS — see Brian Goetz article on http://www.ibm.com/developerworks/java/library/j-jtp04186/index.html.

However, immutability entails more object creation instead of updating mutable fields. Trade-off to be measured.

y many java architects avoid static methods #threading

Hi XR,

Not sure why my architect doesn’t like static methods. I have some stateless method. It takes an object, pass to a static JNI method, then gets an output object, and returns it. I created a simple wrapper class around this single method. I feel making the method non-static requires callers to instantiate the class first. It would give the illusion that there’s state in the instance. It can also lead to unnecessary object proliferation.

My colleague mentioned threading as a reason. He’s not a newbie on threading, but I feel a stateless method is safest when it’s static. It is possible that he worries that this static method may one day access some static field? In that case, making it non-static and requiring every caller to have its own instance can help a bit.

Another colleague (elsewhere) also felt static methods are bad in threading. He’s also experienced in threading.

In threading, My habit is to keep things simple. If a method is stateless, i keep it static and have zero static field in that class. When people look at the code they immediately see it’s stateless, just like Math.abs().

static methods perfect; static fields dangerous

Hi XR,

Over the last 5 projects, I am moving into a new design direction — use static (rather than non-static) methods whenever possible, while keeping mutable static fields to a minimum.

A digression first — a note on local “auto” variables as a superior alternative to instance fields. Instance fields represent object state, which is often shared. System complexity (as measured by testing effort) is proportional to the number of stateful variables — mostly instance fields + some static fields. In contrast, Local variables including some function parameters [3] are temporary, on the stack, far more thread-friendly, and often stateless except temporary state. The best ones are scratch pad variables that become unreachable soon, so they don’t pollute, don’t linger, and has no side effects.

[3] If an object passed in as argument has a large “scope” somewhere else, then it doesn’t enjoy the simplicity of local variables.

Similar to local vars, the best static methods are stateless and thread-friendly — these methods don’t change object state i.e. instance fields or static fields. For example, most static debugger statements are stateless; most type converters are stateless; factory methods are often stateless as they create new objects; most math functions are static; many string utils are static

Static methods are often simpler. If you were to implement the same logic with instance methods, they often end up entangled in a web of dependencies.

I can easily move a static method between classes. Perfect lego blocks.

Object dependency is a major part of system complexity. Look at dependency injection frameworks. Java Swing is complex partly for this — Compared to the average non-swing app, I feel the average swing app has more objects and more inter-dependencies.

If you design objects to represent business domain entities (like accounts, requests, patients, shipments, houses..), then instance methods represent behavior of objects; static methods represent … utilities

When I look at a non-trivial OO system, and estimate the percentage of code in instance methods vs static methods, I find invariably instance methods outnumber static methods by at least 3 times (75% vs 25%). Habitually, I refactor them into static methods whenever feasible. Time well spent, as system invariably gets simpler and cleaner.

C# elevates static methods pattern into a language feature — static class

[[the art of readable code]] P98 says — to restrict access to class members, make as many methods static as possible. This let reader know “these lines of code are isolated from those variables”

Any experience to share?

some methods should always/never run on EDT

If a method *might* run on EDT, then don’t invokeAndWait() on it. You can invokeLater() on it though. See [[java threads]]
If a method should always/never run on EDT, then those methods should call this assertion method below to make an assertion.
      public static boolean assertIsEDT() {
            boolean isEDT = javax.swing.SwingUtilities.isEventDispatchThread();
            if (!isEDT) {
                  // Always create new Exceptions as local variables, never fields
                  AssertionError e = new AssertionError(“One of the methods in this call stack is “ +
                              “found running on a wrong thread. This method is designed as EDT-only.”);
                  e.printStackTrace();
                  throw e;
            }
            return isEDT;
      }
      public static boolean assertNotEDT() {
            boolean isEDT = javax.swing.SwingUtilities.isEventDispatchThread();
            if (isEDT) {
                  AssertionError e = new AssertionError(“One of the methods in this call stack is “ +
                              “found running on the EDT. This method is designed as a never-EDT method.”);
                  e.printStackTrace();
                  throw e;

            }
            return isEDT;
      }