containers+MSA: cloud-affinity

I feel both the MSA architecture and the low level container technology can have a long-term impact thanks to cloud.

The implementations might be replaced by newer implementations in a few years. but some of the ideas may evolve. Now is still the volatile early phase… ##[17] proliferation → consolidation.. beware of churn

Both of them are easy to launch on elastic cloud

Both of them can be started on multiple machines as a wolf pack to handle bigger workload.


MSA seminar by Chris Richardson

One of the main (perhaps #1) justifications for MSA is devops including testability, phased roll-out, resilience …

Useful terminology — A particular microservice has a single service provider and some “consumers”.

REST is the simplest implementation, but async (with msg broker) was highly recommended by Chris.

Q: too many services, too many failure points?
A: async (with msg brokers) improves reliability and availability

— cloud and MSA

container is not the only way to host a service in the cloud. Chris mentioned “lambda” as the simplest alternative.

— each service can use a different programming language. Chris said this is controversial but can be useful. Say we are migrating from golang to node.js. We could implement one of five services in the new language and migrate in phases.

— “validation” in production environment #we don’t say “testing”

  1. deploy a new version of a service to production but has no traffic routed there
  2. expose the new service instance to the same production requests as the live service, but in readonly mode, so there’s no write to data store, no visible output except logging.
  3. Now run it in full mode
  4. route some fake traffic (like a test symbol in mvea) and validate the output
  5. route some real traffic in small quantity and rely on comprehensive service monitoring

— My questions + other audience questions
Q1: UDP as alternative to TCP?
A: UDP is used for low latency in microsec. Many MSA use cases are in millisec

Q2: latency … I would think monolithic is fastest
A: no simple yes/no answer
A: MSA definitely improves scalability. I would think the web2.0 shops do care about latency too.

Q3: distributed transaction/rollback …. I thought this technology is reliable for decades?
A: CAP .. choose between consistency and availability. I think Chris means
A: there’s a paper

Q4: c++…. is MSA unsuitable?
A: the prominent voices tend to be java, dotnet, node.js , golang, and python, but c++ is not unsuitable. Some key software projects are in c++ (I think for efficiency)

cloud4java developers – brief notes

Am an enterprise java developer, not a web developer. I feel PaaS is designed more for the web developer.

I agree with the general observation that IaaS doesn’t impact us significantly.

I feel SaaS doesn’t either. SaaS could offer devops (build/delivery) services for java developer teams.

PaaS has the biggest impact. We have to use the API /SDK provided by the PaaS vendor. Often no SQL DB. Can’t access a particular host’s file system. MOM is rarely provided.

mapReduce – rent cloud resources#IaaS

MR is popular on clouds — offered by AWS/Azure/Google.

I used to question the validity of on-demand/rent proposition. I felt during a peak period everyone is busy and in an off-peak period everyone has low volume. Now I think Hadoop is a nice example. 
It often runs for a short time but involves many compute nodes. This could be in a off-peak period. Without cloud, the Hadoop will have to make do with much fewer compute nodes.
Further, any batch job in general could be scheduled off-peak. 

cloud DB sharding – phrasebook#eg PaaS

I read quite a few books touching on sharding, but one book puts it succinctly into the big picture of noSQL, big data, web2.0 and cloud — [[cloud architecture patterns]]

PWM partition — The position database in PWM was partitioned into regions like Asia, Europe, Latam, …  This is an early example of sharding.
custom built — early sharding tends to be custom-built, like PWM. Nowadays standard sharding support is available in many databases, so no more custom-built.
noSQL and SQL databases — both support sharding
Autonomous — Shards do not depend on (or reference) each other.
Static data tables — are not sharded but replicated.