microservices arch #MSA #phrasebook

  • SOA — is the ancestor
  • communication protocol — lightweight, but more costly than in-process communication
    • http — is commonly used for communication. Presumably not asynchronous
    • messaging — metaphor is often used for communication. I doubt there’s any MOM of message queue.
  • modularity
  • in-process — services can be hosted in a single process, but less common
  • cloud-friendly
  • scalability — hopefully better
  • devops — is a driver
    • testability — each service is easy to test, but not integration test
    • loosely coupled — decentralized, autonomous dev teams
    • deployment — is ideally independent for each service, and continuous, but overall system deployment is complicated
Advertisements

RESTful – phrase book

agile – …

coupling – less coupling between client and server, so server side changes are easier. I think SOAP requires client rebuild.

b2b – still dominated by SOAP

resource-oriented services – …

object – each URL logically represents an object, which you can Get (query), POST (create), PUT (update its content) or DELETE

Q: is REST simpler than traditional SOA or SOAP web services?

Q: the concept of “resource” — does it sometimes mean a database record?

RESTful^SOAP web service, briefly

REST stands for Representational State Transfer, this basically means that each unique URL is a representation of some object. You can get the contents of that object using an HTTP GET, use a POST, PUT, or DELETE to modify the object (in practice most of the services use a POST for this).

— soap vs REST (most interviewers probably focus here) —
* REST has only GET POST PUT DELETE; soap uses custom methods “getAge()” etc
* SOAP takes more dev effort, despite it’s name
* SOAP dominates enterprise apps

http web service in async mode? #le2YH

I think in my dotnet client/server system, we use WCF duplex extensively, meaning the server can push updates to the clients. Each client must subscribe though. I think this is exactly event-driven as you said.

I remember that if I put a break point in the client’s updateReceived() callback method, it gets hit automatically, without the client polling the server.

WCF duplex is a standard and mature feature in microsoft WCF.

The server endpoint uses https, and I believe it’s a web service.

Is it possible that the server push is implemented actually by client poll under the hood? I don’t think so. There’s a polling duplex ..

See http://msdn.microsoft.com/en-us/library/cc645027(v=vs.95).aspx

barebones web server in WCF: no app.config needed

using System;
using System.IO;
using System.ServiceModel;
using System.ServiceModel.Web;
 
namespace StaticWebServer
{
    public class WebServerHost
    {
        public static void Main()
        {
            var host = new WebServiceHost(typeof(StaticContentService),new Uri("http://localhost:8000/"));
            host.AddServiceEndpoint(typeof(IStaticContentService), new WebHttpBinding(), "");
            host.AddServiceEndpoint(typeof(IFetcherService), 
                    new WebHttpBinding(){TransferMode = TransferMode.Streamed}, //to support large files like 70MB
                    "");
            host.Open();
            while (true) Console.ReadKey(true);
        }
    }
 
    [ServiceContract]
    public interface IStaticContentService
    {
        [OperationContract]
        Stream GetStaticContent(string content);
    }
 
    /// <summary>
    /// In total, exactly *2* files deployed to C:/temp --
    ///
    /// Since I copied my compiled EXE (#1) to C:/temp, I needed a dummy C:/temp/www/a.html (#2).
    /// My browser could then show http://localhost:8000/www/a.html 
    ///
    /// - no config file -- You can remove app.config completely.
    /// - no dll, 
    /// - no IIS, 
    /// - no windows service.
    ///
    /// This WCF service is hosted in a console host.
    /// </summary>
    class StaticContentService : IStaticContentService
    {
        [WebInvoke(Method = "GET", BodyStyle = WebMessageBodyStyle.Bare, UriTemplate = "/www/{*content}")]
        public Stream GetStaticContent(string content)
        {
            Console.WriteLine("GetStaticContent() " + content);
            var response = WebOperationContext.Current.OutgoingResponse;
            string path = "www/" + (string.IsNullOrEmpty(content) ? "a.html" : content);
            if (File.Exists(path))
            {
                response.ContentType = "text/html";
                response.StatusCode = System.Net.HttpStatusCode.OK;
                return File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
            }
            response.StatusCode = System.Net.HttpStatusCode.NotFound;
            return null;
        }
    }
}

SERVICE means … in WCF, SOA etc

The concept of Service is central to wcf, but the “service” word is overloaded and vague. Here are a few distinct meanings

* windows service – daemon process. I think a unix network daemon is similar. However many windows services and unix daemons aren’t exposed to the network

* web service

* a remote callable [1] thingy, like a RPC. Any remote callable would qualify as a service!
** I feel web service qualifies as a type of of rpc service.
** a network daemon qualifies too.
** I feel in wcf, “service” means something like this.

Remote means you don’t call it like calling ToString() or localtime() within the same process.

Callable means on-demand, online, like a 24-hour call center, ready to serve callers.

How is it implemented? A base form is a Function or Procedure, but frequently there are “service OBJECTs” involved. In fact, the service objects are in the official Model of a service;) but I actually feel these objects are unnecessary to the basic concept of service. Nevertheless, “Service”, WebService, and SOA are big buzzwords, and these people just love the OO jargon. I feel these jargon terms add confusion without adding value. Too much abstraction can lead to health problems;)

In WCF, there are many jargon phrases containing “service”, which is a central concept in WCF.
* service host — the Process hosting the service.
* service reference — just a fancy name for the proxy class.
* ServiceEndpoint

who generates the local proxy for the remote

In any distributed-OO infrastructure (ejb, rmi, wcf, remoting…), someone needs to generate a proxy class based on the server-side
API. If you aren’t paying attention, you will not notice where, when and how exactly this is generated. But this is a crucial detail
not to be missed.

There’s a bit of ambiguity over 2 related concepts — Instantiating a proxy instance (at run-time) vs Generating a proxy class
source code before compiling.

(I like to say “client-side proxy” but the adjective is superfluous. )

— WCF —
Usually i generate the “service reference” from a wsdl or the service endpoint URL.

SOAP serialization (to xml) – c#

Suppose a c# serializable class has 10 fields, and 3 of them are string fields. When we populate the 10 fields, we must say xxxSpecified=true on the 7 non-string fields, otherwise xml will not include them. If we don’t populate the 3 string fields and don’t set xxxSpecified on the other 7, then entire object is silently skipped. I spent many hours discovering this rule.

This is part of the standard c# serialization behaviour. I guess it’s standard for SOAP. http://open.convio.com/webservices/apidoc/concepts/visual_studio.html says

“To distinguish between null elements that should be specified as nil and elements that should simply be omitted, the standard XML serializer generated by the .NET framework adds a special flag to each non-String attribute. This flag must be set to true for the attribute to be serialized as part of a SOAP message”

Simple rule — if generated serializable class show xxxSpecified properties, then better use them.

a web service defines a schema + wsdl

For a given web service, I think there's exactly one “operation” (or “web method”) defined in the wsdl.

The input/output parameters of the operation are usually “umbrella” objects with a predefined “structure”, originating from the

basic C struct. As an alternative to a predefined structure, a Dictionary/Map of {string key -> anyType value} is extremely flexible

but somehow not popular. In any case, a schema is specified.

In a nutshell, a web service is specified and defined by

– a wsdl

– a bunch of data schemas

async web service – windows app ^ browser app

http://ondotnet.com/pub/a/dotnet/2005/08/01/async_webservices.html?page=3 mentioned that …

If the application needs the web service output in the later stages of the thread, then the WaitHandle is the best approach. For example, if the web service queries a database and retrieves a value needed in the parent process and then displays the final result on GUI, then WaitHandle should be used. We need the parent thread to block at a certain stage.

Most web apps come under this scenario. In all other scenarios, we can use callback. Mostly Windows apps use this approach.

In my mind this is the inherent difference between windows and web apps — web app has a dominant thread returning data to the browser while other threads are Unwanted or optional.

Windows apps (swing, winform, wpf etc) typically use a lot of threads beside the UI thread. Most tasks should avoid the UI thread, and keep the UI snappy.

SoapHttpClientProtocol – capture inbound xml

    /// Decorates the generated SoapHttpClientProtocol subclass with inbound snooper
///
/// Based on http://orbinary.com/blog/2010/01/getting-the-raw-soap-xml-sent-via-soaphttpclientprotocol/
public class ServiceDecoratorI : ServiceDecoratorO
{
private ExposedHttpWebResponse _lastResponseNonNull = new ExposedHttpWebResponse();
protected override WebResponse GetWebResponse(WebRequest request)
{
_lastResponseNonNull = new ExposedHttpWebResponse(request);
return _lastResponseNonNull;
}
public string InBoundXml
{
get
{
return XDocument.Parse(_lastResponseNonNull.Content).ToString();
}
}
}

class ExposedHttpWebResponse : WebResponse
{
private readonly HttpWebResponse _response;
public override Stream GetResponseStream()
{
var readData = new byte[1024];
int readBytes;

Stream contentStream = _response.GetResponseStream();
var result = new MemoryStream();

do
{
readBytes = contentStream.Read(readData, 0, 1024);
result.Write(readData, 0, readBytes);
} while (readBytes > 0);

contentStream.Close();
_response.Close();

var reader = new StreamReader(result);
result.Seek(0, SeekOrigin.Begin);
Content = reader.ReadToEnd();
result.Seek(0, SeekOrigin.Begin);
return result;
}
internal ExposedHttpWebResponse() { Content = ""; }
internal ExposedHttpWebResponse(WebRequest request) { _response = (HttpWebResponse)request.GetResponse(); }
public string Content { get; private set; }
public override long ContentLength { get { return _response.ContentLength; } }
public override WebHeaderCollection Headers { get { return _response.Headers; } }
public override string ContentType { get { return _response.ContentType; } }
public override Uri ResponseUri { get { return _response.ResponseUri; } }
}

SoapHttpClientProtocol – capture outbound xml

<![CDATA[ // Decorates the generated SoapHttpClientProtocol subclass with outbound snooper public class ServiceDecoratorO : MainRequestResponse__HTTP { private XmlWriterSpy _writer; public string OutBoundXml { get { return (_writer == null ? null : _writer.Xml); } } protected override XmlWriter GetWriterForMessage(SoapClientMessage message, int bufferSize) { _writer = new XmlWriterSpyE(base.GetWriterForMessage(message, bufferSize)); return _writer; } } ///

/// This class has special behavior when writing end-elements ///

class XmlWriterSpyE : XmlWriterSpy { public XmlWriterSpyE(XmlWriter implementation) : base(implementation) { } public override void WriteEndDocument() { _me.WriteEndDocument(); _bu.WriteEndDocument(); return; } public override void WriteEndElement() { _me.WriteEndElement(); _bu.WriteEndElement(); var xmlStr = _sw.ToString(); //if (xmlStr.Contains(BBGviaSOA.ticker.id)) // Log1.log(xmlStr); } } /// Wrapper class for XmlWriter that “spies” on the Xml being written /// and later makes that Xml available as a string to anyone who wants /// to inspect it. See also my blog description of this class: /// http://orbinary.com/blog/2010/01/getting-the-raw-soap-xml-sent-via-soaphttpclientprotocol/ public class XmlWriterSpy : XmlWriter { protected XmlWriter _me; protected XmlTextWriter _bu; protected StringWriter _sw; public XmlWriterSpy(XmlWriter implementation) { _me = implementation; _sw = new StringWriter(); _bu = new XmlTextWriter(_sw); _bu.Formatting = Formatting.Indented; } public override void Flush() { _me.Flush(); _bu.Flush(); _sw.Flush(); } public string Xml { get { return (_sw == null ? null : _sw.ToString()); } } public override void Close() { _me.Close(); _bu.Close(); } public override string LookupPrefix(string ns) { return _me.LookupPrefix(ns); } public override void WriteBase64(byte[] buffer, int index, int count) { _me.WriteBase64(buffer, index, count); _bu.WriteBase64(buffer, index, count); } public override void WriteCData(string text) { _me.WriteCData(text); _bu.WriteCData(text); } public override void WriteCharEntity(char ch) { _me.WriteCharEntity(ch); _bu.WriteCharEntity(ch); } public override void WriteChars(char[] buffer, int index, int count) { _me.WriteChars(buffer, index, count); _bu.WriteChars(buffer, index, count); } public override void WriteComment(string text) { _me.WriteComment(text); _bu.WriteComment(text); } public override void WriteDocType(string name, string pubid, string sysid, string subset) { _me.WriteDocType(name, pubid, sysid, subset); _bu.WriteDocType(name, pubid, sysid, subset); } public override void WriteEndAttribute() { _me.WriteEndAttribute(); _bu.WriteEndAttribute(); } public override void WriteEndDocument() { _me.WriteEndDocument(); _bu.WriteEndDocument(); } public override void WriteEndElement() { _me.WriteEndElement(); _bu.WriteEndElement(); } public override void WriteEntityRef(string name) { _me.WriteEntityRef(name); _bu.WriteEntityRef(name); } public override void WriteFullEndElement() { _me.WriteFullEndElement(); _bu.WriteFullEndElement(); } public override void WriteProcessingInstruction(string name, string text) { _me.WriteProcessingInstruction(name, text); _bu.WriteProcessingInstruction(name, text); } public override void WriteRaw(string data) { _me.WriteRaw(data); _bu.WriteRaw(data); } public override void WriteRaw(char[] buffer, int index, int count) { _me.WriteRaw(buffer, index, count); _bu.WriteRaw(buffer, index, count); } public override void WriteStartAttribute(string prefix, string localName, string ns) { _me.WriteStartAttribute(prefix, localName, ns); _bu.WriteStartAttribute(prefix, localName, ns); } public override void WriteStartDocument(bool standalone) { _me.WriteStartDocument(standalone); _bu.WriteStartDocument(standalone); } public override void WriteStartDocument() { _me.WriteStartDocument(); _bu.WriteStartDocument(); } public override void WriteStartElement(string prefix, string localName, string ns) { _me.WriteStartElement(prefix, localName, ns); _bu.WriteStartElement(prefix, localName, ns); } public override WriteState WriteState { get { return _me.WriteState; } } public override void WriteString(string text) { _me.WriteString(text); _bu.WriteString(text); } public override void WriteSurrogateCharEntity(char lowChar, char highChar) { _me.WriteSurrogateCharEntity(lowChar, highChar); _bu.WriteSurrogateCharEntity(lowChar, highChar); } public override void WriteWhitespace(string ws) { _me.WriteWhitespace(ws); _bu.WriteWhitespace(ws); } } ]]>

bridging over c# and java – solutions

Web service is a common solution in BoA, GS and Citi. MOM probably supports a larger volume (but a popular website supports large volume too.) MOM adds development complexity — C# need 2 threads

– 1 thread sends request
– another thread blocks for response. In Neo, response could take a while.
– this is typical request/response, probably using initiator’s mailbox and tmp queue

In Neo, Java can also server-push updates (triggered by benchmark price change), so one-way web service is insufficient.

web services j4, features — briefly

web service is an old technology (RPC) given a new lease of life 10 years ago.

* [#1 selling point] cross platform for eg between dotnet frontend and java backend
* loosely coupled
* good for external partner integration. Must be up all the time.
* beats MOM when immediate response is required.
* web service (soap) over MOM? should be feasible. One listener thread for the entire client system — efficiency

web service in Lead management prj

Easy to make a method a web service, easy to write a web service client.

Tomcat + axis can implement a web service. Websphere was used for the project.
Multiple clients applications need access to the same method on the server.

J4 web service wrt RMI? a dotnet client is planned.

SOA AR 4 q&&A

A: corba and dcom. seems like “dist obj” architecture”

A: Traditionally, connections between applications have been tightly coupled, as with CORBA (Common Object Request Broker Architecture). For example, flipping a wall switch to turn on a light is a tightly coupled system. However, making a telephone call to order a book is a loosely coupled system.

A: provider, consumer and a broker, just like JMS. Broker can also be a registry.
provider would PUBLISH its list of services to the registry

A: a Web service is little more than an SOA. However, an SOA doesn’t require the use of Web services as we understand them, and some types of Web services can be deployed without an SOA.

Simple SOA block diagram http://www.computerworld.com/computerworld/records/images/story/HowSOAWorks_large.gif

Q: 3 basic roles (processes) for an SOA interaction?

Q: relation to web corba?

Q: relation to web services?

Q: similiar ideas to SOA?