[Arch] EDA and SOA in a complex world

In the recent iX magazine (German) Josef and I wrote an article about event-driven architectures (EDA) and the connection to service oriented architectures (SOA).

An idea I personally like in the context of EDA is "Seed, Sense, Amplify" that was introduced in the book "It's Alive, The Coming Convergence of Information, Biology and Business" from Christopher Meyer and Stan Davis. In my understanding the core idea there is, that in a complex world, it becomes more and more difficult to plan ahead and to predict what customers want or more generally spoken, which ideas might work out and which will not. So Meyer and Davis describe strategies, where "natural" variation is brought into business concepts, where offers are slightly changed for different customers, where customers become more freedom in doing what they want (seed).

The idea is to be more agile, but not laissez-faire, so beside the "seeding" step also a properly planned "sensing" step is required. In this step the actual behaviour of the customers is analysed. In the last step: "amplify" the desired behaviour is amplified (and the not desired one diminished.

Now to allow such strategies also the underlying IT architecture has to support this degree of flexibility yet allow the implementation of the sense and amplify steps. "Traditional" heavy weight data management and process management solutions often prove to be too inflexible to deal with such agile concepts alone. Help is needed and as state changes in different IT systems are often technically expressed in events/messages that are sent over message buses they can be processed with respective (complex) event processing middleware. And this is where our research and in that case our article comes into play.

Eventually following the idea of EDA means building a system that is not only analysing events, but that is driven by events (as we also try to explain in the article) and can lead to a sense and respond infrastructure (Roland give some ideas about our strategies to that respect in his diploma thesis). I'd like to illustrate the agility EDA might provide in certain situations on an example we also used in the article:

Let's consider a webshop, where the user orders goods using the web-interface (and the application server). In a "conventional" architecture, even a SOA a controler component (probably in the appserver) might "orchestrate" the order process by calling several services like the credit check, the customer check, the stock-keeping application, eventually initiating the shipping and following up the payment procedure. When implemented in a SOA way the components are de-coupled at least to a certain degree. Still, the whole system is kept together by the controler compoent. Now, how could we model this "EDA-like":

• First we should use a message oriented middleware (MOM), that supports the publish/subscribe paradigm (e.g., ActiveMQ, Tibco and the like)
• The next important concept is to omit the controler component by inverting responsibilities, meaning: not the controler decides what service should be invoked, but the components themselves decide what they are interested in (which requires a clear strategy of defining message types)
  
• If the customer orders goods, the web-application server sends a "order" message to the MOM. It does not "address" specific other components, just announces that a new order has arrived
  
• Now arbitrary components can register for new incoming orders as they need that data to do their job like the credit check component and the stock-keeping component. Both might do their checks and then send new messages into the system like "credit checked" or "stock checked" including a reference to the order message like the order-id
• Eventually the order processing component might wait for all messages that are required for the next step like order, credit-check and stock-check, then initiate the shipping and send the next message that shipping has been done and so on.

I don't want to elaborate that in detail, but it can be easily seen, that the responsibility has been inverted, which leads to a more decoupled architecture as every component can be easily exchanged or "multiplied" in case of performance problems as no sender actually sends to a specific receiver.

Additionally this style of architecture allows a significant level of agility (to come back to the inital scope of the posting) as new components can be easily introduced into the system (e.g., reporting, statistical analysis, components to connect to contractors to initiate reorders and the like) without the need to continuously rewrite the controler component that then gets more and more complex and intransparent.

O.k. enough for now, furter interesting sources might be the mentioned book, Brenda Michelsons blog and Jack von Hoofs blog. Our article can be found in the recent iX magazine July 2007.