[Tech] Event Driven Architecture Standards

Event Driven Architectures (EDAs) are a important and rather recent trend in enterprise application design. However there is apparently still a lack on common standards in this field. It seems, that this discussion is just starting to move on.

Rober Covington created a compilation of current EDA and ESP related standards available on

http://soa.omg.org/SOA-docs/EDA-Standards.htm.

While reading through the standards my suspicion that there is a lack of a clear separation of events is still missing and leads to confusions. People tend to mix up events in sense of user interface events and business events that are in scope of esp and cep processing engines.

[Pub] Our new Baby is online :-) (actually in beta)

After some work and some delays, we are proud to finally present a "brother/sister" project to this Blog, the "Best-Practice-Software-Engineering" Samples and Documentation project: please check out this link:

http://best-practice-software-engineering.ifs.tuwien.ac.at/

This project will support our teaching in various courses, but should also be a foundation for other "educational" efforts. We present on that page a set of "best-practices" examples plus documentation. At the moment we have two stable examples: one is a "basic" example using a Java Swing Frontend and a Database backend plus logging, Springframework, Unit Testing, Maven builds ... The second example uses the same model and business logic, but provides a JSF frontend and Hibernate for O/R mapping. The third example (still at work) shows a distributed solution including webservices.

Besides these examples we provide information to used software patterns inlcluding interface, delegation, data access object, dependency injection and so on, plus an introduction and motivation for the used technologies.

This whole project is published under Apache license, hence can be used and extended by everyone. Currently we are running the beta-phase also with our SE course in the summer semester, and we hope for feedback and suggestions to enhace the samples and/or the documentation. It will also be updated rather frequently, so please stay tuned!

[Pub] AOSSS Conference

Two weeks ago I had two talks at the 8th Asian Open Source Software Symposium. It was quite interesting for me to watch this conference, as it showed a significant trend for Asian countries (not only Indonesia) to try to follow a strong Open Source policy. Interesting was the statement of a representative of a big American GIS company with strong Microsoft background: they ran into significant prob lems in Japan ("they are not talking much about open source, they are doing it"): e.g., it appears to be government policy in Japan, that Windows servers are not allowed behind firewalls. Hence they had big problems in installing their GIS backend-server, as this one was only running on Windows, and they discussed a lot of complicated workarounds. Meanwhile, they have an open source system that also runs on platforms like Linux.

However, this was just one impression, but a symptomatic one. I personally spoke about the ECDL/ICDL as an option to bring basic IT education (also in the OS environment) to a country like Indonesia and in my second talk about the connectio of the OS development process and modern economic strategies. This leads from the assumption, that Software becomes more and more a commodity towards new software production and "consumption" patterns with a new role that users play in the game.

Maybe check out my presentation and give some feedback...

[Pub] Webservices with Axis and xFire

In the current print-issue of Infoweek.ch Markus Demolsky and I wrote an article about Apache Axis 2 and Codehaus xFire. This article is also online available (in German).

Both frameworks can be used to build modern Web-Service driven architecture, however choosing the right one for the projet is not always an easy decision:

Apache Axis

Apache Axis has a long history. The first Webservice project in the Apache pool was Apache SOAP donated from Lotus. This was more a proof of concept than a framework for operational use (as I realised myself by suffering incredible pains during a project). However the next step on the road was Apache Axis 1. Both projects are still available on the Apache websites. However, SOAP should definitly not be used for projects, and also Apache Axis 1 is actually deprecated.

The current installment is Apache Axis 2. It is again a redesign of Apache 1 and is up-to-date with modern standards like:

• SOAP 1.1 and 1.2
• REST style services
• WS-reliable messaging
• WS-Coordination and WS-AtomicTransaction
• WS-Security
• WS-Addressing
• basic message exchange patterns in WSDL 2.0
  

and better performance.Under the hood the new object model Axiom (based on a pull parser) promises less memory consumption and better performance. Additionally improved handling of message-style interactions (Axis 1 SOAP RPC) MEPs that include synchronous, asynchronous messaging are added.

A difficult issues is data-binding in document-style webservices. Generally spoken, the problem is to map object hierarchies (from Java, .net, ...) to xml data-structures. As there are specialised frameworks for data-binding (due to the complexity of the problem) Axis 2 supports pluggable data-binding. There is the option to use ADB, the Axis data-binding, but also XmlBeans, JiBX, JAXME and JAXBRI.

An interesting additional option of Axis 2 is the REST support! Services can be provided in "SOAP Webservice" style as well as in REST.

Codehaus xFire

The "Next Generation SOAP framework" is currently available in version 1.2.4 and provides an easy to use SOAP framework. It is a servlet based application and uses the new StAX parser to ensure high performance. As Axis xFire supports a lot of current Web Service standards.

In order to expose components as web service, developers can choose between the xFire API, XML configuration or the new Web Service Annotations. A unique feature of xFire is the support of Web Service annotations. Developers can annotate their service components with predefined web service annotations which are then processed by xFire to create the WSDL.

As Axis, xFire provides a pluggable data binding architecture. There is the option to use Aegis, the xFire data binding, but also XMLBeans, Castor, JiBX, JAXB. Transaction management, Security, logging and loose coupling are important tasks when developing enterprise applications. Therefore current software systems are built on top of containers, like Spring or Plexus. The team of xFire had detected the importance of containers in software systems and therefore provide a wide range of container integrations (Spring, Plexus, Loom, Pico or Yan).

Fazit

Even with modern Web-Services frameworks writing reliable and portable (non-trivial) webservices is not necessarily an easy task. Even if IDEs suggest that writing a webservice is "just a click" in the wizard, this is actually not the case. A lot of configuration options are available, data-binding is a complex issue and so is interface specification with WSDL 2.

Documentation of both frameworks is ok, but not good. At least not sufficient to learn webservices. To be able to develop robust webservices it is highly recommended to get a good knowledge about the according standards like SOAP, WSDL (the needed WS-* standards) and the concept of data-binding and servlets and probably also of Java annotations. If this knowledge is available, the documentation of the projects should be sufficient.

Read more...

... in our Infoweek article.

Alexander Schatten, Markus Demolsky

[Tech] Software Reusability

A Myth or Reality?

Several days ago a topic with the title "Software Reusability: Myth Or Reality?" was posted on Artima. Following the comments on this post since the first publishing I think you should take a look there too. Developing SW for 6 Years now (by far not that long as the poster) I have to agree with him. The problem with reusability is that there are several issues that must be fulfilled that libraries objects .... can be reused.

1) Clean self speaking code

This is one of the biggest challenges in SW projects. You need seasoned developers with a passion for SW development and time to write code in that way. Today commercial projects first off all don't have time to do this. Time is a big factor in this game. On the other hand teams in companies have developers with different levels of expert knowledge and it's very difficult to teach "junior" developers until normal working time.

2) Documentation

From my point of view, documentation is important but not as important as the first point I mentioned. We have to differ between 3 kinds of documentation:

a) Tests: they are the best documentation of a program because their actual status (which is documentation) can be verified easily by executing them.

b) Code Documentation: all the xDoc stuff developers write over methods classes and even in code to explain the behavior of the code they have written. The problem with it is that they are getting useless as far as they are not updated with code changes (which is often the case). In difference to Tests they can be just verified if someone reads them and expects something that is not fulfilled.

c) Diagrams: all the UML Stuff which in my opinion is even more difficult to hold up to date.

Answers?

I would answer the question raised in the article as follows:

1) Is reusability something that can be trained for?

I don't think that schools/universities can train developers to program with reusability in mind. Only persons that had to use libraries or old implementation in a project now how it feels not being able to use 1000LOC that where written for a similar domain and can't be reused for the new project. This feeling makes most of them think about possibilities to write code in a other way.

2) Should reusability be a requirement?

It depends on the project. There are projects where it definitely should be a requirement. The interesting question here is how we will measure it? I would say there is no metric that can describe how reusable code is - except for the first two points tests and code documentation I mentioned above. In my opinion it is not enough to say we have 89,9% test coverage and 70% in code documentation. These two values just say nothing about code reusability.

[Event] Software Engineering for Everday Business - Review of the Last Event

Today we had the last event of the series "Software Engineering for everyday business" in coopertion with the chamber of commerce (Vienna) and the Austrian computer society OCG:

Dietmar Winkler introduced the day defining the term quality and giving some introduction about quality factors and quality management, quantification of quality and measurement issues. He continued describing quality assurance strategies on the example of V-Model (XT), Rational Unified Process and SCRUM. Then the process enhancement cycle following PDCA (plan, do, check, act) was described.

In the next presentation, Dietmar explains strategies to review and audit Software artifacts. He explains different review strategies and reading-techniques (explaining typical checklist-based approaches) and the planning of Software Reviews. He recommends the usage-based reading (UBR), that is priorised according to business value, as best-practice inspection technique following studies that have been performed at our institute.

The second speaker, Denis Frast, introduces the term testing and discusses the cost and efficiency of testing. Validation and verification are discussed in the context of the V-model. Fundamental test-strategies are described:

• "Private" tests
• Module test
• Integration test strategies (increment, Big-Bang...)
• System-test

Integration tests can be leveraged by modern component-based software engineering strategies using containers like Spring, as these technologies allow to flexible change the concrete binding between components and objects. This is important to be able to rewire systems according e.g., to use mock/stub components for testing and integrating the system in various test or production environments.

Also the psychology of testing is important to acknowledge. It suggests to implement own test-teams that are only focusing on testing and are not developers of this system (module). Developers themselves sometimes unconciously try to proof their program right (show that it works) and not necessarily try to push it to or over the limits. This is understandable, as a programmer is successful when he makes no mistakes, whereas a tester is assumed successful when he finds errors.

We can eventually identify two general strategies to derive test-cases: black-box and white-box methods. Black-box tests are more derived from specifications, whereas white-box tests are based on code-structure.

Finally Alexander Schatten gave a brief overview on modern tools to automate project-tests:

• Unit Tests
• UITests
• Codestyle Checks
• Profiling

He particularly focused on the capabilities of tools like checkstyle. This tool offers a great variety in options to check sourcecode. Starting from coding conventions, over metrics to code-duplications. This tool will be probably described in an special blog article soon. Eventually he lined out the features modern profiling tools like the Eclipse TPTP offers. They allow to solve difficult issues in complex software systems like tracking down performance and memory problems or detect other runtime issues.

Please download all presentations from the Event-Website (partly in german).

[Tech]EAExpression - building a new DSL

Today everybody speaks about DSL's (Domain Specific Languages) and how they can help solving problems in a specific domain better and easier than general purpose languages.

Senactive is developing a Sense and Respond System (InTime) which uses runtime objects - so called events - to send information from one component to another. For more information on the system see our website. The problem we had was, how we can describe criteria like filters or rules on runtime objects within design time that can be easily described from users and evaluated within the runtime. Our fist approach was using XPath, because C# has a very flexible XPath engine where you can implement the navigator on your own objects (maybe I will post about this another time or Rupert as he did it :-)). We implemented the navigator some time ago and the events can be navigated using XPath expression. The power the language gave us was really greate (functions, adressing ...) but the tradeoff was, that the language is not really intuitive for business users.

So we decided to implement a new language called EAExpression (Event Access Expression) Language. The language should be easier to understand for business users but should also have a similar power XPath gave us. So we decided to implement a DSL and after some brainstorming we came up with the following key points our language must provide:

• addressing events - this includes events itself and their attributes which can be primitive types, collection, dictionaries, other events ...
  we use a "." notation for this e.g. Event1.Attr1.Attr2 can be used to address the Attribute "Attr1" of Event1 which in this case again is a Event where the Attribute "Attr2" is evaluated. For Collections we use the Syntax Attr1[1] and for Dictionaries Attr1["Test"]
  
• Constant values (e.g. Strings "Test", integer 12, float 12.5f, boolean true|false, ...)
  
• calculation at least we need to calculate +,-,+,/ and % (modulo)
• boolean expressions AND, OR, XOR, NOT
• comparison expressions =, if possible chained
  

After we knew what we wanted to support, we were looking for a flexible and easy to use Lexer and Parser and came up with ANTLR. It is really a very powerful tool with greate amount of helpful documentation that is needed if you develop a new language the first time. ANTLR is a Java tool which can generate code for the Lexer, Parser and TreeParser to Java, C#, C++ and Python.

Starting to play with it felt like sitting in a compiler course at the university :-) . I really didn't believe i would need this stuff once again - I always said "compilers are just for geeks" but i changed mind as i got deeper into the stuff again. I took us about one and a half week to learn the stuff needed and build up the lexer and parser. But if I need to do it again I would calculate a maximum of two days for a related grammar.

After the expression is evaluated, there is an AST (Abstract Syntax Tree) of your language code which can be easily navigated with ANTLR buy implementing a TreeParser. The TreeParser can now be used to generate code that should be executed in the runtime. This was the hardest part within building our language because of missing support in the C# API to evaluate the numeric operations for the datatypes in a generic way. We were looking for a Expression library that could support us but in the end we had to do it on our own. We used Wrapper classes for the datatypes in order to evaluate the calculations type safe, an other solution would be to use reflection.

In the end we build a wrapper classes for the whole Language called EAExpression like the XPathExpression Object in C#.
Now you can do things like this:

----Code---
Event1 ev1 = new Event();
ev1["Attr1"] = 12;
ev1["Attr2"] = 15;
ev1["Attr3"] = 1;


EAExpression expr = EAExpression.Compile("Attr1 < Attr2 + Attr3");
bool val = (bool)expr.Evaluate(ev1);

----Code----

This is just a very simple example where the first 4 lines shows how an event is created.

We have learned several things while building the language:

• don't be afraid to create one for a specific purpose - some times it is really useful to do it
  
• it was much less work than I expected, because there are several tools out there that can help you
  
• the language you build should be as simple and easy as possible. Don't try to do fancy stuff or allow several ways to do the same thing. This will be more confusing than helpful to users.
• in the end we needed to introduce defined functions e.g. Now() for the DateTime. Now in C# which was a little bit tricky
  

Things we don't have until now:

• Autocompletition and syntax highlighting for user inputs within the GUI - I will post as soon as we have it and how we will solve it because i think it is a very important part of languages for business folks.
  

[Event] Software Engineering for Everyday Business

On thursday Jan, 18 2007 the final event of the lecture series "Software Engineering for Everyday Business" in cooperation with the chamber of commerce Vienna and the Austrian computer society takes place at Museumsquartier, Architekturzentrum.

In this final installment Dietmar Winkler, Denis Frast (and I) will discuss methods of quality assurance in software projects. Topics will be (among others):

• Enhancements of Softwareproducts and -processes
• Reviews and inspection
• Methofs of software testing

The lecture will be held in German, the slides will be mainly in English and will be available at the lecture page as usual.

[Tech] Generate the Persistence Layer for iBatis from the Database Schema

As I am writing some articles about iBatis these days, I came across the iBatis "Subproject", or "Tool" Abator.

I think it was obvious from my recent posts, that I have quite some sympathies for the iBatis project. However, writing the persistence part with iBatis (and also other persistence framworks) still means a lot of typing. Typically you have to create the following artifacts:

1. Create database schema and build a database from it
2. Write the SQLMaps for each table, including all SQL statements, i.e. typically inserts, updates, selects and deletes
3. Write the domain models (transfer objects)
4. Write the data access objects following the DAO pattern typically using the Spring framework's iBatis templates.
5. Write the unit tests for each data access object

So counting this means you have to write four artifacts (assuming the database is existing) for each table you want to access. This is obviously awkward. So, how can Abator help you: Actually Abator works like so:

1. You create an XML config file (rather simple) containing the information about the databas (URL, JDBC driver and the like)
2. Fine tune the code generation setup, e.g., what type of DAOs should be generated: iBatis style (deprecated) or Spring DAOs.
   
3. Add the tables you want to access into the config file
4. Make some optional configuration steps (like, should the pojo be named different to the table, should some fields in a table be omitted, should they by named differently in the POJO, ...)
5. Define the target directories
   
6. Start Abator

Then Abator creates the SQLMaps, the model beans, the DAO in the desired technology (so you get steps 2-4 in the upper artifact-list)! Now, the question arises, if this makes sense, as you also have to write the Abator definition XML and learn to use the tool. Moreover this is a "one time" thing: the generation step will be done only once and then you will most probably modify the generated classes and SQLMaps to better fit your needs.

I personally think I will make sense in many cases, particularly when the data base consists of many tables. Writing the Abator config is rather simple and it generates all the artifacts of n-classes within one step. These generated artifacts are often a robust point to start from, much faster then writing everything from scratch.

Additionally it very easy to understand Abator and allows "newbies" to get a set of SQLMaps, objects and DAOs plus one example class for each table showing how to use the DAOs. This helps to get a quick an easy introduction into the iBatis concepts.

Remark: Please read the design philosophy part of the Abator documentation. It is clearly mentioned that this strategy is a database-model driven strategy. If this is fitting for the project it seems to be great, if the project or developer focus is more a object-model driven one (which is seldom the case in enterprise projects) this approach might not be fitting.

[Tech] Best-Practices for Spring Configurations

I've found an interesting article about Best-Practices for Spring configuration. More and more Java applications are based on the Spring framework. Over the time, software engineers develop their own best practices, but hardly anyone publishes them. Here are some suggestions described in the article:

• Avoid using autowiring
• Use naming conventions
• Use shortcut forms
• Prefer type over index for constructor argument matching
• Reuse bean definitions, if possible
• Prefer assembling bean definitions through ApplicationContext over imports
• Use ids as bean identifiers
• Use dependency-check at the development phase
• Add a header comment to each configuration file
• Communicate with team members for changes
• Prefer setter injection over constructor injection
• Do not abuse depencency injection

Certainly some software companies and developers have their own best practices, but the above mentioned one are general and should be used in every Spring based project.

Maybe our readers want to contribute their practices?