[Misc] Open Protocol vs. Twitter: 1:0 ?

In a current ZDNet Blog Posting, Sam Diaz analyses the recent technical issues Twitter has (again). Twitter is growing dramatically in the last months and apparently the Twitter backbone is increasingly in trouble. The same happened already about a year ago.

The analysis of Sam Diaz is of course correct, but in my opinion he still completely misses the point in discussing technical issues why Twitter might or might not catch up with the upcoming demand in the service. The point actually is, that the communication concept of Twitter is appealing to many people, which is good, but in the history of the Internet it was never a good idea to rely on a proprietary protocol in any important communication channel.

So the real question is a much more generic and actually should be: how can we get rid of Twitter as fast as possible and replace it with an open protocol and a scalable distributed architecture, comparable to Email, XMPP chat and the like. There are good reasons why proptietary protocols largly failed on global communication systems like the Internet; those that are still around are a continuous pain in the ...

I confess, I am using Twitter as well, but it is of course a lock-in situation. If you want to follow the interesting stuff, you currently have to use Twitter. However, now we still have time to replace Twitter with something like Laconica (identi.ca) or anything similar down the road. Even better, Twitter might open up it's system and try scaling it that way. However, now is the time to act: Twitter is still a toy, but it is on the way to become a serious communication system we might depend upon in some years. And I believe, no one wants to depend on a communication system that is proprietary and unreliable at the same time.

[Process] Successful Distributed Agile Development

Andy Singleton from Assembla wrote a nice blog entry about success-factors in distributed agile development. He focusses on 6 factors:

1. Fix-schedules for releases
2. Continuous Build
3. Ticketing
4. Daily Report and Chat
5. Team-Activity Streams
6. Recruitement

I am missing automated testing and QS though, which I believe is a cornerstone in distributed development, particularly in combination with a continuous build/integration setup. Read the full article here!

[Tech] Mavenizing AppEngine!

As I nagged yesterday about the fact, that AppEngine has no proper Maven-build system, already today the guys at Sonatype reacted ;-)

They describe preliminary attempts in how to "Mavenize" AppEngine projects; hope they will be able to fix also the last issues!

[Tech] Google AppEngine (and Java)

AppEngine is a rather recent new service from Google. It is probably Google's answer to Amazon's cloud-computing platform, yet targets a very different market. Where Amazon offers a a broad range of services and high flexibility (with the disadvantage of higher administration effort) Google targets web-developers that want to publish web-applications. AppEngine started with a Python environment, now since some days the long anticipated Java-version (Java 5 and 6) is online. Now what are the benefits of using AppEngine?

Java

First of all, it is possible to deploy applications without having to install, administrate and maintain an own server (instance). Google provides a runtime environment (sandbox) into which Python or Java applications can be deployed. Access to these applications is (for clients) only possible via http(s). So this is a feasible approach for web-applications or RESTful services.

An additional advantage is, that Google deals with scaling issues, i.e. it scales the applications dynamically to the demand. This is a significant advantage for startups, that have no clear idea about the number of customers they are going to have or how fast this number is growing. For the scaling to work, though, some restrictions have to be considered. Most notably this concerns the persistence strategy. E.g. applications (and libraries!) are allowed to read files from the filesystem, but are not allowed to write. For all persistence issues, the Google datastore has to be used. However, what is nice with the new Java-sandbox is the fact, that Google apparently tries to follow established standards. For persistence Java developers can use JDO or JPA or a low-level interface for the distributed datastore.

I wonder, however, how logging can be handled in that environment. Logging is usually done to a file or to a JDBC datasource. A JDO logging target I have not seen before; ideas anyone?

Generally spoken, arbitrary Java libraries can be deployed and used in the AppEngine as long as they do not violate the AppEngine sandbox. Also due to the scaling-approach not all libs/frameworks will run unchanged. As yet, it seems not quite clear for example, which Java Web-Frameworks will run seamlessly in the App-Engine. Googles webtoolkit (GWT) should work, other framework communities are currently testing their frameworks for compatibility, e.g. in the Apache Tapestry and the JSF framework Apache myFaces discussions are running on the mailinglists.

Build Automation and Development Process

The development process is, in my point of view, as also with other Google environments like GWT a mixed blessing. Everything is Eclipse centered, which is not really a good thing: Google provides an Eclipse plugin for the AppEngine including a runtime environment for testing applications offline. This is great for daily development activity, but not for a stable build- and testing environment. Unfortunately Maven support (like archetypes) are completly missing at the moment. Google is apparently pretty hostile towards Maven and focuses mostly on IDE integration, which is definitly not a sound way for a modern build automation. IDE "wizard-based" SE approaches usually turn out to be unstable and problematic, particularly in team-projects. This might be nice for a fast hack, but is no basis for a larger project. It seems, that some support is given for Apache Ant though.

Hopefully other developers will provide a Maven integration for the Java AppEngine. With the current approach not even an IDE-less deployment is possible.

Conclusion

So, despite of the build-issues, I believe that the AppEngine is a great option to deploy web-applications in Java or Python. For small applications (small in the sense of "low web-traffic"), the AppEngine is free, after exceeding certain thresholds (CPU, storage, bandwith...) one pays according to the resources needed. Google provides a web-interface to set daily financial limits for individual resources. E.g. one wants to spend a maximum of 5 $ a day for CPU time and so on.

Looking forward to the first experience reports, particularly with web-frameworks like Wicket, Tapestry or Cocoon.

[Misc] Operating Systems for Netbooks

Netbooks are different from notebooks and desktops; they are used in other contexts, have smaller screens, not so powerful hardware, less diskspace and so on. Windows is hardly the best system for regular desktops and notebooks (in my opinion) and definitly only a temporary solution for the netbook market. Microsoft even had to reanimate Windows XP for that purpose. In effect, Windows is to fat, too insecure and not optimised for small screens and additionally hardware producers have to pay license fees to Microsoft in a very tight market, where some Euro can make a difference.

Since some months the companies like HP apparently are evaluating Google's Android for Netbooks. Now, I am all for Android; But on mobile phones. I am doubtful that Android is a good solution for netbooks though. Android was designed for the typical smaller scale mobile applications e.g. with only "one application" active on the screen. And resource management that focuses on one active application. Now, a Netbook might have a small screen as well, but from the usability point of view it is probably closer to the notebook than to the mobile phone.

Hence customers expect applications in a style they know them from their desktop, like Office or Internet applications (mail, browser, ...). Using Android reduces the number of suitable applications dramatically, and Android applications written for Android cell-phones will most likely not perform well on Netbook hardware.

Now, Linux was already used on several Netbooks like on the EEE-Series. Why not stick to Linux? Probably adapt an existing distribution like Ubuntu to better fit to that specific environment. Then immediately the whole range of destop applications are available including applications like Open Office.

From the software engineering aspect, I would believe, that it is better not to mix up mobile platforms with Netbooks, in the end either applications have a bad user experience on the specific platform they were not developed for (allthough both use the same OS).

But maybe I am proven wrong?

[Tech] HSQLDB Version 1.9 alpha is out

Finally hsqldb 1.9 (alpha, though) is released. This release was announced for, I believe, nearly one year. It seemed to me already that hsqldb is a rather dead projects. I am glad they made the next round, because in a way I still like that system a lot. Sure, Apache Derby is most likely the superior system, and H2 looks very promising too (but is still, as I understand it, a "one man show" without community) however hsqldb has some tiny details that make it very nice: First, it always had a really tiny footprint and was extremly easy to understand and use.

And I particularly liked the feature to fine-tune the memory management, i.e., should the data be stored on disk, purely on memory... and this on a per-table basis. Plus, with one simple command it is possible to write the whole database as SQL statements into a file from which it is also loaded again from that file. A feature that is e.g. missing in Derby. This often turned out handy during development-phase.

Now for version 1.9 they seem to have rewritten significant parts of the software and added an impressive list of new features. What I have to figure out is, if they have finally implemented proper transaction isolation. In my opinion this was (beside the single-threaded kernel) the biggest issue in the previous versions, where dirty read could not be avoided. I am a little bit confused with the announcement(s) now, because they wrote that they have rewritten the core, however, in a forum posting the developers announced, that transaction isolation is not handled in the new release 1.9 but is planned for 2.0. The news announcements on the sourceforge are a little bit confusing for me. Does anyone have a better idea about this issue?

However, good luck for the stabilisation-phase of the new release!

[Misc] Hello World

"Hello World" programs are well known since at least 1978 and often a starting point to get to understand the most basic issues of a new computer language.

Wolfram Roesler collected "Hello World" programs in 421 different computer languages!

Particularly interesting for me was "Hello World" in Chef :-)

[Arch] Cherish your Architecture

Since a few month I am following the world of architecture tools much more and there are interesting things on the way:

• hello2morrow has changed the evaluation mode for one of it's product: SonarJ (Community Edition) is now free for trial if you have less then 500 classes. I strongly recommend to use a tool like this at least for every new project start. Quality metrics and architecture definitions are now just a few clicks away and do really suppress 'the big ball of mud' every software engineer knows.
• Another nice product in the field of software quality measurement is sonar from codehaus. It really produces wonderful quality views for projects and I highly recommend to try it out. What disappointed me a little was that Java projects should be Maven alike and the two minute tutorial will not cause you to set sonar up in two minutes (installing, loading and producing metrics in SonarJ is much faster). But nevertheless the output is full of innovative ideas.
  

It was also interesting to learn that sonar has ideas to integrate something like the architecture rule checking framework macker. You can try this out for yourself and integrate it now into your Ant buildfile if you have a vision about your architecture. Do you have?

Here is an example of an architecture definition given on their website (think of everything being enclosed with XML brackets):

macker
..ruleset name="Simple example"
....access-rule
......deny
........from class="**Print*"
........to class="java.**"
......deny
....access-rule
..ruleset
macker


Do you have an idea of the intention?

Obviously a helpful feature but keep in mind that other tools (like the ones from hello2morrow) allow this definition much faster because you simply draw your architecture.

So check everything out and be aware of your architecture!

[Misc] Managing Commercial Software Projects

At IT-Conversations there is a recent interview with Jon Udell talking to Andy Singleton about "Managing Commercial Software Projects". This interview is highly recommended. Actually I figured that Singleton is following pretty much on the same track as I am. However he is making some bold statements; in projects he is using no phone-conferences ("the more phone conferences in a project the more problems the project is in") or VoIP/Video, no time-estimations are done when not explicitly demanded and he is following Open Source practices of distributed development using mostly asynchronous tool-chains.

He is apparently providing a set of development tools "best-practices" at assembla.com: One of the core concepts there is to assemble people around event-streams (of activities). I think, probably the main idea that I totally subscribe too is this: It is actually much more important to have awareness what others are doing than investing a lot of time into planning efforts.

However, listen to to the full interview, recommended!

[Misc] It becomes quiet around BPEL?

BPEL stands for Business Process Execution Language and will be used to execute business processes. But there are other standards which can also be used to execute your business process. What about XPDL - XML Process Definition Language? Nevertheless, many BPM vendors adopt their workflow engine to BPEL in order to survive on the BPM market. It's a mistake. Most of the products provide a BPEL engine as an additional modul, because most of the BPM/Workflow Engine products work successfully without using BPEL. At the same time as BPEL was pushed, the BPMN - Business Process Modeling Notation - was hot discussed, a notation to model business processes. When you look on the BPMN homepage you will find a BPMN to BPEL transformer, describing the mapping of BPMN elements to suitable BPEL elements.

I find an article with the topic "BPEL: Who needs it anayway?" written by Keith Swenson, discussing the usage of BPEL in the industry.

"There are a few vendors who promote BPEL as as the one-and-only-true-way to support BPM. In fact, it is good for some things, but fairly bad at a large number of other things. It is my experience that BPEL is promoted primarily by vendors who specialize in products we might rightly call “Enterprise Application Integration” (EAI). These companies have recently taking to calling their products “Business Process Management”. Potential users should be asking the question “Is BPEL appropriate for what I want to do.” In that aim, there should be a large number of articles discussing what BPEL is good for, and what it is not, but there are very few articles of this nature."

He also mentioned that BPEL supporter make the following assumptions:

• The people making the processes are programmers
• The activities in a process only need to send, receive or transform XML data
• Any standard will be better than no standard

What about the human integration in BPEL? The human integration is not supported in the standard, yet. You can use the WS-BPEL Extension for People, where each vendor implement this on his own way.

In the article he illustrates how to execute a BPMN diagramm directly using XPDL. The diagram is interpreted directly without converseion to another model.

To summarize, I see there are a lot of successfull SOA projects which do not use BPEL. It goes also without BPEL. However, there are scenarios where BPEL is very useful and makes sense, but I will mention, that BPEL is not the "All-Solution" standard. I am strained on the future of BPEL.