[Arch] Lazy O/R Mapping

As you might know already, I am not the biggest fan of O/R mapping tools, but I understand, that they can be of great help in many projects. However, a nice article discusses the dominant position of Hibernate, which is really a pity. Many developers these days apparently believe Hibernate is the only option out there. It is not. As this article points out there are other probably better solutions like Apache Cayenne (and of course my favorite iBatis, not O/R anyway).

The main reason for this blog entry is a reference from this article though, pointing to a Javalobby article that gives a nice introduction into the lazy loading problem (for all O/R rookies, highly recommended) and problems that are assoziated with Hibernate there.

[Misc] Global Warming and the SE Power of Ten

Ok, I confess, I did not find a better title for this blog entry. Now the question is, where is the connection between global warming and software engineering. Actually, this is exactly one of the things I try to find out these days.

We all face a severe challenge. Global warming is real, and so is the fact, that resources, particularly also energy resources are limited. IT is often seen as a "cleaner" way to do things; supposedly doing things in the virtual world is less resource intensive than doing them in the real world. E.g. if four people have a Skype meeting this should be less resource intensive than having three people traveling by plane to the meeting location.

Yet the IT industry, and now I come to the point, particularly also the software industry was not so particularly interested in efficiency in the last decade. We develop software that somewhat runs on the current hardware, because with the next generation of hardware it will be fine. This is actually embarrassing. Consider software engineering practices: there is a lot of talk about clustering, putting more iron to the backend if the application is slow, but who is really skilled in analysing an application, figuring out where the hot spots are, and optimizing those? Don't worry, just start a new server, that will do it.

Now the consequence is, that power consumption by IT (servers) increased dramatically over the last years, ars technica writes, that US servers meanwhile consume more power than color tv nationwide and the energy consumption is doubling every 5 years (!). Meanwhile even companies like Google realized that facts and initiate research in the field of renewable energies.

Some other examples struck me recently. I compared three recent game consoles: Playstation 3 and XBox 360 consume approx 200 Watts during playing (some sites even quote numbers up to 300). The Nintendo Wii approximately 20 Watts. So this is a factor of roughly calculated 1:10. Playstation 2 takes approx 50 Watts, Gamecube approx. 20 Watts.

Second example: the XO Laptop from the OLPC projects consumes about 2 W during regular work, a conventional Laptop about 10-45 W, again we have a factor roughly of 1:10, maybe more.

Now, it is clear, that the Nintendo Wii is not as powerful as a Playstation 3 and the XO laptop is not as powerful as a Macbook Pro. Yet, is the difference 1:10? That is the question. I was playing a rather new Playstation 2 game and was astonished how the quality of the graphics can still increase compared with games 5 years ago. The same observation as with the old C64. Consider the quality of the games in the early 80s to the later ones. Clearly, developers learn how to operate the device and got over time the best out of the box; and it can be astonishing what is in these devices.

The point I want to make is this: Up to now energy consumption was hardly an issue for us Software Engineers, as it seems. The result is, that inefficient programming wastes a lot of hardware capacity because we just do not care. The OLPC project was very important as it showed us, what can be done with a laptop using 2 W! Now, I do not care if the XO is the next big thing or the Asus EEE, Christoph wrote a good article about this issue, but the point is, that also we as Software Engineers should start thinking about resources much more than we have done so far. It is just embarassing when a PS 3 consumes 10 times the energy of a Wii (which is probably also not optimised as it could be) or if a laptop consumes 10 times the amount of an XO and the user is just typing a plain text.

Now what can we do? How can we incorporate this issue in teaching and training young engineers. One thing that comes to my mind immediately is the use of profiling tools. Listen for example to the presentation from Rasmus Lerdorf (yes, PHP, I know *g*, still...) from IT conversations. I was not so interested in PHP, but it was very interesting what he told about profiling and optimizing web-applications. It seems, that in many, if not most web-application there is again easily the potential for a factor of 10 in efficiency gain. What does that mean: not only is our application faster, during regular operation it consumes less energy, takes less servers, hence consumes less hardware resources doing the same operation as before.

And particularly the latter is of importance: These days we tend to think of operational costs, CO2 emission and energy use in operation. However, most of the energy is consumed before the device, the server gets into operation by manufacturing it. So whenever we can avoid a new server, we should!

Maybe there are other things to be done? For example: what about a typical server, where several applications run parallel: As in energy consumption and power plant/grid planning: when all apps like to do resource intensive things at the same time, we have to provide a server that can handle the peak load. Could it be possible to let application communicate with e.g., the task scheduler about the current "computing cost" on the machine? And if they are too high to postpone or "nice" the currently planned activity?

I must confess, that I have not more ideas at the moment, but I wanted to get this issue out, hoping for some interesting ideas and reactions from the reader.

[Tech] About Google Android: Old Wine in New Skins?

To give the quick answer immediately, I don't thinks so.

I was dealing the last weeks a little bit with Android, and I must say, that I like the concept a lot. There are several differences to existing systems that might be noteworthy:

Liberté

Android is published under a liberal Apache license and defines the complete stack starting from the Linux kernel up to the application API. This whole stack is Open Source, i.e. handset developers are free to implement this without license fees. Let's hope now, that the members in the Open Handset Alliance see the opportunity as well.

The concept of Android is in my opinion directly targetting the typical mobile-phone concept, where network carriers pretty much define the functionality of the handset. This was and is IT in cuffs and was a reason why hardly useful mobile applications and user interfaces existed. The new systems (not only Android, there are others like Open Moko) could change the game. A mass of developers could finally discover the mobile platform and develop innovative applications.

I personally think that this openness is also a chance for usage in developing countries, where mobile phone penetration typically is much higher than PC usage. Cheap, and locally produced Android handsets could provide richer access to information resources compared to conventional mobile handsets.

Egalité

First, opposed to e.g. the iPhone, all Android "end-user" applications are equal. I think this is a very important conceptual difference to other platforms. That means, that even "core" applications like the addressbook or the phone-manager could be exchanged. This is quite a different concept compared to typical Java ME applications that run "as Java" applications started from a special folder in the mobile. A home-brewn Android application runs on the same level as default-applications provided by the mobile phone.

Interestingly this seems to be a concept feared by Apple (iPhone) and other companies, however, IT history showed, that locked up systems were hardly capable to bring innovative solutions. Let's wait and see if this concept will work or not, I am personally optimistic.

Fraternité

Communication (between applications) is a core part of the concept. I like particularly three aspects:

• Activities and Intents: An Activity is e.g. a screen that interacts with the user. The interaction is actually expressed by an intent. Now the nice thing is, that these intents can use late-binding over applications. That means: If a user clicks e.g. on a telephone number in the address book the intent "dial phone" is initiated, however, which application "processes" this intent can be decided in the runtime configuration.
  
• Rich persistence and communication APIs including a relational database, a key/property store, network access, XMPP (Jabber) libraries and so on.
  
• Content Providers define interaction and data access between applications.

Finally, Development in Java

Applications have to be written in Java, which is good news for the large Java community, however there is one issue to be be taken into consideration: The Android platform uses the so called Dalvik virtual machine, which is a Google internal development and has the drawback (?) that it is not binary compatible with the other Java VMs.

This means specifically that it is not possible to use .class or jar libaries directly on Android. This does not work by two reasons (1) as mentioned, Dalvik cannot use .class files, these have to be converted to .dex files e.g. with the dx tool from Google. (2) The Dalvik VM implements only a subset of the Java 2 SE library.

However, this problem should not be too significant. One month after the Android launch first projects published special Android packages, e.g. the excellent db4o object oriented database, which could become quite popular on Android.

From the developers point of view, Google provides a quite good documentation, command line tools and Eclipse plugins including Hardware emulators for different types of mobile phones. I also suggest to check the Android Developer challenge!

[Tech] Maven as standard build tool?

For three years I have my first contact with Maven (started with version 1) and was appreciate about their dependency management. Okay, there was a lot of configuration to do and the performance in bigger projects was unsatisfied, but I think my build processes became clearer through the use of Maven, especially in the deployment area. Maven 2 was a complete redesign of Maven 1 and they introduce new major features, like transitive dependencies, Mojos and other nice thinks. First projects which are based on Maven 2 ends with a lot of unused jars in my lib, because there were problemens with the transitive dependencies, especially with their scopes. Unfortunately, Maven 2 do not work very well when developing Eclipse based applications (RCP, Plugins).

We still use Maven 2 as standard build system in our company and make profit in many areas:

• Company based settings
• Company repository, including 3rd party libs and inhouse components --> No more jars in our SVN
• Repository was set up with Artifactory
  
• Dependency management
  
• Documentation
• Integration tests and their documentation
  
• All projects have the same structure
• Continuous integration
  
• Major Open Source projects are based on Maven and many of them will migrate to maven in near future
• There are a lot of plugins and you can write your own plugins
• Archetypes, providing a basic structure for your projects or component development.
  

My impression about this blog I got on the InfoQ site, discussiong about "Maven the right tool for build". Our Best-Practice sample also based on Maven 2.

[Java] Introduction to Wicket

Wicket is a young component based web framework under Apache License, currently available in version 1.3. I've found a good introduction article hosted on TheServerSide , discussing the fundamental concepts behind the framework.

Wicket focus on Seperation-Of-Concern by providing the developer a rich component based model to develop components, which are associated with HTML pages. The logic of the components are written in Java, using the full power of the Java language and the IDE. The design of the pages and components are done by the web designer using powerful HTML design tools. Therefore you have a clean separation between presentation layer (HTML/CSS) and application layer (Java).

The first part of this article gives you a detailed overview about Models in wicket, which are used to associate your web componentes with your domain objects. Then he walks through a small Wicket example and explains the basic architecture about a Wicket applications, including WebApplication, WebPage and how components are associated with HTML pages. The example provides a solid basis to understand the basics of Wicket.

[Java] SpringSource acquires Covalent Technologies

SpringSource the company behind the popular Spring framework announces their acqusition of Covelant Technologies, providing service and support for apache products. Spring and Tomcat are the most used technologies in middleware. Read more about the acqusition in Rod Johnson blog.

[Arch] Consistency Rethought

So easy was it some years ago, when we had a database server and some telnet clients or forms-application that interacted with the database. ACID was the way to go (consistency, transaction-wise). See for example Jim Grays paper from 1981. Now we go Internet, distributed. And it became clear, that this would change the reception of consistency. In a recent article in his blog "All things distributed", Werner Vogels, the CTO from Amazon.com discusses these issues in detail: Eventually Consistent. A must read, I believe.

He quotes also Eric Brewer with his CAP theorem:

"Eric presented the CAP theorem, which states that of three properties of shared-data systems; data consistency, system availability and tolerance to network partition one can only achieve two at any given time."

And in modern distributed (web) applications it seems to become clear, that e.g. system availability is something you do not want to sacrifice. It might be irrelevant for my personal webpage, if there is a downtime of some hours, but when you operate a webshop, social website, webmail and the like, this seriously damages the trust in your service. Not to mention the financial loss, when people cannot set orders for that given down-time. The consequence? Read Werner Vogels article and get Eric Brewers keynote!

Btw.: during VLDB 2007 we had the opportunity to interview Werver Vogels and Eric Brewer to very similar topics. Check out the blog article (Interview with Vogels, Interview with Brewer, and get the audio!

[Misc] Design or Platform skills?

Martin Fowler writes in his last blog article:

"Imagine a hiring situation. There's two candidates both with a few years of experience. In the blue corner we have someone with good broad design skills in the style of design that you favor [...]. However she knows nothing of the particular platform technology that you're using. In the red corner we have someone who has little knowledge (or interest) in those issues, but knows your platform really well - edge cases in the language, what libraries are available, fingers move naturally over the tools. Assume all else about them is equal (which it never is except for thought experiments like this) and that your team doesn't have any gaping holes that this candidate might fill. Which one would you prefer?"

In the following discussion he analyses why he would prefer the person with the design skills, but read it for yourself!

I mean, he is discussing the topic quite balanced, still I have the feeling that he underestimates the importance of platform skills. My impression is, that in the last (10) years platform knowledge became significantly more important (as the "big" platforms, e.g. Java, .net grow in terms of tools and features). Actually, writing a non-trivial application in, say Java, demands a very good knowledge about available libraries, the "best-practice" tool chain (build automation, testing), component frameworks, server frameworks, configuration management, testing frameworks, JVM issues, and so on. There are so many odds and ends you better know for not wasting unnecessary time or going into a unproductive direction.

Plus, I would wonder how a person that has really deep knowledge about the specific platform and tools (as I mentioned it above) could be unskilled in design issues anyway. Just be dealing on an intimate level with modern frameworks like Spring, Guice, Eclipse RCP, Maven, GWT and other open source projects you get an idea about how to design applications properly!

Plus I find it easier to get a new team member on speed that is very good at the platform details but has some lack of knowledge in patterns and design then vice versa. Why? Because in the starting phase this person would rather implement "details" that are designed by more experienced staff or work close with experienced staff. However, I know that this new person does not make platform-rookie mistakes in the implementation process that turn out to be "time-bombs" in the later phase of the project. Getting a broader "design-oriented" view is only a matter of time for a bright person.

Am I naive here?

[Java] Generic Data Access Objects with Java Generics

The Data Access Object (DAO) pattern is an essential design pattern in enterprise applications. In a DAO all data access to the underlying data source is encapsulated by providing a generic interface. Service components use the DAO in order to load and save data to the data source.

A DAO interface usually provides common CRUD (Create, Read, Update, Delete) operations. The implementation of such DAOs can become a painful and boring work, because almost the same code is written for every DAO. Consider an application with hundreds of domain objects, where each object must provide a DAO!

In my post “Has JPA killed the DAO” I summarize the necessity of the DAO based on the discussion from Adam Bien. In this post I mentioned that JPA provides a generic data access functionality. In the present post I will show how to write a Generic DAO interface by using Java 5 Generics, Hibernate and Spring.

Java 5 Generics allows to parametrize and to create typesafe DAO interfaces. So it makes sense to adapt generics to implement a generic DAO. The following code snipped provides such a DAO interface

public interface GenericDao {

 public T saveOrUpdate(T entity);

 public void delete(T entity);

 public T findById(ID id, boolean lock);

 public List findAll();

 public List findByExample(T exampleInstance);
}

The interface definition gets two parameters: T for the type of the domain object and the ID for the type of primary key. These parameters are then used in the interface methods. There are no specific domain objects but still typesafe by using Generics. Now we have an implementation with Hibernate and Spring. In this case we use HibernateDaoSupport from Spring:

public class HibernateGenericDao extends HibernateDaoSupport implements GenericDao {
 private Class type;

 public HibernateGenericDao(Class type){
     this.type = type;
 }

 @Override
 public T saveOrUpdate(T entity) {
     getHibernateTemplate().saveOrUpdate(entity);
     return entity;
 }

 @Override
 public void delete(T entity) {
     getHibernateTemplate().delete(entity);
 }

 @Override
 public List findAll() {
     return getHibernateTemplate().loadAll(type);
 }

 @Override
 public List findByExample(T exampleInstance) {
     return getHibernateTemplate().findByExample(exampleInstance);
 }

 @Override
 public T findById(ID id, boolean lock) {
     return (T)getHibernateTemplate().load(type.getClass(), id);
 }
}

Also the implementation needs no specific domain objects, we still work with Generics. The type attribute in the class is important. It specifies the type of the domain object for which the DAO should work and is further injected by Spring when using Dependency injection:

Instead of using two separate DAO implementation we only have one generic DAO and instantiate the DAO for the business object. The constructor gets the type of the domain object. That makes sense, wow.

If you need domain specific DAO operations extend the Generic DAO!!

Hope this blog impresses you to take a look to Java 5 Generics and Generic DAOs.

[Event] Berlin / March: Object Database Conference

Let me wish you a happy new year and welcome this year with a little announcement:

On 13+14th March there will be an Object Database Conference in Berlin.

Some facts, before I explain why this is important for Software-Engineering and not only for the Database / Persistence area.

A) The conference is split into a Science Day (13th March / Thursday) and an Application Day (14th March / Friday). There will be peer reviewed talks and papers on 13th and many practical talks on Friday.

B) The conference already has joined up an impressive line of (keynote) speakers. To name a few: Christof Wittig (db4objects), Robert Greene (Versant), Leon Gudzenda Objectivity), Mike Card (OMG), Roberto Zicari (odbms.org), Carsten Czarski (Oracle), Rald Westphal (.NET Expert), Carl Rosenberger (db4objects Chief Architect), ...

C) There will be a lot of hot topics as Standardization (OMG), Spring, Eclipse, Android, and more. All connected to object databases.

D) Talks and Papers are still welcome. Please tell your colleagues about the conference link http://icoodb.org

E) And finally Berlin is always worth a trip, the conference itself does not have the normal exaggerated prices and hence is quite affordable.

So you might think: Well I am in love with MySQL / Hibernate and I have understood the relational data model. That's pretty perfect and you might have a wonderful solution at hand that suites pretty well in most cases.

But as an advanced Software-Engineer you should consider the following:

1. You should know the entire persistence space. Are XML Databases as Tamino, eXist, or Xinidce important? Yes they are! Are Object Databases important to fit perfectly in specific areas? Believe me: they are, they do. The conference will help to clarify this and lets you play the entire persistence keyboard.
2. Did you know that some of the largest and fastest databases are run on ODBMS (Obejct Database Management Systems)? Did you know that on the other hand embedded object databases can be found everywhere? And did you know there is a smarter and mostly faster database on Google Android then SQLite?
3. And for the Software-Engineers reading this:
   

Even if you are plan to run a monster big and relational database you can benefit from Object Databases (ODBMS):

a) Add a simple Object Cache to your application using ODBMS. It's really trivial.

b) Are you building a prototype first where the persistence layer comes later? Simply add a DAO Layer that saves ANY object (even the deepest) with only 2 lines of code any nothing more to do! Later when your relational persistence comes, you can easily switch to the EJB 3 / JPA / Hibernate Layer if needed. But till then your persistence layer simply works even simpler then with these 'cool' class annotations.

c) You need to test your Business Code? You use Mock-Objects generated by jMock, asyMock or *Mock? You sometimes want to write a mock without the database connection? Use an object databases plugged in your mock to provide your mock with arbitrary domain objects to test with.

To conclude: We hope that these examples gave you lots of arguments to join the ICOODB.org conference, spread the word about ICOODB.org or get in touch with us.

Any questions? Please write a comment here or get in touch with us:
info[at]icoodb[dot]org

And perhaps CU in Berlin!