[Misc] Subversion turns into an Apache Project: so what?

Since a few days it is official: The Subversion project has submitted to become an Apache project. It seems that the incubation phase will start soon. Now my question: Subversion is conceptually dead, so what difference does that make? Ok, let's discuss this a little more in detail:

The thing is: most developers (even myself) meanwhile understand the concept of DSCM systems and all available projects are stable, fast, have good communities and are reasonably documented. Even tool support (IDEs, ...) is decent. Having understood DSCM I wonder why I would want to go back to a centralised system like SVN. There is no benefit in there for me. If I want to work server based so be it: I can do this with Mercurial for example in BitBucket, with Git in GitHub or by simply installing the DSCM on an arbitrary server that has ssh access.

I had a discussion with an Apache commiter recently about this fact and about the future of Subversion. He believed, that Subversion could (or better should) go through a complete redesign to embrace features provided by distributed source code management (DSCM) systems like Bazaar, Mercurial or GIT. I personally question this future of Subversion. We already have three pretty good systems and a very competitive game is played here since the last two to three years. Subversion would start with a delay of probably three years. Until a stable version of a (partly) distributed SVN is out, all other systems will be settled and far ahead.

There is, however, one major feature DSCM-systems can not provide by definition: this is collaboration via locking. This is an important feature for collaboration on (large) binary files like Photoshop documents, multimedia files, vector graphics and the like. Merging of such documents is practically not possible, but many (software) projects partly rely on a significant number of documents of that sort. Keeping those in a DSCM system is not the best idea. My future scenario for distributed (software) development hence consists of two repository types: a distributed one for sharing text-based files (like sourcecode) and a centralised one that provides versioning and a very good locking (check-in/out) support for (large) binary documents.

Now as I understand it, Subversion is neither really good in locking nor management of binary data either. Also tools-support for non-programmers (who often work with such binary documents) is not so great. So what is the future of Subversion? I believe already today it is pretty much a legacy like CVS. Off course, there are so many projects using Subversion, that we will probably have to deal with it for the next decade (not that I would like to). However, the migration wave has started and most new projects will use one of the mentioned DSCM systems. How could a re-write of Subversion help? Well, the principles are so different that Subversion with distributed characteristics would be either a new project (and I doubt that we need a forth DSCM system, as mentioned above) or would keep a lot of the disadvantages of the old system.

But maybe Subversion could focus on the locking-based approach: this is a very much needed feature for many projects and I also do not see much competition (in the Open Source environment) here. A good repository for binary data could be a reason to stick with Subversion for parts of the development efforts.

Your ideas?

[Tech] 7 Languages in 7 weeks

Dear Readers,

als you are all interested in programming languages I would like to point your interest to this link which I was pointed at (lots of thanks to the source!):

http://rapidred.com/blog/seven_languages

It is from the blog of Bruce Tate we all know as one the Java Experts and his stunning books.

As far as I know he started an interesting project because he was also interested by the polyglot language area. As I heard he raised a vote about the topic and the languages about his next book! The vote together with his opinion lead to the following languages which I would like to comment on:

• Ruby -> my personal favourite. Perhaps not the coolest now but the expressiveness and the DSLability for me is outstanding.
• Io -> Possibly the newest and coolest because the vm / object approach looks interesting.
• Scala -> Good to have Javas crown prince in here because we all have to learn it.
• Erlang -> My multiprocessor king (even if it struggles with strings. argh). Especially hot in the #nosql database scene.
• Clojure -> I already posted about the great clojure. I really love it although its really hard to learn.
• Haskell -> Good that they / he included the right educational functional concept.
• Prolog -> This surprised me a little. But Bruce writes that he wants to stretch the readers. And I never thought this could go with a nearly 40 year old language.
  

So have a look on this book at http://pragprog.com. It's is a definite buy for me even if it hasn't been written yet.

[Arch] Resource for Software Architecture

I've found a realy good resource for german audience about software architecture hosted on MSDN. On the MSDN Architecture Center you'll find:

• Actual news and trends about software architecture
• Basic information about software architecture (concepts, styles, etc.)
  
• Podcasts
• Tool Previews
• A free english architecture journal
• Forum and Knowledge Base
• Tips and Tricks
  

Advance your architecture skills by looking at this resource. It is worthwhile itself to make a view to this resource.

[Conf] Zurich Open Source Jam

On August 13th, more than 50 other people, interested in open source software, attended the 8th Google Open Source Jam in Zurich, which is an informal (bar-camp like) meet-up at Zurich office (also available in other parts of the world) and a perfect opportunity to meet other open source developers as well as Google engineers in a relaxed atmosphere. As it is open to everyone, people held several lightning talks on a great variety of topics:

• "G-WAN", Pierre Gauthier
  
• "Dynamics of Open Source code", Markus Geipel
  
• "Involving students in Open Source", Lukas Lang
• "Open Source in Africa", Michel Pauli
• "BTstack", Matthias Ringwald
• "Free Software & basic income", Thomas Koch
  
• "NxOS, an OS platform for Lego", David Anderson
• "Open Source in the Humanities", Tara Andrews
  

My talk was related to open source student projects, accomplished within the scope of the course "Advance Software Engineering", held at QSE. Four projects were completed successfully in the last two years and got integrated to the codebase:

• Castor JDO Extensions
• Castor JPA Annotations
• Cocoon StAX
• Cocoon Profiling
  

Similar to Summer of Code, these students have been mentored by experienced open source committers from the Apache Software Foundation and the Codehaus. Developers and students, participating in open source projects themselves, commented a lot on this topic: "I wish, I had something similar when I was a studying", said a Google engineer.

Afterwards, we continued to have interesting discussions. After some time I found myself in an exciting discussion on software engineering at Google. First off, I'd like to mention that employees never make clear statements concerning their work as they are bound to confidentiality. Even though no specific software development process was confirmed, one could identify tendencies:

Don't repeat yourself (DRY). Code and software reuse as a basic principle. The Google Code repository was created as a collaborative platform to manage, document and review free/libre open source software (FLOSS) projects. Indeed, employees spend up to 20% of their time contributing to open source projects.

Don't reinvent the wheel. "At Google we don't reinvent the wheel, we vaporize our own rubber", told me one of the engineers (they use heaps of metaphors like this) meaning that a vast majority of the software in production use is built on top of parts or complete open source libraries. Aside from releasing software like the Web Toolkit, Android, Chromium, etc. back into open source, Google contributes to a diversity of FLOSS projects (e.g. Linux kernel, Apache projects, MySQL, Mozilla Firefox) [1]. However, they keep implementations of key technologies a secret claiming that for instance their webserver, apparently a Tomcat re-write, was "to specific to benefit from" or just don't publish it for competitive reasons [1]. The same goes for Google File System (GFS), BigTable and MapReduce. In a nutshell, scientific publishing [2] of these core technologies at least led to great open source implementations (e.g. Apache Hadoop) which are open to everyone.

[1] A look inside Google's open source kitchen, http://www.builderau.com.au/strategy/architecture/soa/A-look-inside-Google-s-open-source-kitchen/0,339028264,339272690,00.htm
[2] Google Publications, http://research.google.com/pubs/papers.html

[Process] Distributed Source Code Management and Branching

I am using Mercurial a lot recently (and love it); I really do wonder, why I struggled so long with Subversion. When I first heard the GIT presentation from Linus Torvalds (which is, hm, very entertaining) the whole distributed SCM thing sounded very esotheric for me. However I decided to give it a try, also motivated by the great Chaosradio Express 130 Podcast (German). Yet, I decided to go with Mercurial and not Git; allthough this created some flame-wars within our group, because one of my colleagues is a big Git fan. So be it ;-)

For me Mercurial is a great, easy to install, and pretty easy to understand system. The commandline is really straightforward and the help-texts well written (and interestingly - internationalised). Maybe I follow up another blog post with more details to Mercurial another time.

As easy branching and merging are among the main advantages of the new distributed SCMs, I want to recommend for now the very nice blog-post by Steve Losh "A Guide to Branching in Mercurial". This article provides a good and conclusive introduction to different methods for creating branches with Mercurial and also explains differences to Git and some of it's shortcomings (*g*).

p.s.: For my taste, just one thing is missing: some details on merging.
p.p.s.: Please no comments on my "Git shortcomings" statement, they will be censored out anyway ;-)

[Misc] Clojure & Clojure Book Review

It looks like we are living in a fantastic time concerning programming languages. Creating a new language has never been easier then before. With the two great platforms Java and .Net it's not extremely difficult any more to generate intermediate code from the language you are dreaming of. And even the pragmatic bookshelf has a book in writing on "language patterns" to cover this topic.

Thus I assume that nearly every developer is looking out to bet on A) the best horse or to see B) a language from where you can learn the most. Unfortunately the whole world seems to bet on Scala from the sea of hot languages. And indeed Scala is quite cool and might win the race with good reason. Scala is hot stuff for reason. Nevertheless I never got really warm with Scala and it's hard for me to describe why. It's something in the syntax I really can not explain. My dream language must have a code that looks quite perfect. Years ago I had that feeling coding Ruby (and I love Ruby) because the code simply looked great (although I don't like other things in ruby as e.g the clumsy class definitions (there was a project on rubyforge to fix it. Does someone know the name?)).

So I am still constantly looking out for new languages and of course the pragmatic programmers have brought this book into my view:

Stuart Halloway, "Programming Clojure", 2009

So I might share my thoughts on this book and on the language.

For me still Clojure is more attractive then Scala because:

1. It's the toughest edge for your brain.

Perhaps you belong to the same generation as me: I never really used Lisp, Scheme or the dialects alike. But as I am a Java, D and Ruby Coder => Lisp and thus Clojure ("Lisp Reloaded") is the hardest challenge to learn. And according to the last two book reviews you should always go the hard way to learn the most. This might be even true for you.?! For me it turns out that the amount of round brackets is not the problem.

2. Clojure enforces the use of no variables but immutable data structures (as the successful Erlang does!). The code / structures itself mostly is the container for variables you would define use normally. This has two really strong advantages:

1. No variables mean less errors and less to debug
2. No variables and immutable data means no side effects
Hence it's not so easy to write Clojure code with much side effects.

3. Clojure has the strong concurrency concepts - if you really need mutable data - using STM (Software Transactional Memory) / MVCC, Agents, Atoms, giving you ACID Transactions without the "D" (which is only valid for databases).

4. The Java integration is really smart. Have a look:

(new java.util.Random)

simply generates an object you can easily use to work with.

Java and Clojure can call themselves easily vice versa. This might be a really good argument for you to save investments. Clojure doesn't really try to build up all the java libs from scratch; it reuses them in a clever way.

5. Clojure is indeed fast because it generates pure Java byte code.

And of course all the other stuff that a hot language must have:

• Closures (I still can not believe that they are still discussion this hot feature for Java... it's a shame)
• List Comprehensions
• a workaround for tail recursion and currying and other weird stuff as
• very lazy sequences, trampolining, etc.
  

What also challenged my mind is that Clojure has only three constructs for flow control (as we learned from Oz a simple and nevertheless powerful language doesn't really need much):

• An "if" See an example (if (> num 100) "yes" "no"))
• A "do" which is an iteration of statements - introducing side effects - and thus discouraged in clojure
• And a "loop/recure" that can be used to build everything you need in flow control (Stuart Halloway calls it the Swiss army knife).
  

So before I discuss the downside of the language and the book, let me bring up the two points that I loved most in clojure (even if it takes a lifetime to understand them 100%):

1. It has Metadata incorporated in the language right from the start! So you can tie pre- and post-conditions, tests, doc, arbitrary macros to any kind of data or functions. Whow! Do you remember how long it took Java to introduce Annotations? And to me they are still not 100% part of the language but an add on.

2. Clojure has powerful macros and multimethods. This means DSLs are incorporated. So if you loved the way Ruby can build DLSs (and the pragmatic programmers will bring up a nice book on this Ruby DSL topic!) you will get a step further in Clojure.

The book itself uses all hot Clojure features to create a build system called lancelet (every language since Rubys Rake seems to do this a little bit cooler then Ant does).

What I disliked in the book is that it's still difficult for beginners although it is very well written with beginners in mind. One example: I still havn't found the page where I can read a string from the console. And this is a key feature for beginners to test something. The interactive REPL is not enough here. What I mean is that the book has a thousand brilliant examples like fibonnaci. But fib is a two edged example. It just calculates. It creates no emotions in the reader. A better example is the snake game the book creates on a few pages later (here we get input from a keyListener...).

The language itself has two downsides for me:

1. If you are used to the huge Java Collection Lib you are astonished that Ruby boils down all data structure to just a super powerful Array and Hash (and the others are rarely used). That's cool. So you get the impression that Clojure get's on step further in stating that everything is a set (like Lisp stated that everything is a list). But when working with Clojure you are suddenly confronted with not only sets but vectors, lists, maps and trees. Now the book tells you that you always use the set abstraction to work with this. To me this doesn't really help if e.g. the vector notation differs significantly from a set and list definition. I still don't get used to this. but it must be surely my personal inability.

2. The most important drawback is the key feature at the same time: Clojure has an extreme steep learning curve! To become a true expert in Clojure, i.e. to think and dream in Clojure you need to stress every neuron in your brain. So it all boils down to the question: Is it really worth investing half a year in a hot language like Clojure to be able to produce code that is 3 times more accurate imperative Code?

What do you think?

Regards
Stefan Edlich

Links:

• http://clojure.org
• http://www.pragprog.com/titles/shcloj/programming-clojure
• http://ociweb.com/jnb/jnbMar2009.html
• http://en.wikibooks.org/wiki/Clojure_Programming

And finally have a look at this nice language comparison: Java.next
Make a list. For every topic give points from 0 to 3. What is the most elegant for you?

[Misc] Two Pragprog Books Reviewed

Book Review: The passionate programmer and Pragmatic Thinking & Learning

Recently I am getting more and more attracted by the books from "The Pragmatic Programmers / Bookshelf" (link). So I share my thoughts with a review of three books for you. Here I review:

1. Chad Fowler, "The Passionate Programmer", 2009
2. Andy Hunt, "Pragmatic Thinking and Learning", 2008
   

So let's start:

1) Chad Fowler is well known in the Ruby and Rails scene. So he he shares his visions using 53 chapters. Each with a message for you and explained. These messages look a little like the XP programming message. And sometimes they really read like these XP rules:

• "29. Learn how to fail" (testing)
• "18. Automate yourself into into a Job" (daily integration builds)
• "28. Eight hour burn" (no overtime)
• ...
  

But indeed his stories are nice to read and they go far beyond XP rules. They bind personal experience together with passion and and a possible new perspective for you. So his main point is to step out of daily routine, step back, get better and build up new goals for you.

Chad is quite strong in selling his point and most of the points are really fun to read (for example "20. Mind Reader" or "45 You've already lost your job"). So this book is not for experts who have already found their mission in doing independent consultant work for an apache product of which they are a top committer. It's a book for the employee wishing to get motivated and possible building up new perspectives in his career. And of course for beginners that might be reading an XP book at the same time. Chad includes nice actions for each point so that each point can be validated for yourself.

What distracted me a little is the analogy to other jobs. Many writers today cite that they have played (jazz) music in a band. And the challenges in a jazz band are quite the same to a software developer. Chad elaborates a lot on this topic. Even Andy Hunt (see the next review) draws this analogy and many other books (e.g. Presentation Zen by Garr Reynolds) can not leave this point - e.g. be the worst guy in your team - untouched.

Nevertheless it's a fun read if you want to break out and the book should also be recommended in software engineering / programming lectures.

2) Andy Hunt also wrote a remarkable book in combining cognitive sciences with software development. And indeed this is neither a neuroscience book nore a software development book. It is a wonderful walk through topics like:

• Journey from Novice to Expert
• This is your Brain
• Get in your right mind
• Debug your mind (how your mind works)
• Learn deliberately
• Gain Experience
• Manage Focus
• Beyond Expertise
  

This book could also be named "Your brain - The missing manual for software developers". So it's a wonderful guide to understand your brain and how to improve. The book is full of nice graphics, anecdotes and actions the reader should do. Through the book Andy collects Tips which are grouped together at the end of the book in a nice reference card.

If you read this book you will find some topics not new for you as mind maps or wikis. But Andy puts this in a context, gives many advices and he touches a lot points which will be new for you. For example the intense description or the L- and R-Mode helps a lot on how to reflect and use this modes in daily life. And there are a lot of other great new topics you can experience (as morning notes, SQ3R, etc.).

There were really just a few pages that were uninteresting to meas his elaboration on "expect the unexpected" or his way of categorizing generations.

Nevertheless it's a very practical book covering wide ranges of topics as drawing (there are some drawing exercises for you inside) up to yoga techniques. And everything could be applied for your daily life, job or your software development. So this book is a clear buy recommendation and even better a good present for your hacking friend or partner.

[Arch] UML Tools for Mac OS X

Following up a question I received via Twitter, and the fact, that a significant part of the developer-community is using Macs, I thought this might be a good opportunity to discuss some "UML Options" for the Mac. Now, this article is not meant as a definitive answer, I would hope for some follow-ups by readers in the comments.

Ok lets start: First there is heavy weight stuff, most notable Visual Paradigm. A warning: this is a fat tool. However, among the fat tools it is the one I liked the most. I am not using it any more, but it is generally rather easy to use and very feature rich. However, it is a pretty expensive commercial tool. Yes, they have a "community edition", because it is cool to have a community edition these days. But this one was (when I used it last year) rather a joke. See it as a test-preview.

There are also other commercial tools as well, e.g. Omondo. I have not much idea about this one though. Anyone?

On the other end of the spectrum are tools like UMLet (or Violet), which are also Java-based and work more or less good also on the Mac. These tools are very basic and one should not expect much. They are definitly not suited for "real" projects or commercial application, but can be a nice option e.g. for educational purposes. Sometimes one just needs to create some simple UML diagrams for a presentation, paper or book. For such purposes these tools might be useful. Plus both are Open Source tools.

The probably best free (but not Open Source) UML tool, and the one I would recommend is BOUML and this is sure worth a try. The main issue I have with nearly all free/OS UML tools is, that they are often driven by a single person or just very few developers. Hence the future of the particular tool is always a little unclear. To make things worse, there is no accepted open file-format for UML diagrams, that would allow easy exchangeability of the tool. Hence selecting a UML tool is always sort of a lock-in situation.

Also a consideration could be ArgoUML, which is also an Open Source tool and maybe the oldest one around. Has some issues as all OS tools, but apparently has a functioning community.

Finally there are some more or less general purpose drawing programs, that can be used for technical diagrams like EER or UML models as well (with some limitations) like OmniGraffle or Concept Draw and finally also OpenOffice Draw can be used for general purpose vector-oriented diagrams.

Would be happy about comments, experiences and further suggestions!

[Tech] Monitor your WS calls

If you develop applications, which consumes web services from other applications or integration platforms, debugging can often be very deflating. If you don't use the correct debugging tools, you don't see the generated SOAP messages which are delivered between the parties.

A very useful tool is the Open Source SOAP monitoring tool from predic8. The tool does the same as the TCP monitor from Axis, but provides a more user friendly UI and more settings and features:

• Monitoring of SOAP and HTTP messages
• Rule based SOAP routing
• XML formatting and syntax highlighting for SOAP messages
• Interception and modification of messages
• HTTP chunking
• HTTP 1.1
• Loading and saving of configurations
• Rich graphical User Interface
• Resending of messages

The monitor acts as a proxy. Therefore your client application must send the SOAP/HTTP messages to the proxy monitor, which delegates the messages to the real endpoint. A Quick Starter Guide is also available.

[Pub] Mule Tutorial

In the current issue of the Java Magazin I published a tutorial to develop loose coupled systems with Mule. The tutorial illustrates the usage of an Enterprise Service Bus in an airport domain, where different airport systems communicate with each other over the ESB. In the example I use a set of important Enterprise Integration Patterns and show how these patterns are implemented in Mule. Some patterns I used are:

• Event Driven Consumer
• Content Based Router
• Filter
• Transformation
• Message Splitter
  

The transports and connectors I used from Mule are:

• JMS (Active MQ as message broker)
  
• Quartz Transport
• File Transport
• XMPP transport for instant messaging

The source code of the tutorial can be downloaded here.

Have Fun!