[Misc] Software Carpentry

In "Interviews with Innovators" Jon Udell talked recently with Greg Wilson. Greg Wilson is well known for his "Software Carpentry" courses. These courses did not focus on computer science students but mainly on students from other scientific disciplines like Chemistry, Physics or generally engineering studies. His goal in the "carpentry" courses was, as I understand it, to teach scientists who need Software for their work the essential tools and practices. I have the feeling, that this is pretty similar to our "best practice" efforts like our websites, this blog and the book.

However, the motivation is very interesting. One nice example he gave comes to my mind: He occasionally noticed that scientists (using apparently Matlab and the like) use functions mainly for code that is not used any more. Pretty counterintuitive? Well: they use to write the code (as it is possible in many scripting languages also e.g. in Python) in "Spaghetti" script style; as soon as they make major updates they put unused code (that they still want to keep for some purposes, because they also are not aware of versioning tools) into a function, because code in a function is not executed automatically. Nice approach ;-)

However, aside these extreme examples there is, in my opinion, a lot to learn also for education in "regular" computer science classes. As Wilson puts it in an example:

"The research money and PBS Nova programs focus on artificial hearts when in fact all the increase in longevity comes from clean water, anti-smoking campaigns, better nutrition, vaccination and the like. These routine public health measures that no longer are exciting, so they are actually loosing ground."

The same is probably also true in Software Engineering education: 95% of  engineering, but probably also of IT students will not develop the next "Google" but some Web-Forms that connect to a database and produce some reports. But this should be done in an efficient and maintainable way.

I believe this is an interesting thought: we write develop and focus on the latest "cutting edge" technologies, probably leaving the majority of developers behind and probably even worse: confusing them more every day with always new approaches and tiny improvements instead of focusing on the problems the majority of developers and companies have.

Listen to the interview and share your opinion!

[Tech] Cloud Computing

As I think, I already mentioned here, I believe, that Cloud Computing (and Software as a Service, but this is a slightly different topic) are true game changers in our understanding of software infrastructure and development/deployment. Currently things are still quite rough around the edges, but I believe, that in like 3-5 years the default option of application deployment will be in one cloud service or another. Putting iron into the cellar or storage-room will be what it should be in my opinion: mostly a stupid idea ;-)

In the current stream of IT Conversations George Reese talks about practical aspects and experiences with current cloud-services like Amazon S3, Simple DB, virtualisation... Recommended!

[Misc] Managing Commercial Software Projects

At IT-Conversations there is a recent interview with Jon Udell talking to Andy Singleton about "Managing Commercial Software Projects". This interview is highly recommended. Actually I figured that Singleton is following pretty much on the same track as I am. However he is making some bold statements; in projects he is using no phone-conferences ("the more phone conferences in a project the more problems the project is in") or VoIP/Video, no time-estimations are done when not explicitly demanded and he is following Open Source practices of distributed development using mostly asynchronous tool-chains.

He is apparently providing a set of development tools "best-practices" at assembla.com: One of the core concepts there is to assemble people around event-streams (of activities). I think, probably the main idea that I totally subscribe too is this: It is actually much more important to have awareness what others are doing than investing a lot of time into planning efforts.

However, listen to to the full interview, recommended!

[Arch] Apache CXF and "Code First" Webservices

Dana Gardner has an interesting interview in his Briefings Direct podcast (transcript) talking about the Apache CXF project. Apache CXF is one of the leading "Open Source Service Framework", supporting a series of service protocols like SOAP, XML/HTTP, RESTful HTTP, or CORBA. Additionally CXF plays in the concert with Apache Camel, ServiceMix and ActiveMQ.

I believe, that CXF is a great project and as I said a leading webservice framework. I also perfectly agree on their statement, that Open Source frameworks in the middleware/SOA field will more or less take over the market (I would probably not buy shares from Oracle/Bea *g*). But I feel a quite a bit uncomfortable about two fundamental concepts that were stressed several times in the interview:

1. Code First approach and slightly connected:
   
2. Abstracting too much from Webservice protocols like WSDL: "So, a lot of these more junior level developers can pick up and start working with Web services very quickly and very easily, without having to learn a lot of these more technical details.", Kulp
   

I personally believe that this is not the right way to go. I think we have to take the criticism of the REST folks (among others) seriously here, and the question we have to ask ourselves is: why in the first place do we want to use SOAP webservicess?

Because it is an easy remoting approach that a junior level engineer can click together with a wizard due to excellent frameworks and UI components? Sure not! First: a remote methode call is something fundamental different compared to a local method call; it probably should be treated as such. But even more important: Services on the basis of SOAP have some underlying assumptions:

• platform independence is needed
• Services on a rather coarse granularity are exposed
• interoperability over system/company borders are imperative, i.e., the service interface is in the center and should be considered properly and not change any minute
• remote service calls are rather the exception (coarse granularity, aggregated functionality),
• strong formalisation is needed (XML schema, service description, security...)
  
• i.e. performance losses due to XML (un-) marshalling are acceptable considering the advantages gained by this "neutral" approach
  

and probably some more. The point however is, that SOAP services are not just an ordinary remoting approach they are to expose Services under specific conditions. (For other remoting problems probably other technologies like RMI or REST are better suited.) In the cases where SOAP is a good architectural choice I would suggest that the platform neutral service description i.e. WSDL should stand in the center of attention. The idea is, that different parties can express a neutral and platform independent service description plus data-description in W3C schema as a foundation for cooperation.

The next step, not the first step is to implement the service. So my feeling about code-first approaches are that they can lead developers and architects into a dangerously wrong direction (just two annotations and the webservice is done, so we are made to believe). What we would really need are not code-first webservice frameworks, but easy to use WSDL editors/modeling tools.

Additionally a Webservice infrastructure is by definition a complex beast. Trying to abstract all underlying protocols from the developers easily gives them a wrong idea about the actual complexity of their undertaking. When (e.g. interoperability, security) problems occur, they probably have no idea about the reason and the means to fix them. So give us good Service modeling tools, but no code-first approaches. This leads us into the wrong direction.

Just my two cents.

[Tech] Tim O'Reilly: From Web 2.0 to Robotics and Genomics?

Tim O'Reilly speaks in an interview at the Web 2.0 Expo in Berlin about the "second coming of the web", that they named Web 2.0 at time ("a stupid name") and he shares some thoughts about current "social networks" and "social graphs" like implemented in Facebook, and why he is not really happy with many of the current developments. He again stresses the importance of data over applications and sees several "undiscoverd data bases".

He also refers to Open Social. An interesting initiative from Google to provide a common API for various social applications, that could provide a basis for software developers to create larger clusters of social applications. Furthermore he discusses privacy aspects and the change in the attitude of people towards privacy.

However he is apparently not happy with current developments, which are not adequate to the possibilities new devices (like the iPhone) and "Web 2.0" generally would allow. He also compares innovation cycles: the PC was hot in the 80s and got very boring in the 90s also due to the dominance of companies like Microsoft. Similar developments could be seen with the phone companies, that overslept most of the new developments and might happen to todays giants like Google.

So the underlying question is: Do companies and particularly developers understand the potential of "Web 2.0" ?

[Event] OLPC Presentation, Podcast

Last week Aaron Kaplan and Christoph Derndorfer from OLPC Austria presented the OLPC (one laptop per child project) at Vienna University of Technology. The presentation was vivid, also because several laptops were presented and the interaction was demonstrated (including a drop from approx one meter height).

Presented topics included the idea of the project but also several details to technical implementations and Software Engineering aspects. To provide this information to a larger number of people I recorded a (German) podcast with Aaron and Christoph today. It is an enhanced podcast, so please do not miss the pictures, screenshots and links.

For the English speaking audience I recommend the "main" information source of the OLPC projects:

http://wiki.laptop.org/

[Event] VLDB Interviews 2: E. Brewer, M. Stonebraker, M. Brodie

As already explained in the previous posting, we had the great opportunity to record interviews with the keynote speakers of VLDB 2007. In the second podcast episode we got three more interviews with Eric Brewer, Michael Stonebraker and Michael Brodie (from left to right):

Eric Brewer talks about technologies that can help emerging countries in building up IT and communications infrastructure, Michael Stonebraker (who also writes a Blog: "The Database Column") and Michael Brodie talk about trends in database technology, enterprise data management and limits of current technologies and product.

All three are asked about research and about the time-to-market issue for new ideas Werner Vogels brought up in the last interview. Additionally innovation at "big" companies vs. startups is discussed and how venture capital can be used to get out your innovative ideas.

Again to mention: the Podcast series in general is German, the interviews are English. Thanks a lot to Dr. Ross King from Vienna University who again asked the questions in the interview.

[Event] "Does Amazon do research?" Amazon CTO Vogels in Interview

Today I had the pleasure to record the interview with Werner Vogels, the CTO from Amazon.com, who held the keynote speach of the Very Large Databases Conference currently held in Vienna.

The interviewer is Dr. Ross King from Vienna University and the interview is part of our "Woche der Informatik" podcast. This podcast is actually in German, however for the English speaking audience the VLDB keynote interviews (starting with this one) are accessible as well. Just ignore the German introduction and jump right into the talk with Dr. Vogels (chapter marks...).

Actually the keynote (and the interview) was very interesting, some points Dr. Vogels discussed where:

• The problem of state management ("state management is a dominant factor in scaling")
• Amazon as company: "Amazon is a technology company that accidentally works as a retailer"; he also shows a series of other E-Commerce sites like Marks and Spencer, Mothercare, Smug Mug and others that are actually build on top of Amazon technology.
• Amazon apparently goes the way (similar like EBay): from a retailer (auction house) to an e-commerce technology provider/platform.
• A dominant issue in the talk was scalability. COTS products typically do not scale the way Amazon needs it (we tried out mainframes - for one year). Vogels refers to the stability and self-organisation features of biological systems and names particularly "Apoptosis": cell-death; allthough daily 50-70 billion cells die every day, the biological system is stable, aka the human stays alive.
• Thus Amazon services are build highly redundent. The loss of a complete datacenter would not harm the customer experience. He additionally shares two experiences that might contradict certain academic ideas:
• "Everything fails, all the time"
• Systems do not fail by stopping, they might actually do all sorts of weird things in between.
  
• Vogels claims, that Amazon did SOA before it became a buzzword.
• So eventually his bottom line is "Architecture for change".

The presentation was very vivid, and I think some of the ideas were also captured in the interview (and btw. Dr. Vogels has his own Blog: All things distributed. However, parts of the ideas he expressed reminded me strongly to a very good book I like to recommend: Stan Davis and Christopher Meyer. It’s Alive: The Coming Convergence of Information, Biology and Business. Texere Publishing, 2003.

Check out the podcast website!
Or go directly to the feed page.

So, to eventually answer the question in the title, a last quote I personally liked: "Does Amazon do research? We call it production."