Wednesday, January 30, 2008

The Groovy Releases

Today, Groovy 1.5.4 was released, the 30th official release of a Groovy version. For this month's entry tracking the Groovy Programming Language here at Gavin Grover's GROOVY Blog, let's look at the official releases of Groovy.

The Early Betas

The very first release was of Groovy 1.0 beta 1 on 11 December 2003, coming in at 4.9Mb. According to the release notes by James Strachan, it had compilation straight to Java bytecode, groovyc Ant task and command line script, an early Swing-based console, and a simple command line terminal. It had full support for properties and fields, native syntax for maps lists and regexes, autoboxing, Ruby 2.0-style closures, both static and dynamic typing, operator overloading, text templating, GPath, GroovyMarkup (supporting DOM, SAX, Swing, and Ant), and Groovlets. A few weeks later, beginning a tradition of releasing just before Christmas, beta 2 also featured full support for subscripts on lists, maps, and strings.

A month later, beta 3 brought both backwards and forwards inclusive and exclusive ranges in the collection string and map subscripting, more core Java polymorphism, the break statement, ternary expressions, and the Groovy GDK additions to the JDK. Guilaume Laforge had joined the project at that stage. As with every release of Groovy, there were many bug fixes. The most notable quote in the release notice: "Whilst the language syntax is not quite frozen for the final 1.0 release its getting very close (we hope the next release to freeze the syntax for backwards compatibility) and the projects codebase is getting stable and solid now." This was on 23 January 2004, a mere 6 weeks after the first release. A week later, the Groovy logo was chosen. Salute that green star!

Six weeks later, beta 4, weighing in at 5.2Mb, brought class imports with wildcards, default method and function parameters, I/O and process GDK methods, BigInteger and BigDecimal integration, closure currying, and the templating engine. A few days later, James, Richard Monson-Haefel, and Geir Magnusson Jr submitted Groovy for standardization to the Java Community Process as JSR-241. The project was promptly voted in, the expert group was formed on 30 March 2004, and there's been no progression during the 4 years since. The other major Java-syntax-compatible language, Beanshell, took the same path a year later as JSR-274.

The next few betas were released every two months or so, but there was no syntax freeze and final release. Beta 5 hit the web on 12 May 2004, a week after Guillaume became a Groovy despot, with parser and bytecode fixes, along with an expanding GDK. Beta 6 on 15 July 2004 had a new type inference engine for optimizing calls. Beta 7 on 29 September 2004 clocked in at 10Mb. This was, in fact, the first version of Groovy I myself used.

A New Parser

Guillaume writes that around this time Groovy development almost stalled. The developers met at DevCon1 (then called GroovyOne) in London to restart the project. This was when Russel Winder, Jochen Theodorou, and Dierk Koenig joined James, Guillaume, and Jeremy Rayner on the development team. On 17 December 2004, continuing the Christmas release tradition, with Guillaume taking over the release announcements, beta 8 arrived, all 11Mb of it, bringing lighter error messages and a better GroovyShell experience. A month later, beta 9 brought JDK 1.5 compliance and an improved GroovyServlet.

At this time, the lexer/parser was overhauled, the new "JSR" one being included inBeta 10 on 28 Feb 2005 as an "Early Access Preview", in addition to the old "Classic" one. For the first time in this announcement, Guillaume widens the announcer from just himself to "the Groovy development team and the JSR-241 Expert Group". Six weeks later, on 5 April 2005, the next beta of Groovy used the new "JSR Parser" by default, including the old "Classic Parser" for backwards compatibility only, with fairly extensive changes required to the source.

Instead of "beta 11", the new beta was called "Groovy 1.0 JSR 1" to tag it as the syntax intended for standardization. Guillaume wrote in the release notes: "We'll have a few jsr-x versions till this summer, two or three more before the final 1.0 release." In the "JSR 2" beta version on 15 June 2005, there were a record 1000 test cases, the error-reporting was much improved, and interfaces could be written in Groovy for the first time. Two months later on 16 August 2005, the "JSR 3" beta version was released. The summer ended with no final release.

The Baton Passes

Three months later, the second Groovy developers conference (DevCon2) occurred in Paris, with the 15Mb"JSR 4" beta released the week before on 21 November 2005. The release notes mentioned improved compilation and class loading, enhanced inner class imports, improved startup scripts, an upgrade to ASM 2.1, synchronized blocks, and improved namespace support for XML and builders, including quoted method names. Guillaume wrote: "The two main aspects remaining for Groovy to reach its 1.0 final milestone is to clarify the name resolution and scoping rules. Those two concerns will hopefully be addressed during the Groovy JSR meeting, and we're going to implement these rules as quickly and as thouroughly as possible. Keep in mind that those rules might be a little different than our current rules. However, we hope these rules will be more coherent and closer to what we're used to in Java."

At that DevCon 2 meeting, there was disagreement over whether or not Groovy's closures and builders should be distinct syntactic entities. Shortly after the meeting, Groovy's founder James Strachan moved on to other projects, having passed the leadership baton onto Guillaume. That year, 2005, has been the only year Groovy missed a Christmas release, the "JSR 5" beta not being released until 13 February 2006. That release brought multi-dimensional array support, calling method names defined in strings, semantic changes to "def" and binding variables, and improvements to the scoping algorithms.

Said Guillaume: "This is the last release of the JSR-xx line. The next release will be the first RC-x release before the final 1.0. We're planning to release RC-1 in about two months, and the final Groovy 1.0 release should be out in about three months. As you might know, and as decided during the last Groovy conference in Paris, the two main tasks towards the final version of the projects are the rework of the scoping algorithms, and the rewrite and enhancements of the Meta-Object Protocol. JSR-05 contains these new scoping algorithms. [...] The next step before RC-1 is the work on the MOP and name resolution algorithms." Unfortunately, the developers underestimated the time it took to complete this work on the MOP (meta-object protocol) and name resolution. RC-1 was released 10 months later.

Groovy weathered another unfortunate occurence at this time. Since July 2005, Guillaume and Graeme Rocher had been working on the Groovy on Rails project, some web infrastructure to duplicate that of Ruby on Rails. When they released version 0.1 on 30 March 2006, they changed its name to Grails, dropping "Groovy" from the name. The Ruby on Rails lead developer, David Heinemeier Hansson, had emailed saying he considered the "Rails" name to be exclusive to the Ruby on Rails project. I suspect the real concern of Hansson's email was just as much to limit the "Groovy" name as to protect the "Rails" name, and he succeeded in both aims. Perhaps at that time Guillaume and Graeme didn't fully understand the value of the "Groovy" brand in the way Jonathan Schwartz understands the value of the "Java" brand. I'm sure they've since wizened up.

Countdown to 1.0 Final

Although Groovy had clocked up 15 official releases, each better than the previous, the lead developers seemed concerned about the negative effect on marketing of not having one tagged "1.0 final". However, they didn't want to release 1.0 final without the MOP finished, so they released another beta, "JSR 6", four months later on 28 June 2006, with syntax changes for properties, class loader improvements including class initializers, and mocking for unit testing.

Then there were no official releases for another 5 months. I still remember that long wait, when I wrote so much Groovy code experimenting with interceptors in JSR-6 and sometimes wondering if version 1.0 would really ever arrive. But arrive it did on 4 December 2006. RC-1, the first release candidate, brought class coercion using the asType(Class) method, the 'in' operator, coercing Maps and closures to interfaces, and increased dynamicity using "$methodName"(*args). John Wilson, Paul King, and Guillaume Alléon had joined the development team by this stage.

On 23 December 2006 came RC-2, the only one of Groovy's 30 releases to not include the source code, just in time for Christmas. I downloaded it when it arrived, and ran my test scripts through it. Almost everything mildly complex using interceptors failed, even though they'd run OK through RC-1 and JSR-6. Without source code, I couldn't see what had changed in 3 weeks. When I tried to look at the source online, I couldn't: an earthquake had damaged the internet cables servicing where I live, and they weren't fixed for 6 to 8 weeks. Only my email account was accessible.

A week later on 2 January 2007, Groovy 1.0 final was released. On 29 January 2007 were the release parties around the world and the 3rd DevCon meeting, in Paris. By the time I downloaded 1.0 final sometime in late February, my MOP-dependant test scripts still didn't work, and the Groovy developers were talking online about a new MOP in Groovy 2.0, to be released sometime after Groovy 1.1. I think I might have moved on from Groovy at that stage if it weren't for Groovy's groovy name. But I put some time into learning and documenting for Java newbies the core JDK and GDK methods, coding that didn't require the MOP. In either 1.1 beta 1 or beta 2, my MOP-dependant scripts were working again, but by then, my coding interests had moved on.

Targeting Java 5.0

On 30 April 2007, a week before JavaOne, Groovy 1.1 beta 1 was released, with annotation use and static imports from Java 5.0, an ExpandoMetaClass, and many other smaller improvements. On 5 July 2007, beta 2 added generics from Java 5.0 to Groovy, joint Groovy/Java compilation, a ConfigSlurper, the classical C++/Java-style for loop, and named parameters without parentheses, as well as general performance. And on 20 September 2007, beta 3 brought enums from Java 5.0, coercion of maps and closures to concrete classes, the Elvis operator, closure/builder name resolution enhancements, and GroovyShell improvements.

The Groovy development during this time seems to have been the most productive ever, with estimates being roughly accurate. Of course, without the tough challenge of the MOP, the programming tasks were a lot more doable, Jochen had been working fulltime on the Groovy codebase since December 2006, and the developers have obviously learnt from mistakes in the previous years. Soon after the Groovy DevCon 3 meeting in January 2007, John Wilson suddenly left the Groovy development team. However, he's since been working on "Ng", a MOP for a Groovy-like language, which may proof to be of benefit to the development of Groovy 2.0.

On 10 October 2007, Alex Tkachman, former COO of Jetbrains, announced G2One, Inc, a company with funding from Bay Partners, a VC firm from California. G2One is a Groovy & Grails consulting and training firm, founded by Alex, Guillaume, and Graeme. Jochen also promptly joined them. A few days later, on 12 October 2007, Groovy 1.1 RC 1 was released, bringing improved performance and various extra features, such as string to class coercion and overloadable unary operators. Guillaume refered to the announcers as "The Groovy development team and the G2One company". A few days later, on 15-16 October 2007 in London, was the Groovy DevCon 4 meeting. Unlike previous meetings, this one was quite closed, the new commercial realities of running a consulting company for venture capitalists who expect profits and capital gains no doubt causing this. The following day, on 17-19 October 2007, the first ever Grails XChange was held, having been postponed from May.

On 2 November 2007, RC 2 was released, followed by Groovy 1.1 RC 3 on 28 November 2007, both performance improvement and bug fix releases. Immediately afterwards was a discussion on the mailing lists resulting in Groovy 1.1 being renamed version 1.5 for marketing reasons.

Present and Future

On 7 December 2007, Groovy 1.5.0 was released, only 11 months after version 1.0, probably the best project-managed calendar year in Groovy's history. Two bug fix and performance improvement releases, 1.5.1 on 21 December 2007 (the Christmas release) and 1.5.2 on 29 January 2008, have since followed, the latter quickly being followed by bug fixing 1.5.3 on 31 January 2008 and 1.5.4 on 1 February 2008. In the latest release notes, Guillaume attributes the recent success in bug fixes and performance increases to G2One's creation. No doubt full-time employees passionate for Groovy helps development a lot, but funded companies must make money, and clocking up chargeable hours with Grails consulting is far more lucrative than pro-bono Groovy development, so the tension between investing time and cashing in is always there from executives and investors.

The current roadmap defines a very tentative structure for future Groovy development, focusing on smaller sets of features in each release. 1.6 is slated to bring annotation definitions and multiple assignment, 1.7 to bring incremental compilation, upgrading to ASM 3.0, and AST transformations, 1.8 to bring nested and anonymous classes, 1.9 to bring Antlr 3.0 upgrading and concurrency features, and finally 2.0 to bring a new MOP, with homogenized features. Focusing on a small set of features at a time worked during 2007, so that's the best way to go. But the new MOP in version 2.0 threatens to bring back 2006-style development. 2007 also saw the timing of Groovy releases revolve around marketing events such as JavaOne and Grails eXchange, which could also impact release quality.

So how will the future releases of Groovy play out?