Using Xtext’s Xtend(2) language

The 2.0 release of Xtext also saw the introduction of the Xtend language. “Wait a minute, didn’t that already exist in Xtext 1.0 and openArchitectureWare 4.x before that?” I hear you saying. Indeed, the new language is should actually be called Xtend2 but since it’s meant to replace both Xtend(1) and Xpand, the makers have opted to drop the 2 suffix and assume that you’d know the difference. In any case, the languages differ in the file extensions used: .xtend for Xtend(2) and .xpt/.ext for Xpand/Xtend(1). Xpand and Xtend(1) are still part of the Xtext distribution, apparently for convenience, backwards compatibility and easy of migration, although there’s no support on these languages and execution engines any more. I also noted that the Xtext generator still relies heavily on Xpand/Xtend(1).

I’ve been using Xtend (i.e., Xtend2, but I’m going to drop the ‘2’ from now on as well) for some time now as a replacement for Xpand/Xtend -and even JSP- and I wanted to share my experiences and impressions -mostly good- with you and discuss a number of differences between Xpand/Xtend and Xtend.

The good

Xtend provides a decidedly functional programming-flavored abstraction over Java. Xtend files are compiled down to Java source code on-the-fly (courtesy of an Xtext builder participant – more on that later). The pros of this approach are performance, integration and debuggability using ye ole’ Java debugger. The generated Java code is fairly readable and 1-to-1 with the original Xtend code so tracing back to that is not really difficult, although full traceability would have been a boon here. I’ve never gotten into the groove of Xtend(1)’s custom debugger, preferring println()-style debugging over it. Compilation also means that you can refer to the compiled Java classes from outside of Xtend. It’s even possible to use Xtend as a type-safe replacement language for JSP.

The rich strings are by far the biggest improvement over the old Xpand/Xtend combination: the intelligent white space handling is nothing short of brilliant. They bring templates into the functional realm: a template is now “just” a function, so you can freely mix templates and “regular” code as it makes sense. Don’t forget that a juxtaposition of rich strings evaluates all but only returns the last, though – there’s no automagical concatenation.

It’s extremely easy to hook into the Xtext builder mechanism using a builder participant. In fact, this is the out-of-the-box situation so you only have to open the generated <MyDSL>Generator.xtend and implement the doGenerate function to have your UI language plug-in automagically fire up that generation on projects with the Xtext nature. Since the Xtext builder  works in an incremental manner, the generation is only triggered for the files which have been touched or which (transitively) depend on it, whereas it used to be “the whole shebang”. If you factored and modularized your language in a sensible way, this means that turnaround for the users of your generation is much, much quicker.

The extension mechanism works just as in Xpand/Xtend. Both the dispatch and create semantics are just a bit more general than their Xpand/Xtend counterparts which is good. I also like the as-syntax for casts: because of type inference, the end of an expression feels like a better place for the cast and it support the usual thought process (“Oh wait, now I have to cast this too!”) better as well.

The bad^H^H^Hnot entirely fortunate

To be fair: I’ve only found a few things less fortunate and they are definitely not show stoppers – they all have reasonable workarounds or are quite minor to begin with. But, since this is the interWebs I’m going to share them with you regardless 😉 You might run into them yourself, after all.

The biggest thing is that Xtend derives its type system directly from the Java type system. Xpand/Xtend had a type system which allowed you to compose various type systems, with the EMF and JavaBeans type systems being the most widely used after the built-in type system which came with a whole lot of convenience for the common base and collection types. In Xtend, you essentially only have an improved JavaBeans type system so you’ll have to rely on what Java and its libraries offer you or else you’ll have to knock it out yourself.

In particular, the following useful features of Xpand/Xtend are missing:

• «EXPAND … FOREACH…» construct. This means that you find yourself typing«FOR item : items»«item.doSomething»«ENDFOR»quite often, especially since it’s often quite natural to factor out the contents of the loop. The workaround for this consists of a utility function-replacement using closures but the result is slightly less aesthetically pleasing than it used to be.
• Syntactic sugar (shorthand) for property collection. “collection.property” was Xpand/Xtend shorthand for “collection.collect(i|i.property)“. In Xtend neither are possible and you’ll have to use .fold or .reduce to achieve the same and the equivalent code is nowhere near as readable as its Xpand/Xtend counterparts.
• Using this as a parameter name. It was perfectly alright to use this as a parameter name in the definitions of Xtend functions and Xpand DEFINEs. It was even OK to say «FOREACH items AS this»…«ENDFOREACH». Since “property” is shorthand for “this.property” (it still is) this allowed you to create extremely succinct code. The User Guide mentions that it should be possible to re-define this but I couldn’t get that to work, so you’ll have to qualify all object access with the name of the parameter or variable.

On the IDE side of things, I miss Java’s automatic ordering and cleaning of imports. Also, the content assist on Java types in the Xtend editor doesn’t do the “captical trick” where “DCPF” expands to “DefaultConstructionProxyFactory”, e.g..

Lastly, Xtend doesn’t offer control over visibility of features in the generated Java and also doesn’t support abstract, static or (non-default) constructors. This has led me to use Java for building APIs and implementing the typical abstract/support part of it and Xtend for the “final” implementation – which tends to benefit most from the succinct syntax and features such as closures and rich strings.

The ugly

Again: I’ve only found a few of these and none of them show stoppers.

First of all, the compilation to Java does not always yield compiling code. (This is in full compliance with Joel Spolsky’s Law of Leaky Abstractions, since Xtend is an abstraction of Java.) Although the User’s Guide mentions that uncaught, checked exceptions thrown from your code are automatically wrapped in unchecked exceptions, this is not the case: instead, they are added to the throws-clause of the containing methods (corresponding to an Xtend function). This can break an override function or it can wreak havoc on code using this function. I’ve found myself writing a lot of try–catches to cope, which detracted from the otherwise quite succinct syntax quite a bit. Obviously, I would have had to write these anyway if I were using Java, but that’s not the point of Xtend, I think. (To compound the matter: you can’t explicitly declare a throws-clause in Xtend.)

Also, generic types involving wildcards (‘?’) are not water tight, although it’s fair to say this is a major problem with the Java type system in general and often enough extremely hard to get right. Not using wildcards is almost always possible, so that’s the obvious workaround.

Conclusion

All in all and despite my few, slight misgivings, Xtend is quite an improvement over Xpand/Xtend. I’d heartily recommend to start using it, both as a replacement to Xpand/Xtend (and JSP) as well as to Java itself.