An incovenient dichotomy: specific vs. generic

Recently, I had a bit of a realisation: the LISPers of this world have a point.

But before explaining that, a question: how much of your application is really specific to your business domain? The meat of the realisation is that the answer typically is: not all that much.

This is because by the nature of applications and how they are embedded in their environments, they are not purely descriptions of the business domain but rather transformations of that domain interspersed with more “mundane” details like UX (design, style, layout, navigation, etc.), authentication, authorisation, validation, persistence, external interfaces, internationalisation, concurrency (these days: asynchrony), error handling, etc. So, however you measure the size of your application’s code base (lines of code, zipped megabytes, etc.), either a very small portion of it will actually represent your business domain authoritatively or the details of the business domain are duplicated and spread out across the entire code base.

Consider as an example a Web application to sell financial products. Such products typically come with a lot of details and business rules, such as applicability (essentially validations on who can buy a product), their actual semantics (i.e., a specification of what should happen when someone buys that product), etc., adorned with “stuff” that exists primarily for people to read. The majority of the Web application does not pertain to any of that, but is rather about guiding prospective customers in some way to the product they want/need, providing general information about the company, points of contact, etc. As for the financial products themselves: the semantics will only be executed on a mainframe backend, while the applicability would translate into some kind of questionnaire checking for that.

So, on the one hand modeling the financial products provides all kind of benefits since you can derive the questionnaire and semantics from the model. Having a DSL to do that modeling makes sense since each financial business domains typically use very specific concepts and jargon. On the other hand, creating a Web application is hardly domain-specific in any way: the number of concepts involved is quite large, but they hardly vary in essence over a broad range of applications. Of course, how these concepts are applied in a specific application should be quite restricted by means of a uniform/house style and an application architecture.

In my experience, it does not pay to create app-specific DSLs capturing this generic range of concepts. (*gasp* Did I just say: “Don’t build DSLs.”?! Yes, I did.) It is simply a lot of work: you have to determine which concepts to capture, taking into account possible extensions, changes to the technology stack, etc. You typically also need an underlying type system and powerful expressions sub-language. Then you need a generator or interpreter tying in with the underlying architecture, technology stack and application environment. This is not trivial stuff and certainly not for the general developer, even with the capabilities of modern language workbenches.

Instead, creating applications should be done through a generic application language which supports raising levels of abstractions essentially continuously. And this is where LISP comes in: not that LISP is this GAL (or should be), but we could, and should, be inspired by the concept of macros. (I say levels of abstraction, since I also realised that there is not one level to abstract over. Rather, at various points in your code there are various axes to abstract over.)

A lot of constructs in general purpose languages are essentially about raising that level of abstraction, or can be used in that way. E.g., (non-polymorphically called) methods/functions are just code blocks with a name to them, inheritance is another way to avoid duplicating code, etc. These constructs force you to raise the level of abstraction by reasoning from the GPL constructs towards the intended, or naturally-suited level of abstraction.

LISP-style macros, on the other hand, work in the opposite direction: they are effectively templates transforming some code into code that exhibits a lower level of abstraction. Provided that you can chain such transformations together to arrive at code that’s actually executable, you can vary the level of abstraction incrementally and essentially continuously. Note that I say “code”, not “data”: making a distinction there would force us to choose where data ends and where code starts – not only is that a hard topic, it’s also not relevant for our purposes!

To go back to our example: the financial products could be modeled as GAL code which is then transformed into parts of the Web application. You can view the modeling GAL code as an internal DSL. The GAL tooling could even help to expose this internal DSL as an external DSL.

So, is LISP this GAL? No. LISP is a very nice language but no amount of tooling is going to be able to general developer to become productive with it while also keeping the LISP community itself on board. In a next blog, I’ll try to say more about what this GAL and its tooling should look like.

In conclusion: specific vs. generic is a false dichotomy. Rather, we need to be able to raise levels of abstraction in a meaningful way, i.e. incrementally and essentially continuously. That way, we can leverage genericity while still embracing specificity.