DSLs from the Perspective of Extensible Languages

By Didier Verna

Creating a domain-specific language (DSL) is inherently a transversal activity: it usually requires knowledge and expertise in both the application domain and in language design, two completely orthogonal areas of expertise. The difficulty is that either one needs to find people with such dual competence, which is rare, or one needs to add manpower to the project by means of different teams which in turn need to coordinate their efforts and communicate together, something not easy to do either.

One additional complication is that being an expert developer in one specific programming language does not make you an expert in language design -- only in using one of them. DSLs, however, are most of the time completely different from the language in which the embedding application is written, and a general-purpose programming language (GPL), suitable to write a large application, is generally not suited to domain-specific modeling, precisely because it is too general.

As a consequence, it is often taken for granted that your application's DSL has to be completely different from your application's GPL. But what if this assumption was wrong in the first place?

The need for designing a DSL as a completely new language often comes from the lack of extensibility of your GPL of choice. By imposing a rigid syntax, a set of predefined operators and data structures on you, the traditional GPL approach gives you no choice but to implement your application's DSL as a different language, with its own lexical and syntactic parser, semantic analyzer and possibly its own brand new interpreter or even compiler. A much less widely accepted view, however, is that some GPLs are extensible, or customizable enough to let you implement a DSL merely as either a subset or an extension of your original language. The result is that your final application, as a whole, is now written in a completely unified language. While the end-user does not have access to the whole backstage infrastructure and hence does not really see a difference with the traditional approach, the gain for the developer is substantial. Since the DSL is now just another entry point for the same original GPL, there is essentially only one application written in only one language to maintain. Moreover, no specific language infrastructure (parser, interpreter, compiler etc.) is required for the DSL anymore, since it is simply expressed in terms of the original GPL. The already existing GPL infrastructure is all that is needed.

The purpose of this presentation is to envision the process of DSL design and implementation from the perspective of extensible GPLs, and to show how the capabilities of the original application's GPL can have a dramatic impact on DSL conception. More precisely, the objective of is twofold:

1. showing that by using a truly extensible GPL, implementing a DSL is considerably simplified, to the point that it may sometimes boil down to writing a single line of code,

2. exhibiting the most important characteristics of a programming language that make it truly extensible, and hence suitable for DSL'ification. Such characteristics are most notably dynamicity (not only dynamic typing, but in general all things that can be deferred to the run-time), introspection, intersession, structural or procedural reflexivity, meta-object protocols, macro systems and JIT-compilation.