Creation Review

Building the runtime technology for a new language to match the capabilities of existing mature languages typically requires tremendous effort over decades and, in some cases, never happens because language adoption rates never justify the needed investment. But many of the technologies required are actually not substantially different than the technologies that have been created for existing languages. There are always quirks and peculiarities for each language, but the fundamental technology is really very similar in nature. What makes it extremely difficult to repurpose existing technology for a new language, however, is that the effort to create a new language runtime typically focuses almost entirely on the shortest path to becoming operational for one particular language. “Shortest path” typically means specializing the technology for that language which impedes reuse for other languages. This process has already been repeated many times for many different languages, resulting in several challenges that affect all communities to varying extents:

1. Opportunity cost : every community invests limited resources to independently implement and maintain code that is broadly similar in capability but expressed in different ways. How much more would we all accomplish without this wasted effort?

2. Long robustness ramp: different implementations tend to run into and fix similar kinds of bugs over their lifetimes. Early design flaws can become extremely restrictive and hard to fix as the community grows around a runtime implementation

3. Slow capability adoption: hardware and operating system capabilities take much longer to become consistently available and, in the meantime, the developer community can be be disadvantaged on some platforms

4. Hampered productivity: frameworks for development, diagnostic, profiling, monitoring, management, deployment, testing, etc. require much more effort to build and maintain across many languages or we build broadly similar (but different) tools for each language (see #1)

5. Barrier to entry: the more hardware, operating system, and tools become popular the harder it becomes to get a new language to be fully capable. Not all language designers necessarily want to become experts building these capabilities.

6. Slow forward progress: slower innovation in languages, possibly even foiled in some cases by significant runtime implementation costs

One approach to improve this situation is to build other languages within an existing mature runtime environment like Microsoft’s Common Language Runtime or a Java Virtual Machine. For example, Scala, Groovy, jRuby, Nashorn, and many other language projects leverage the Java Virtual Machine (JVM) to run code written in other languages. None of these projects, however, have become the de facto implementations for their target language, in part because the JVM is primarily designed and continues to make implementation trade-offs so as to run Java code very efficiently but not necessarily other languages. The implementation trade-offs needed to support other languages require and encourage workarounds and complexity that would not be needed were it not for the fundamental design constraints (i.e. the “Java-ness”) of the JVM itself. For reasons like this, most languages have a native C or C++ runtime implementation that is considered the reference implementation for the majority of that language’s users. More significantly, however, the success of this kind of approach depends on migrating a community from one runtime implementation to another, across what can be a significant number of implementation differences that manifest for developers and users as varying forms of “my program doesn’t work the way it used to work”. To date, very few large language communities have been able to succeed with this scale of migration.

A second approach could be to build new runtime components from scratch that are designed from the outset for reuse. But building even one runtime for one specific language is incredibly hard. Building such componentry to support any runtime but without any specific stakeholder (while conceptually every stakeholder) is almost guaranteed to fail.

Neither of these two approaches seems like a sure bet, but the idea to leverage a mature JVM’s core technology feels like the best direction. The JVM technology already exists and has proven itself for at least one mature language community. But bolting other languages on top of Java semantics has not yet shown to be a broadly viable solution.

Instead, we propose to reorganize the runtime components of an existing commercial JVM implementation (the IBM Developer’s Kit for Java) to separate the parts that implement Java semantics from the parts that provide key runtime capabilities.

The OMR project will be formed around these latter language independent parts: a runtime technology platform consisting of core components as a toolbox for building language runtimes. An ecosystem of developers working together to augment the capabilities of this platform while collaborating with developers for tools and frameworks simultaneously fosters industry-wide innovation in managed runtimes, the languages they implement, and the collection of frameworks and tools that will accelerate our industry’s ability to build even more amazing things.