Standards & Reference Implementations

In a previous blog entry, I spoke about the relationship between Standards and Open Source software, concluding that “open source standard” was an oxymoron.  One topic that I did not discuss was the relationship between a Standard and the so-called “Reference Implementation” (“RI” for short) of that Standard.  This relationship is not as straightforward as it might first seem, and is, in fact, the subject of important discussions in several of the EDA Standards groups with which I am currently involved.   Because of this, I have been doing a lot of thinking about the issues surround RIs, and hope the following can help focus those discussions.

First the basics:  an RI is a piece of software that is associated with a Standard.  This “software” need not be code that can be compiled into an executable program—the RI that the SPIRIT Consortium developed for their IP-XACT Standard is text-based xml.  Nor does an RI necessarily come from the same group that developed the associated Standard—for example, the Fraunhofer Institute, not OSCI, developed the RI for SystemC AMS.  Finally, an RI need not be made available under an open source license—the very successful Si2 Open Access RI, for example, is available under a non open source license.

The allure of an RI is not hard to see.  It provides a concrete embodiment of a Standard, i.e., something that both users and tool developers can sink their teeth into— something besides  a “mere”  precise (and non-executable) specification.  Indeed, on multiple occasions (including in the past two weeks) I have heard sophisticated users declare that a Standard’s RI is the only item in which they truly are interested.  Often the users who take this position summarily dismiss the actual Standard as being only a “paper spec”.  Other users understand the power of a true Standard, but also value an RI as a means of facilitating early adoption of a Standard before it is implemented by vendors.  Everyone, or at least almost everyone, seems to love RIs.

However, all that love aside, every Standards organization makes it clear that when one of their specifications and an associate RI conflict, the specification, i.e., the Standard, takes precedence.  In other words, the RI is an accompaniment, not the main course. But what sort of accompaniment is an RI?

There are, as far as I can see, three possible relationships between an RI and the Standard to which it refers:

1. Lockstep—the Standard is always released with an accompanying RI, and each RI is designed to fully embody the Standard with which it is released.  In this case, the Standard and the RI can be viewed as two sides of the same coin, with the usual caveat that the Standard takes precedence in case of dispute.
2. Proof-of-concept—the RI is meant to make new standardized technology more approachable, i.e., to prove the Standard’s relevance/usability.  As this standardized technology becomes mainstream, its proof-of-concept RI may be allowed to “get rusty”, or may even be discontinued.
3. Snapshot—the release of Standard and its associated RIs are decoupled.  Periodically the group that standardizes technology and develops associated RIs takes a “snapshot” of the current RI, and produces a formal specification that embodies the new version of the Standard.  Additional (backwards compatible) RIs that add onto the RI captured in the latest Standard are subsequently developed and released.  Eventually, the group will take a snapshot of one of those new RIs, and after appropriate formal documentation, an updated standard will be released. The cycle will then repeat.

A Standards group may decide to not have any RI associated with its Standards—the IEEE Design Automation Standards Committee (and, I believe, the entire IEEE Standards Association) works this way.  However, if a Standards organization’s working groups are permitted/expected to produce RIs, then that organization’s governing body must decide what the relationship is to be between the generated Standard(s) and the RI(s).   Nothing good can happen, for example, when a Board of Directors is split between those who expect a lockstep relationship between its Standard and RI, and those who favor a proof-of-concept relationship.  One part of the group will expect that any released Standard will be accompanied by a comprehensive RI, while the other faction will expect a retirement plan for any RI as its associated Standard goes mainstream.  Similarly, warning flags need to be raised if a working group is chugging along assuming a snapshot model between the Standard and RIs that it will produce, while the organization’s Board of Directors is still assuming a lockstep relationship.  There are advantages and disadvantages to each Standard/RI relationship, but there are no advantages to not settling on one such relationship and sticking with it.