RelaxNG – The Best Way To Write XML Schemas

In a previous article I explained why XML Schema 1.0 is incapable of implementing the XML ignore rule and why this means that validating multiple versions of the same document is impossible using XML Schema in the general case. In this article I look at RelaxNG and show why it can handle multiple version validation without a hitch. I then discuss how to use RelaxNG in the real world, including with Web Services and conclude that the time has come for the XML and especially Web Services community to transition from XML Schema 1.0 to RelaxNG.

Solving Backwards Compatibility with RelaxNG

RelaxNG Does Backwards Compatibility

In the previous article I showed the following XML Schema which I said was defined by Party A:

<xs:schema>
   <xs:element name="DoSomething">
      <xs:complexType>
         <xs:sequence>
            <xs:any maxOccurs="unbounded" processContents="skip">
         </xs:sequence>
      </xs:complexType>
   </xs:element>
</xs:schema>

The same schema can be expressed in RelaxNG. RelaxNG is a standard schema description format that uses a much more powerful and significantly simpler approach than XML Schema. Although RelaxNG has a XML format, I find the compact text format more readable:

start = DoSomething
DoSomething = element DoSomething { any* }

any = anyAttribute | text | anyElement
anyAttribute = attribute * { text }
anyElement = element * { (anyAttribute | text | anyElement)* }

The first thing I notice when rewriting the schema is not just how much more compact it is but also how much more straightforward the definition is. I realize that simplicity is in the eye of the beholder but what I behold is a lot simpler to me than XML Schema. I do wish that RelaxNG had a built in definition for 'any' but adding in three lines that I can import into any RelaxNG schema isn't that big a deal.

The next step is where RelaxNG really shines. In my original article I had said that party B came along and wanted to use the xs:any to define an extension to the schema. Their extended message would add the brand new "DidSomething" element whose value is an integer, the message would look like:

<DoSomething>
   <DidSomething>123</DidSomething>
</DoSomething>

The rest of the article then explained why it was impossible in XML Schema to define a schema that said, in essence, "Validate any schema with the root element DoSomething, ignore the contents of DoSomething unless it contains the element DidSomething in which case make sure there is only one element, DidSomething, and that it contains an integer." Put another way, the previous sentence describes a situation where a schema can both validate all messages that are legal according to Version 1 and Version 2 of a schema. This capability is critical for backwards compatibility. Unfortunately, XML Schema 1.0 can't support backwards compatibility. In my previous article I suggested a technique that would enable XML Schema 1.0 to support backwards compatibility but it requires introducing a two stage processing path.

RelaxNG has a much simpler solution to the problem, it supports backwards compatibility out of the box. Here is the schema in RelaxNG that cannot be written in XML Schema:

start = DoSomething
DoSomething = element DoSomething { resAny* & DidSomething? }
DidSomething = element DidSomething { xsd:integer }
resAny = (anyAttribute | text | resAnyElem)
resAnyElem = element * - DidSomething { any* }

The previous schema is actually much more powerful than what I had tried to do in XML Schema 1.0. In my previous article I had just tried to get XML Schema to validate that DoSomething would only contain either one DidSomething element or an unlimited number of other elements. But what the previous RelaxNG schema actually does is say 'DoSomething can contain any element in any number but DidSomething. In the case of DidSomething there can only be 0 or 1 instances of DidSomething and DidSomething must contain an integer.' In other words, the V2 schema can itself be extended into a V3 schema because RelaxNG is powerful enough to allow for any arbitrary element to be inside of DoSomething and still enforce the restriction that if there is a DidSomething element then it must appear exactly one time and contain an integer. In other words, in my previous article I just tried to get from V1 to V2 using a technique that could never let there be a V3. With RelaxNG I can support an unlimited number of versions using the features provided natively in the schema language. XML Schema 1.0 just can't express semantics like that.

The reason RelaxNG is more powerful than XML Schema 1.0 is that RelaxNG does not require that the schema be unambiguous. When it is processing a XML instance RelaxNG will keep on going as long as it can find some rule to apply. In the case that there are two or more rules that could apply RelaxNG will follow all possible rules until it determines which rule fits 'best'.

Bonus Points – RelaxNG and XML Namespaces

In my previous article I hadn't talked at all about XML namespaces because XML Schema's handling of XML namespaces is a real mess. But RelaxNG can handle namespaces without a problem. For example, let's imagine that party A and party B both work for the same company and define elements in the exact same namespace. The backwards compatibility issues raised by this situation cannot be handled at all by XML Schema 1.0. The reason is that XML Schema 1.0 can only express 'any' semantics across an entire namespace. But RelaxNG is a lot more flexible.

namespace company = "http://example.com/thecompany"
start = DoSomething
DoSomething = element company:DoSomething { resAny* & DidSomething? }
DidSomething = element company:DidSomething { xsd:integer }
resAny = (anyAttribute | text | resAnyElem)
resAnyElem = element * - company:DidSomething { any* }

RelaxNG is able to trivially express the semantics that DoSomething can contain any element from any namespace, including the namespace DoSomething itself is defined in, except for the DidSomething element which is defined in the same namespace as DoSomething. In that case other restrictions apply. In other words, unlike XML Schema 1.0 which can only describe extensibility behavior at the namespace level, RelaxNG can describe extensibility at the element/attribute level. Note, however, that RelaxNG can also deal with namespace extensibility across entire namespaces. For example. resAnyElem could also have been defined as "element * – company:* { any * }". This would prevent any extension in the company namespace from being used.

RelaxNG is pretty awesome but not perfect

Having the XML world switch to RelaxNG would be an unambiguous good. Even if XML Schema 1.0's backwards compatibility issues were fixed the syntax and model is still painfully complex and likely to remain a permanent hindrance to average XML developers having any chance of defining and owning their own non-trivial schemas.

Still, RelaxNG is not perfect. It only provides indirect support for an open content model which forces RelaxNG authors to perform some fairly unpleasant contortions such as defining the resAny production rule above. Minimally it would be nice if RelaxNG introduced a formal open content model. That would enable one to write extensible schemas without having to shove "any" rules everywhere and without having to produce custom any rules such as resAny.

For all I know there already exists a shortcut that takes care of the open content problem. I'm still pretty new to RelaxNG. I hope to find out that there is an even easier way to do what I did above. But even if there isn't, it's clear to me at least that RelaxNG on every level is superior to XML Schema 1.0.

Using RelaxNG in the Real World

There already exists a lot of RelaxNG software, so programmers can start to use RelaxNG immediately.

For programmers who just want to validate a XML instance there are RelaxNG validators for just about every platform out there.

But where RelaxNG really shines is when it is used for sophisticated XML processing. Two programs in particular show what one can really do with RelaxNG.

The Relaxer project allows one to take a RelaxNG schema as input and to output, well, tons of stuff. But one option I find of particular relevance is creating a relaxer object which is a series of Java classes that represent the underlying XML as described by RelaxNG. Essentially one gets setter/getter methods that let one navigate through and create XML objects based on a RelaxNG schema.

There is another project called RelaxNGCC whose approach I really like but they don't seem to support ambiguous schemas which rather defeats the whole purpose of using RelaxNG in the first place.

If one is just doing stand alone XML parsing then switching to RelaxNG is as easy as writing a RelaxNG schema. There are also tons of conversion tools available that can turn existing XML Schemas into RelaxNG schemas. Although it really isn't necessary there are also RelaxNG enabled editors available.

One area where using RelaxNG can be a little challenging is Web Services. Most Web Service stacks take a WSDL as input and generate a bunch of native interfaces to enable sending and receiving messages based on the XML schema 1.0 definition inside of the WSDL. This would seem to leave no place to use RelaxNG but there is a work around.

Just about every platform out there supports receiving/sending Web Service messages as DOMs. This provides the foundation for using RelaxNG in web services. The trick is to write one's WSDL using xs:any's as the contents of all the message types defined using XML Schema. Feed this WSDL to the web service platform and then write the RelaxNG schema and feed it to Relaxer to get back the appropriate Java objects. So long as Relaxer is given the right command line arguments it will output classes that can both initialize its data structures using a DOM as input and output a resulting data structure as a DOM. The DOM classes become the interchange format between the Relaxer generated classes and the underlying Web Services stack.

I realize that moving DOMs around isn't exactly the most efficient way to handle things but I was amazed to see just how often people use DOMs for Web Service message processing so performance must not be that big an issue. In any case, especially for Web Services that can handle higher latency, the DOM input/output mechanism allows one to use RelaxNG with existing Web Services systems.

Conclusion

The full story of how XML Schema 1.0 got created really needs to be written. I had a pretty good view from the sidelines but not good enough that I feel comfortable telling the story. But the short version is that it involved a lot of politics and a lot of people, myself included, who didn't have the resources to pay attention when they should have been. The result is a standard that I believe is unacceptably complex, fails to meet even the most basic needs of the Web Services community and has held back the development of XML messaging.

Although there is talk of fixing some of XML Schema 1.0's flaws in the 1.1 effort I believe that those efforts will almost certainly be too little, too late. At some point a community has to recognize that it has gone down the wrong path and have the courage to change course. That point should have come several years ago with XML Schema 1.0 but for a number of reasons, most having to do with the immaturity of Web Services, it didn't happen. So here we are, several years and many millions of dollars later heavily invested in a technology that I believe can't meet the majority of our needs. I find it especially ironic that XML Schema 1.0 is so limited that the core Web Services standards, SOAP and WSDL, both can't be described using it. Even new up and coming efforts, like BPEL, and old standards like WebDAV also find that XML Schema just isn't up to the job.

What is even more ironic is that the majority of schema experts I know agree that XML Schema is the wrong choice. Even if the extensibility issues were fixed the complexity issues make it too hard to use. Every major Web Service company I know of has looked seriously at moving to RelaxNG. But in the end they all ran away because the cost was huge and customer reaction was uncertain. Who wants to spend an enormous amount of money revamping entire infrastructures to support a new technology when there is an existing technology that although flawed is accepted by most customers?

I suspect that the change will only occur when either customers rise up in revolt (something that is already starting with more advanced customers) or when companies without existing XML Schema investments are able to build more compelling messaging design, management and evolution infrastructures using RelaxNG.