Nothing said in this post (or on this blog for that matter) reflects the views of the blog owner's employer of the moment.
Dedication: To George Lucas for as Harrison Ford once said to Mr. Lucas, "You can type this shit, George, but you sure can't say it."
Announcer: Join us for the exciting first part of episode 6 of the Interoperability Wars – Revenge of Babble!
Queue to opening music.
Focus on Emperor Babble's private chamber.
The Emperor standards quietly completely hidden in his black robes while Darth Sudsy, covered in his black carapace, leads in Luke Restafarian in a battle torn uniform, long dreadlocks dragging and heavy fatigue evident in his face and stance. Guards stand by the door at stiff attention.
Opening music fades away.
Darth Sudsy: As you commanded my emperor, we have sacrificed General Weasdel in order to capture Luke Restafarian.
Emperor Babble: Excellent, Weasdel's death will serve us well in lulling the forces of interoperability into thinking they are making progress. Welcome Restafarian, it is time you understood your true place in my plans.
Luke Restafarian: Do what you want to me emperor, but the noble cause of interoperability will prevail!
The Emperor turns to the center of the chamber where a form, almost blinding in its clarity, takes shape:
GET /someuser/profile HTTP/1.1
host: example.com
content-type: application/xml
<profile xmlns="http://example.com">
<professional>
<workTitle>...</workTitle>
...
</professional>
<personal>
<spouseName>
...
</spouseName>
...
</personal>
<clothingPreferences>
<favoriteColors>
<shirts>...</shirts>
...
</favoriteColors>
...
</clothingPreferences>
...
</profile>
Darth Sudsy is momentarily taken aback from the appearance of the pure interoperable data while Luke Restafarian seems strengthened by it. The Emperor turns back to Luke and then speaks.
Emperor Babble: I see it strengthens you Restafarian, no matter. But look again at your blessed interoperability.
When the Emperor turns back to the form the form begins to morph, growing darker and more sinister:
GET /someuser/profileFeed HTTP/1.1
host: example.com
content-type: application/ATOM+xml
<feed xmlns="http://www.w3.org/2005/Atom">
<title/>
<updated>2000-01-01T00:00:00Z</updated>
<author>
<name/>
</author>
<id>http://www.example.com/someuser/profileFeed</id>
<category scheme="http://example.com/categories" term="Profile"/>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/professional</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<workTitle>...</workTitle>
...
</content>
</entry>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/personal</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<spouseName>
...
</spouseName>
...
</content>
</entry>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/clothingPreferences</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<favoriteColors>
<shirts>...</shirts>
...
</favoriteColors>
...
</content>
</entry>
</feed>
Luke, having recognized the syntax, clearly expects to get another wave of strength but instead he feels sickly. The emperor, looking slightly stronger, turns from the form to look at Luke.
Luke Restafarian: What have you done? That structure is clearly taken from Princess Ape-Pea's system, she is a true follower of interoperability, I should be getting stronger but somehow it's making me ill.
Emperor Babble: You begin to understand young Restafarian. Used properly Princess Ape-Pea's system does indeed honor all the qualities of rich interoperability. But look more carefully at this particular example of her system. Is it not beautiful? Its needless complexity, its useless elements, its bloated form, they herald true incompatibility. No developer will be able to manually deal with such an incomprehensible monstrosity. We have taken your pure interoperable data and hidden it in a mud of useless scaffolding. Princess Ape-Pea and your other minions will accept this poison as adhering to your precious principles of interoperability but in point of fact by turning Princess Ape-Pea's system into a generic structured data manipulation protocol we have forced the data to contort into unnatural forms that are so hard to read, so difficult to understand that no programmer will ever deal with it directly. We will see no repeat of those damned Hit-Tip Knights building their own stacks in a matter of hours in order to enable basic interoperability. In this new world even the most trivial data visualizations and manipulations become a nightmare. Instead of simple transversals of hierarchical structures programmers will be forced into a morass of categories and artificial entry containers. Behold!
The emperor, without look away from Luke waits while the form morphs into an even more sinister and darker shape:
GET /someuser/profileFeed/hobbyFeed/hobby1 HTTP/1.1
host: example.com
content-type: application/ATOM+xml
<entry xmlns="http://www.w3.org/2005/Atom">
<title/>
<id>http://www.example.com/someuser/profilesFeed/clothingPreferences/favoriteColors</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<shirts>...</shirts>
...
</content>
</entry>
Luke looks visible ill.
Emperor Babble: Is its complete lack of symmetry not enthralling? Imagine trying to write a program to interact with an element, depending on how the element is downloaded the entire data structure enclosing the element changes. One form when presented in the context of its parent and another when directly accessed. Even better the meta-data changes, sometimes the entry and its values are available and sometimes not. The inconsistencies make direct data manipulation impossible. Even data modeling will become a nightmare.
Luke Restafarian: (struggling for breath) You… will…. not….win!
Emperor Babble: Poor boy, you still do not understand. Of course I will win. Thanks to Princess Ape-Pea we will turn your interoperablity principles against you. We will create an entire industry of blog pundits to provide advice on the right way of shredding data to fit into Princess Ape-Pea's system. The complexities will become so large that not only will no one be able to directly interact with the data using simple hierarchical data processors but each data producer will create its own custom libraries in order to translate its information out of this morass into something more easy to operate on. Arbitrarily interoperability will become impossible, any data consumer wanting to talk to a particular data producer will first have to download custom libraries from that data producer, assuming the libraries are even available in a language the data consumer supports. The code maintenance and incompatible interface problems will bring interoperability to a halt and only a few giant data providers will survive. Diversity will die. Interoperability will die.
Luke Restafarian: Princess Ape-Pea knows the limits of her system and understands that it is not intended as a generic structured data manipulation protocol! She will never help you!!
Emperor Babble: (softly chuckling to himself) Perhaps dear boy, perhaps.
The Emperor signals to the guards who roughly carry the now very weak Luke Restafarian away. The door to the chamber shuts.
Darth Sudsy: Is the boy not right my emperor? Surely Princess Ape-Pea will see through this transparent bid to corrupt her system?
Emperor Babble: My greatest apprentice, Darth Goo-Goo-L and his general G'Day-Tah have already successfully launched my plan. Princess Ape-Pea and her minions have made some perfunctory complaints about General G'Day-Tah's use of her system but otherwise have happily acquiesced. G'Day-Tah can now rely on Princess Ape-Pea and her allies to hound anyone who doesn't use her system for moving around generic structured data. The Princess hasn't even noticed that G'Day-Tah himself almost always avoids discussing the protocol and instead immediately pushes programmers to take a dependency on his libraries. This is inevitable due to the complexities that shoe horning hierarchical structured data into the Princess's system requires and yet the Princess suspects nothing.
Darth Sudsy: I have fought the Princess in the past my emperor, she is a formidable opponent, even if she has made a temporary error certainly she will eventually understand her mistake?
Emperor Babble: And now you understand Darth Sudsy why you will never take my place. For it is Darth Goo-Goo-L who, having defeated my previous leading apprentice Darth Misfit, has now mastered Darth Misfit's greatest power.
Stunned, Darth Sudsy takes a step back. His mask shows no expression but the fear and shock are obvious from his body language.
Darth Sudsy: (quietly, recognizing how badly outclassed he really is) No.
Emperor Babble: Oh, but yes, my nearly worthless apprentice. That is right. Princess Ape-Pea and her minions, Luke Restafarian and the rest do not stand a chance. For Darth Goo-Goo-L has mastered the ancient unstoppable power of Embrace and Extend!
Queue to Closing Music, played quietly enough to hear the announcer
Announcer: Will Princess Ape-Pea see the obvious danger in time and resist the urge to allow her system to be corrupted into a general purpose structured data manipulation system or will she succumb to the awesome power of Embrace and Extend? Tune in next week when you might hear Princess Ape-Pea say "So O-Auth-P-O is not our ally but in fact an evil agent planted by Emperor Babble to force upon us an unnecessarily complex third party trust transfer protocol that requires out of band on-boarding and so throws up needless road blocks to data sharing?"
Queue to Closing Music at full volume.
Roll Credits.
[I have nothing to add here because I’m not knowledgable enough to have an opinion but I can’t wait to see what replies are posted. It seems the only way to get notified of replies is to post a reply myself hence feel free to remove this comment later. Hope you don’t mind Yaron!]
O! M! G! Brilliant
I don’t mind at all. In fact I have several times tried to find an extension for wordpress that would update my feed when a post received a comment but I haven’t found anything that worked yet.
Nicely put, but a bit of a strawman. Why would anyone want to wrap the kind of data in your personal profile example in Atom?* There are plenty of places where it would make more sense to use XML (or other representation) directly. But where there’s content+metadata in the general form of Atom (timestamped microcontent is ubiquitous in the blogosphere), it makes sense to use Atom as a wrapper than making up new descriptive markup. The XML structure is a bit of a red herring, the underlying data model doesn’t need to be hierarchical (in the general case, by definition web data isn’t), personally I’ll be looking at such data using graph-oriented RDF tools.
Even where the content can potentially support the features Atom provides, it can still be neater to use Atom – check O’Reilly’s CodeZoo with DOAP (as RDF/XML) conveyed using Atom.
* One reason people might want to put arbitrary data in an Atom wrapper would be so it’s viewable (transferable, storable) using regular Atom tools. Personally I’m not so sure of the net benefit here, it may well encourage inappropriate shoehorning.
You should try co.mments.
Works well with no intrusions.
That was stunning. I can’t wait to see some of Princess Ape-Pea’s minions respond.
Danny, I basically agree with you. The whole point is – don’t do this. Unfortunately GDATA is, I believe, taking us down exactly this path and I know (but alas can’t talk publicly) about other efforts going in exactly the same direction.
That’s what I’m trying to warn against. If it all works out then much like the hype about Y2K (which turned out not to matter because the actual problems were fixed so nothing substantial went wrong) this article will be seen as much ado about nothing. I can only hope.
Antoine, co.mments seems like a great way to deal with broken blogs like mine that don’t provide notifications in the RSS/ATOM feed for when new comments have been added to an existing article. But rather than forcing everyone to use a new service I would rather just fix my blog. I just don’t know how to do that on WordPress.
Darrel, thanks for the vote of confidence! Also Addy, thanks for your vote and of course a thanks to Dare for his post and to Stefan Tilkov for his post and Sam Ruby for his post. I responded to Sam’s post on his blog.
Yaron,
You are basically making the same point as a few months ago, right? It looks you want to fit a truck engine into a motorcycle. That might work but will look clumsy. Here is a thought, you could have decided to use your truck engine in a truck, or maybe design a motorcycle engine in the first place.
The point being if Atom does not fit your schema, it does not mean Atom is faulty. Atom never pretended to be the format to rule them all. I can’t believe after so many months you are still on that. If your services can’t use Atom fine, XML is not constrained to Atom AFAIK and you are free to openly say “We can’t blend our schema and Atom” and then move on.
A year and a half ago I made a tongue in cheek post about MS’s doing the same thing to atom:
http://hughw.blogspot.com/2006/04/slightly-more-complex-list-extensions.html
Sylvain, this isn’t about my scenarios. I’m not using APP for those. This is about other people who have scenarios I feel are very similar to mine but who are choosing to use APP anyway. I believe they are making a mistake and I’m trying to determine if there is community consensus that using APP for these types of scenarios is an anti-pattern.
I would note, for example, that your comment is one of the only ones that just flat out said that this isn’t a good way to use APP. Most of the responses that addressed the scenario tried to find ways to recast the situation so it would be appropriate for APP.
APP isn’t perfect for everything and it would be good if we could get some consensus on what not to do with APP.
Alright, that makes much more sens to me now. I really had missed you were trying to warn about abuse to the protocol and how it doesn’t fit every scenarios.
You are definitely right, there will be many instances where AtomPub is just the wrong technology to chose. However, if the mismatch is low enough, I believe it can still be a good idea to use Atom.
But it’s quite likely that we see appearing extension that show a design problem somewhere but we can’t help that. AtomPub and Atom are great but there is also much hype around them that some so-called IT agencies will try to push forward as the new swiss army knife. A bit likely like SOAP.
Yaron, cool!
Maybe I missed this – but why wouldn’t it be modeled such that each user is content in an entry which can be modified via the APP? Then the collection would be all the users and you can add search via OpenSearch.
Dan, I don’t think you missed anything at all. The question comes down to – is all the extra complexity and overhead of ATOM a price worth paying to model data directly? Does ATOM add any real value in those cases? Sure, you can use APP to update entries if you use ATOM to wrap them, but so what? What does APP provide in those cases over a PUT or POST directly on the data itself?
BTW, isn’t OpenSearch essentially a competitor to ATOM in that it has a format that clearly could be expressed in ATOM? It seems to me that OpenSearch should just be an extension of ATOM. Also, I thought it is only intended for reading content, not writing it.
While the embrace-extend scenario is entertaining (and worrying!), examples of GData with curl look nothing like Babble’s dark form.
Reality check?
Tobu, the examples in that link use Picasa. Using ATOM to point at lists of things (like Photos) makes a lot of sense. Although I think Google’s use of ATOM in GDATA is really nasty (e.g. shoving data into entry headers instead of into content, etc.).
I think Opensocial is a better example of the problem. There they use GDATA to contain social information. They aren’t pointing at anything. Rather they are collecting together what could be really trivial XML structures that have been bloated all out of recognition by shoe horning them into Feeds and Entries. This is exactly the sort of thing Babble thrives on. Needless complexity.
I’m just starting to get immersed in this stuff, but it seems like there’s a slippery slope one gets on that starts with “I’m going to publish little blog-like entries” that turns to “I’m going to add some simple custom metadata to my entries” to “I’m going to add lots of potentially complex custom metadata to my entries” and ends up as “my entries are pretty much all data”.
At the bottom of this slope the Atom baggage seems pretty heavy, but my current theory is that the benefits of generic content management outweigh the benefits of a lightweight and tidy schema for the data-only stuff.
Compared to the issues with XHTML+RDF or the ambiguities and adhockery of microformats, the GDATA approach doesn’t seem that nasty to me. But hey, I’m just a tourist here so far. :-)