Thursday 18 June 2009

REST, Transactions, Digital Ecosystems. What gives?

...and the pushback continues. This time the argument is: use REST for the simple stuff, and leave everything else for the big (WS) boys. Don't taint REST's pretty little head with complex transaction stuff. This 'just use both, to each his own' philosophy has always smached of 'pretend wisdom' to me. If I understand this position correctly, it advicates maintianing two software stacks and two APIs within the same system, and dealing with their impedance mismatches at various levels.

The other form of the same argument is 'REST for small, RPC for large'. In a static world this could make some sense. In a world where startups are expected to continuously improve their products and can grow exponentially within months, how is any sane architect going to recommend using an approach that they know they will have to fundamentally replace at some threshold of growth? And how is this a better solution than just using REST for all your needs? If REST is the better architectural style for the job, then it should be able to compete an all use cases, not just the simple ones. If not, there may be an argument for making your own architectural style, which I suspect is not the case for the overwhelming majority.

What I frequently encounter is that critics of REST transactions don't seem to acknowledge the use cases that lead some to pursue such capabilities. To illustrate what brought me to transactions, let me add some personal background.

As you may have read, I am involved in a line of research called 'Digital Ecosystems'. While many definitions and endless documents have been written for this little niche of the research world, what I find vital in a digital ecosystems is the emphasis on fully decentralised networks, the capability to make available, automatically discover and compose services in a loosely coupled manner, and of course the desire for an open and voluntary participation model that respects local autonomy. Essentially the dreams of SOA on steroids. When I started working in OPAALS, an EU project in the area of Digital Ecosystems, and at the same discovered REST, what I realised was that the RPC-based models that were being built within the project were not going to work, for all the reasons that RPC in distributed systems has been criticised. So what am I to do having realised this? Should I (i)let sleeping dogs lie and just continue my own research, (ii) declare the design goals 'too out there' and refuse to collaborate or (iii) see how these things can be done within the REST architectural style? As you may have guessed, I picked (iii). So I started making a number of propositions internally, designed to promote REST as an architectural alternative and at the same time explain how the use cases of Digital Ecosystems can be implemented in a REST over HTTP environment, with the help of my colleagues at the University of Surrey. During this process, and by getting some degree of acceptance within the project, we brought the idea of Digital Ecosystems much closer to the web, something that seems like a no-brainer in retrospect.

This goal-oriented attempt has lead to a number of ideas, RETRO being only one of them, and one of the saner ones at that. I hesitate to put the others out there for fear of saying too much with little to back it up. Many of these may seem ill-informed and will certainly face significant resistance at first, certainly until we can produce a working demo that can be used to prove we are not (a) insane or (b) the antichrist. The good thing is that they will be modular such that if people consider any one useful thay can adopt that and ditch the rest, and also can follow a separate evolutionary path, like RETRO, which is benefitting enormously from community scrutiny and feedback.

Wednesday 17 June 2009

RESTful Transactions: Model, Standard, Implementation, or Pattern?

Since we set RETRO loose from the lab a few days ago, there have been quite a few reactions, both positive and not so positive. The discussion has progressed along varius axes, and Stefan Tilkov made a very interesting comment:

In my opinion, transactions in REST should be a pattern, not an implementation


Very interesting. While we named RETRO a 'model', that may have just been us unthinkinly echoing the previous transaction 'models'. The thoughts that follow apply both to RETRO and potential other RESTful transaction models (compensation-based for example).

During development, I realised that RETRO as it currently stands cannot be implemented into a generic, plug-and-play solution that you would add to your custom RESTful API to magically give it transactional capabilities. The reactions of the server depend on the resources that can be locked, their interlinking, and the related side-effects. In the general case, RETRO can only be implemented in relation to a pre-existing API. So no reference implementation for RETRO, then. Perhaps the closest thing we could produce is RETRO over ATOM, or something along those lines, again with the server taking responsibility for managing any other side-effects.

Also, since RETRO is HATEOAS-based, standardising the entire protocol would be a contradiction in terms, as that would be the very definition of out-of-band information. Nevertheless, HATEOAS leaves a pretty big door open, and that is media types. In the model we have mentioned that Transactions and Locks would need their own media types. If there was a way to semantically self-describe a media type (yes, I enjoy causing trouble it seems) we wouldn't need to standardise that either. The way things are now, for RETRO or any other transaction model would need at least some media types to be considered 'known' before being applicable.

So, Model, Standard, Implementation, or Pattern? Well, I think pattern with a little bit of standardisation for the media types should do the trick.

Wednesday 10 June 2009

RETRO: A (hopefully) RESTful Transaction Model

This is a post for discussion around RETRO, a transaction model that aims to enable ACID transactions on the Web, while remaining fully RESTful.

The full model can be seen here.