Thinking about Ajax Wire Formats in FireAnt
Thursday, November 10, 2005 10:41 AM:

Late last year when we were developing FireAnt after some time of thinking through the plumbing, the programming model and the development paradigm we came to an interesting dilemma: what would the over the wire format be? Our basic goal was to send only data (or as close to only data as possible) over the wire to the end user and make the development model easy (e.g. no hand written spaghetti code). While that sounds simple enough, the requirement leads to a bunch of possibilities on how to serialize the data for transport via XmlHttp.

And we knew from lots of performance work we'd done over the course of 2004 what kind of connections, speeds, networks, and bandwidth were available to our customers all over the world. We'd done field tests in China, India, and other parts of Asia and Europe, so we had a pretty good idea of how bad internet connections out there are (1s+ latency, 25%+ packet loss and worse). We also knew that our current (called Wave 10) product didn't fare too well in those scenarios due to the raw amount of data we were sending across the wire so a goal of Kahuna and FireAnt was to reduce it.

The first, and most obvious, was to send the data in XML. It's called XmlHttp after all, right? The second was to just use SOAP. ASP.NET makes SOAP services easy to create. The third and last was to create our own over the wire protocol. If you look under the hood, you'll see we've done the latter and I'll try to explain why here.

Before getting in to a discussion of each format, one distinction that I want to make is the difference in the upstream (client to server) and downstream (server to client) formats. They both have very different requirements. You have a (fairly) limited programming model on the client that's cooperatively multithreaded, so doing a lot of processing on it can get expensive. You also have a sophisticated server with a sophisticated programming language and (for all intents and purposes) unlimited processing resources. 

• XML - using XML is really easy with JavaScript and the DOM, it basically makes manipulating an XML document straightforward as can be. Plus, it's got very little overhead on top of the actual data you're sending in the document. So why didn't we chose it? To start, the size of some of the data types we wanted to send to the client would make the memory requirements large and the amount of xpath'ing that would be required could get hairy. We wanted to keep the programming model on the client simple (so we didn't end up with spaghetti JavaScript). Further, we'd have to write generators to create XML representations of objects on the server to send to the client. On the client to server end, we would have had to create XML generators to encapsulate the data the client wants to send up.
• SOAP - SOAP is great on the server side. You tag [WebMethod] and you get all the server side marshalling generated for you along with type safety. The reason we decided against SOAP two fold. First, generating SOAP on the client to send to the server was going to be a lot of work. Secondly, and most importantly, unpacking SOAP responses in JavaScript was going to get pretty unwieldy without a JavaScript code generator (e.g. something like a "wsdl2js"), which lead us to:
• FireAnt Proxy Protocol - we ended up going the route that let us get the benefits of XML (data only) with being very specific to the fact we're running within a web browser. I like to think of it as SOAP for browsers. On the way from the server to client we send down JavaScript that can be eval'ed (nothing fancy there) and we have JavaScript code that we generated that instantiates objects on the client when the code was eval'ed. On the way up, we send what amounts to form POSTs (e.g. application/x-www-form-urlencoded). We have marshalers that are autogenerated on the server that unpack the form post and call the methods underneath. Much like SOAP we have [FppMethod] akin to [WebMethod], but instead of generating WSDL, we pump out JavaScript directly.

All of the above possible on the wire formats will work perfectly well, but in the context of the goals that we had, building out own worked the best.

As you can see if we had chosen XML and SOAP we would have had to do a lot of work to make it work seamlessly with our development model. Further with SOAP, it's a pretty heavy protocol when dealing with lightweight data (e.g. a GUID, or a command acknowledgement). In building FPP, we basically decided to split the representation of data on the way up versus the way down to hit the goal of reducing the amount of data over the wire. We then generate all the glue that held the client and the server together to make the development model straightforward.

Comments

Add A Comment