random memes }

Javascript on the server

This is all - at least in part - a bit premature.

Since the amount of time I have spent writing web applications is relatively limited, in the past I have largely stayed away from offering value judgements as to which techniques are most useful. As far as new technologies and techniques in general are concerned, you could say that I am a conservative early adopter. What that means is that I tend to try out new things very early, but only incorporate new bits in my work when I am throughly convinced of their value.

By way of example - in the early 1990's a friend (Rick) and I were working on projects where we were able to use C++ for the first time. While both perhaps equally interested in useful new technologies, our approach differed. As soon as Rick identified a notion, he cast the notion into a C++ class, and quickly built up a considerable class library. My collection of classes was much less impressive, as I only cast ideas into classes when I thought I thoughly understood the abstraction captured. Later when discussing alternatives for the next phase of work, Rick repeatedly had to say that he would need a large rewrite to support many of the alteratives. Rick had run headlong into the problem that would later be described as a "brittle" class hierarchy. At the time it seemed that I was not as far "forward", but then I did not have to spend as much time going back.

Now I am not saying that my approach is better (I have great respect for Rick). Rather I suspect that in the long run our differing approaches work out about the same.

Several months back my first significant web application went to customers. This was a one-person project, where I did everything from the front-end HTML/CSS/Javascript/Java applet (with one bit of AJAX-style code), through the server-side JSP/Java (on a custom embedded Tomcat instance), and the backend C++ code (for the compute-intense processing). While it took longer to develop than I'd like (over a year), I believe the end result is pretty decent. A bit of a stretch for a one-person project, but necessary for the problem solved.

All through the project I was looking for better ways to doing things. There are a few notions I have settled that seem to make a lot of sense (and that I likely will not have to refactor out later). This part as least seemed clearly a good idea:

I do not like style information in HTML. I do not like Javascript embedded in HTML. I do not like Javascript generating HTML or CSS. I strongly suspect the preceding value judgements will not change.

While the combined programming model for HTML/CSS/Javascript is perhaps a bit of a mess, the combined sum is probably more than sufficient for building all sorts of interesting web applications. In fact, with IE6 and Mozilla/Firefox as a base, I am pretty much disinterested in new CSS/HTML standards (as in potentially more trouble than they are worth).

On the server side, I was looking for something better than JSP. The mixture of Java and HTML - I just do not like. I am looking for that "sweet spot" that allows the most work completed for the least effort. JSPs just seem - inefficient and inelegant. But what is a better approach?

I have found TopStyle the best tool for working with CSS and HTML - especially when pushing the boundaries a bit for the best possible presentation. Note that my measure of "best" is a user interface that appears simple and obvious to the end user - no matter how much grief this causes the programmer. Flashy graphics and special effects are irrelevant, and usually undesirable.

I do not like the mix of program code and HTML in a JSP (or ASP, or PHP) page. Using a best-of-breed HTML/CSS editor (like TopStyle) is necessary for more than the simplest presentation (and when this is not a job you can foist on someone else). Trying to build and edit the structure and style of JSP pages using an HTML/CSS editor is … less than ideal when you are pushing CSS style and Javascript behavior. The JSP-aware editors tend to be less efficient when iterating on HTML/CSS structure and style.

You can attach style information (stored in *.css files) to a standard HTML file using the ID and CLASS attributes. At least in principle it seems you could attach behaviors and make dynamic modifications using the same basis - with no need for the HTML file to contain anything other than standard HTML.

There are lots of techniques, toolkits, and class libraries out there for web programming - arguably too many. At this point I have probably at least tried most of them. The problem with toolkits is that you get a bundle of ideas - some good, a few great, and a fair number lousy. Separating out the great from the lousy is hard.

In fact during this project I made a mistake that cost me months of wasted time. I was looking for some variation of HTML with markup as attributes - and settled on Tapestry as this seemed closest to what I thought ideal. As it turns out, this was a very bad choice. Tapestry is written to the model of a web page as a TV channel, and heavily oriented to keeping page/view/application state in a server-side session. Since for my needs I wanted URLs that could be bookmarked, and in fact had no need for server-side session state … the upshot was that I ended up ripping out the Tapestry code, and made better progress with simple JSP.

As an aside, I tend to view as suspect "frameworks" that require special handling for bookmarking or the "back" button. If the view changes enough so the user perceives a distinct "place" in the user interface (a "place" that a bookmark of the back button could return to), we are probably better off loading a distinct page with a distinct URL. Page reloads are not always to be avoided - even in the "web 2.0" / AJAX world. :)

So I am looking for something better. What I am looking for is a “sweet spot” - an approach efficient both in execution and in programmer time. I do believe I have lately found at least part of that “sweet spot”.

Good performance in a web application can only be achieved by striking the right “balance” in your implementation. Since there is typically one server and many clients, where possible you want to shift computation to the client, and away from the server. Since the communication channel between the server and client is limited, you want to minimize the volume of data sent between the client and server. Since client-side application behaviors (typically Javascript) have to be downloaded, where possible you want to shift code complexity toward the server, and away from the client.

It seems that part of the key is to ship the variable data from the server to the client in JSON format. The HTML, CSS, and Javascript are then largely or entirely static files (which means standard web browser caching pays a big benefit). Since we generally end up needing the data in Javascript data structures (at one time or another), shipping the data in JSON format makes a lot of sense. JSON needs no additional code (as this is simply a Javascript data structure). JSON is generally smaller than XML or HTML - uses less server-side compute to generate, and less client-side compute to ingest.

The other key is to generate variable HTML against variable data using client-side Javascript and (hidden) in-page HTML structures as templates. The base is a pretty simple bit of code.

// // Clone and transform the template with the given id. // Alter the cloned fragment by applying the map of ids and functions. // Apply the associated function to each node with an id in the map. //

window.expandTemplate = function(id,fm) { var apply = function(node) { var f = fm[node.id] if (f) { node.removeAttribute("id") f(node) } for (var o = node.firstChild; o; o = o.nextSibling) { apply(o) } } var o = document.getElementById(id) if (!o) return null o = o.cloneNode(true) o.removeAttribute("id") apply(o) return o }

Unfortunately this is too simple to call a toolkit, or to be worth coming up with a fancy new name. On the other hand, when I find that a small bit of code is getting a lot done - it does seem to be the right track. (This would be clearer with a full example - which someday I will get around to putting up.)

Note that this approach is a big win, as “o.cloneNode(true)” is almost certain to be radically more efficient than…

  1. Building the HTML string on the server.
  2. Shipping the HTML string across the network.
  3. Parsing the HTML string on the client.
  4. Building the HTML document structure on the client.

In most cases - not only have we reduced the overall compute needed to generate a page, we have shifted compute load from the server to the clients, and reduced the amount of data sent across the network. This is one of those win-win-win scenarios. :)

Once you are shipping variable data from the server to the client in JSON (or simply Javascript) format, it seems to make sense to ship data from client to server also in JSON format. Encoding Java data structures into JSON format is relatively straightforward. Transforming data sent from the client from JSON into Java classes … is a bit more open to interpretation. It is not clear that you always - or even usually - want to transform a dynamic JSON structure into a more static Java structure.

At this point - when I am wondering if using server-side Javascript might not be the most efficient way of coding handlers for JSON data sent from client - Tim Bray posts:

Go Visit Phobos … while the server-side JavaScript notion is pretty weird, go have a look at the code; writing an AJAX app all in one language is seductive on the face of it.

Perhaps server-side Javascript is not weird at all - especially when the Javascript implementation can call easily into Java as needed. Given that one implementation choice is to store structures to disk in JSON format, using Javascript all the way through makes some sense.

Personally I am not yet fully convinced I have found the “sweet spot”, but the combination of client-side Javascript and JSON format on the network looks a bit more than promising.