random memes }

Optimal code

Read another posting recently which could be summarized as "You'd be crazy to write code in C++". While I generally agree with many of the usual points, there are simply times when it is necessary to write optimal code.

Take today's task as an example.

The "publisher" portion of our product at work takes as input a customer's "printed" report, performs some fairly heavy processing, then outputs processed/burst/compressed output suitable for efficient serving over the web. It seems that every year our customers are running larger reports through our product - almost as though they were dumping their entire database into a "printed" report!

In fact, this makes perfect business sense. "Printing" a report to the web is a lot cheaper/less risky than developing a new web application. Also the "publisher" portion of our product is indeed designed to handle very large input files.

Now I have a lot of experience writing extremely efficient code (when needed), and pulled out pretty much every trick in the book to push performance up to the limit of the hardware. With the prior "publisher" component (licensed from a third party) we had customers whose publishing runs were taking all night for ~100MB files. With the current publisher those same reports complete in a few minutes (with modest variation depending on the options used), and is generally I/O bound (with no extraneous I/O).

Still there are a rather large number of possible variations between customer data and the (many) options to publishing. Every once and a while a customer runs across a combination not covered in our testing.

Last week we received a report of a performance problem with a 759MB(!) report. Turns out there was a bit of O(n^2) code I'd missed that also achieved near-optimal heap fragmentation. Naturally this required the right set of options, and was not bothersome even with 500MB input files (why we have not seen this before).

Just finished replacing the naughty bit of code. Memory use is exactly optimal. Performance is excellent. Doubled the input file size to get a (pessimistic) performance number for a non-cached input file. From the log: Elapsed time 262 seconds - processed 185356 pages, 11105172 lines, 1499383577 characters (5722838 characters/second) The output consists of 1854 (compressed) files totalling 119MB. This is all on my modest 2.2GHz Athlon test machine. Heh :).

Turns out we had a second customer regularly publishing a ~936MB(!) report (somehow I doubt this report was ever printed), and they were seeing a related problem (since resolved). Funny thing is we don't hear anything from customers when there is not a problem. So the only time we get confirmation about real-world usage is when there is a problem. We have customers publishing huge volumes, and it just worked - this is a good thing.

My point in all this is that there are some applications where performance is essential. I would estimate that this same application coded in Java (or C#) would take 10 to 100 times longer to run, and require considerably more memory (too much in some scenarios). This application uses custom memory allocation, I/O, strings, hash tables, and lots of low-level character and pointer manipulation. The debug version is heavy laced with assertions (much like Java's runtime range checking - and thus slower), and is run against a substantial set of regression/unit tests before every release. The release version is compiled without the assertions.

I do agree that coding in C++ is relatively risky, and is neither required nor desirable for most applications. Still there are cases where is necessary to extract every last bit of performance, and it is just not possible to achieve the performance needed with a "higher-level" language.

On the flip side, outside some extremely rare exceptions, I do not believe there is a need for a "lower" level language than C++. With careful coding it is possible with C++ to get very close to the limits of the hardware. In fact with an equivalent programming effort you could not develop an equally efficient program in "C" or assembly.

Back to writing Java code ... (in another application).