To set expectations – this is my personal weblog. What I put here is the bits that (to my mind) might be otherwise missing. I am not interested in repeating material covered elsewhere. If you are already well-read, then I hope these bits will nudge your thinking forward a bit (even if you do not entirely agree with my conclusions).

On the other had, if you are seriously short of clues, I am not going to help you. Spent enough time on USENET News (long ago) to note the point of diminishing returns. Since I write these bits as an entertainment (of sorts), I am not going to invest a lot of time with folk who are too many laps behind.

Javascript is at heart a dynamic, prototype-based sort of object-oriented language. Most folk coming to Javascript are acquainted with static, class-based languages – and have little or no experience with anything different. The natural inclination is to transfer learned habits from the old to the new language – and that is a mistake.

Back in the late-1980′s or early-1990′s one writer came up with a classification scheme for object-oriented languages. There were many variations proposed and explored in that time, so a scheme that enumerated the important aspects was very useful. (Wish I knew how to find the article – it was in an ACM or IEEE publication of that period, I believe.) As a guess, I suspect that most of the current generation of programmers is not aware of the possible variations, and assume all object-oriented languages must be like Java / C++ / C#, and expect the same learned habits to still make sense with Javascript.

Disregarding the usual noise, there is one bit which it is worth responding, as it (quite unintentionally) illustrates my point.

New gives you prototype inheritance, performance benefits, and it’s about language semantics.

That is exactly my point! … only the reality differs from the assumptions of the guy making the comment (and I suspect he has a lot of company). Back in the late 1980′s / early 1990′s the pragmatic consensus was to move forward with object-oriented languages that could be made to run efficiently on the then-current hardware. How to generate optimal code for static class hierarchies was fairly well understood, at that time. More dynamic object-oriented languages were simply too hard to optimize.

The v8 Javascript engine offers a good example. The combined memory and CPU footprint of an efficient Javascript engine was simply impossible on circa-1990 computers. What is practical and reasonable on current-generation computers was – twenty years back – completely not practical.

My reading of the articles on the current-generation Javascript engines was that the toughest problem – and main aim of implementors – was to optimize in the absence of static-class hierarchies, and that quasi-static classes are not particularly optimized.

This makes sense. Writing script for web pages is programming in the small. The number of entities on a web page is small, as are the number of repeated instances. Static classes with efficient support for huge numbers of behavior-identical instances are essentially useless in a web page. If you are creating large numbers of instances for a web page, you are almost certainly doing it wrong. (Note the “flyweight” class pattern is useful here.)

My assumption (from what I have read) is that “static” class hierarchies yield no particular benefit in client-side Javascript.

Assumptions should be checked … so I wrote a micro-benchmark.

Method dispatch – Microbenchmark – Javascript
The aim was to measure method-dispatch for three cases:

1. The method is bound to the instance.
2. The method is bound to the “class” (via the __proto__ member).
3. The method is bound to the “class” (via the function invoked via new).

What I see (in terms of exact results) will likely vary as each browser vendor tweaks their Javascript engine. The point – in the case of the current discussion – is that static class hierarchies yield no significant advantage. It does make perfect sense that static-class languages can deliver bare-hardware performance numbers (which I do expect to use). For web page script, I expect static-like class usage to yield no significant benefit.

For programmers coming from C++/Java/C#, the notions that made sense no longer apply. Thus the emphasis on denying any semblance to static-class languages. This was exactly my point.

Still … my Tivo box is relevantly old, and you might expect new features to be introduced mainly on new boxes. This is not a complete excuse – the Tivo is basically a small computer with a hard disk and a network connection. A software upgrade on the old boxes could do quite a lot, within the hardware capabilities of the box … and I was paying a subscription fee.

My old non-HD TV finally died, and bought a new HDTV. Time to upgrade the DVR. Do I want to buy a new Tivo? The new Tivo seems a bit … limited? If I buy the full-up version with the most cost-effective service, the cost is a bit under $900 (without discounts). Do I trust the Tivo company and brand enough to spend that money of their box? Maybe.

When the Google TV announcement came out, that was enough to defer upgrading the Tivo until I saw Google’s product.

In the time between, the (not so) good folks at Tivo changed my mind.

Near two months back, the Tivo folk starting sending nasty emails, claiming my credit card was about to expire, and … let’s just say I did not appreciate the tone. Somehow they had decided my credit expired, two months before the actual expiration. The updated credit card arrived at the usual time, but by then the Tivo folks had sent a month’s worth of nasty emails. Just after I got the updated card, I went to the Tivo site, and updated my credit card information with the new expiration.

About a week later I got another nasty email from Tivo. I sent a response.

Guessing I might have made a mistake, somehow, I went back to the Tivo site, and updated my credit card information … again. Noticed all the fields on the web form offered auto-complete, so obviously I had been to the site recently. (I reset all the saved information in the web browser a couple months back, as a part of testing some of my software.) After submission, I checked the result carefully to be sure that their web application reported success – and it did.

About a week later, I got an even less pleasant email from Tivo. I sent a response, then I called their support phone number.

Has the Tivo company has gone sour? I have noted a pattern where a business with substantial cash flow is bought out, and the new owners cheapen the product and/or service (presumably with the hope of boosting profits). Customers eventually notice, and start to go elsewhere. In response, the owners further cheapen the product (in hope of maintaining profits on smaller revenue?) … and the business does badly.

Has the Tivo company been bought out, or adopted the same philosophy? That would explain the slow change in technology, and the unpleasant customer service.

As a good customer, I did exactly what I should, when I could. Personally, I would rather not do business with a rude company. On the phone with Tivo support, I did not appreciate the assumption that the mistake was mine. (I suspect most customers surrender on this point.) I want Tivo to fix the problem, and to not be rude to good customers. I refused to update my credit card information over the phone. Instead they could tell me when they fixed their broken web application, and I would (for the third time) update my information there.

Today it seems they turned off my Tivo service. The fault is theirs for losing a good customer. Probably not a loss. For now, I can wait to see what the Google folk (and associates) deliver.

A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.

— from Robert A. Heinlein

Refactoring … to what I want yet to do.

• Design and build a house that works well.
• Practical knowledge of firearms.
• Learn to hunt, and butcher the animal, after.
• Learn to fly an airplane.
• Design and build an airplane.

In roughly the above order.

Javascript is a hash (pun not entirely intended). There are good parts to Javascript, and bad parts. For folk attempting to learn Javascript for the first time, they could use some help avoiding the icky bits.

My first cut at rules for a “strict” mode for Javascript:

• No use of “function name(){}” declarations
• No use of “document.write()”
• No use of “new”

No doubt I could come up with more rules, with a bit more reflection, but this is a start.

No use of “function name(){}” declarations
There are two means of declaring functions, and I strongly prefer the second form. Programmers coming from other languages tend most often to use this form:

function foo() {
}

For an entire collection of reasons I prefer this form:

var foo = function() {
};

The second form is better suited for later refactoring, and offers subtle emphasis to the learning programmer of the difference offered by Javascript. The first form can also lead to subtle bugs (which I do not want to explain) – certainly not what you need when first learning a rather different language.

As a partial hint, I tend to make rather a lot of use of namespaces, so my function declarations most often look like this:

var ZOT = {};
ZOT.foo = function() {
};

Or in the more elaborate use:

var ZOT = (function() {
    var foo = function() {
    };
    return {
        foo: foo
    };
})();

This last form makes full use of closures, and allows for private data and functions.

No use of “document.write()”
Programmers tend to want to generate HTML from program code. This is a huge mistake.

To me this is a classic application of separation of concerns. HTML describes page structure. CSS describes page presentation. Script describes dynamic behaviors. Best to keep each concern separate.

No use of “new”
On this point I am certain others will differ. They might even be right. Certainly the “new” operator is of huge advantage in static languages. But that is not the domain of Javascript – certainly in the web browser, and possibly even for server-side usage.

The “new” operator misleads programmers when first learning Javascript. The pattern of usage most appropriate to a static-class language is not the same as the most appropriate usage in an entirely dynamic language.

Conflicted over Crockford
There is a measure of irony in the last point. Douglas Crockford has consistently served to promote better usage of Javascript. Crockford was hired by Yahoo. The quite elaborate user interface library in Javascript developed by Yahoo, is first invoked with “new YUI()”. YUI obviously represents a substantial body of work, at least partially in what seems the right direction. Yet I cannot yet quite get past the beginning.

Have experimented occasionally but steadily on how to make really good pizza. I do not have a massive coal-fired brick oven, so I have to make do with a slightly fancy counter-top oven. Note also that I am allergic to tomato, onion, and garlic – so the most common recipes are not useful.

Early on I made mushroom and sausage pizzas, and got decent results. Turns out a simpler(?) pineapple pizza is harder to get right. My kids have many likes and dislikes, but they all like pineapple pizzas. So the occasional challenge – every week or so – is to make a better pineapple pizza. Got results my kids happily accept a long time back, but I was not satisfied.

Tricks that get closer to that perfect pizza:

• Empty a can of crushed pineapple into a sieve, dump yeast and a bit of flour into the juice, and put the mix into a warm (120°F) oven for an hour. The yeast go nuts, and add a bit to the flavor of the eventual crust.
• Add a bit of fresh-ground black pepper to the pineapple. My daughter did not notice the pepper, but liked the result (as did I).
• Dry out/caramelize the pineapple a bit – one hour at 250°F seems a good start.
• Add the shredded mozzarella to the pineapple, before topping the pizza.
• Add sliced (and drained) olives to the topping. The saltiness of the olives nicely accentuate the sweetness of the pineapple.
• Add a bit of olive oil to the topping.
• Add lemon juice to the topping. The sweetness and acidity boosts the pineapple.
• Add flour to the starter to make a dough, spread on a pizza pan, add topping, and put in a 120°F oven for 30 minutes. The sugar from the lemon juice seems to boost the yeast in the dough, and adds a bit of lemony flavor.
• Set the oven to 425°F for 15 minutes (or less – watch carefully).
• … and I am still working on this. Results are good, and improving.

The cover of “Hazy Shade of Winter” by the Bangles is one rare instance where the copy surpasses the original. The original is great. The cover by the Bangles is better. (Even minus the “babe” factor.)

My only wish is for a longer version.

I bought VMware Workstation several years back. Used virtual machines quite a lot in developing and testing software. The company I work for licensed the enterprise version of VMware, at least in part due to developer experience using VMware. A few of the VMs currently hosted under the enterprise version of VMware started out running under VMware on my desktop.

A few years back – well after EMC acquired VMware – I got the distinct impression that the desktop version of VMware was somewhat neglected. I was losing increasing amounts of time to locked-up VMs, messed up keymaps, and even main desktop lockup. This provided an incentive to try Sun’s VirtualBox, which proved more reliable for my usage.

Oracle gained ownership of the VirtualBox software with the Sun acquisition. Is there any strong reason for Oracle to offer and develop the “free” version of VirtualBox, or will the desktop version of VirtualBox suffer the same apparent neglect as did the the desktop version of VMware?

Of late I was reviewing the documentation we sent to our customers with the last software release, as we are rolling up to a small update in the near future, and a larger release a bit later. In the documentation we send to customers, we have a prominent declaration of support for running under VMware. This was an easy declaration to make, as developers used VMware heavily in development and testing. We knew our product ran under VMware.

Now this is somewhat less true. Certainly the more elaborate test setups (mine) are all run under VirtualBox. I expect things all still work under VMware, but I personally no longer have a basis to make that assertion. For the next release, we will likely add an statement of support for VirtualBox.

As to the obvious bit I had missed:

Many of our customers are still not using virtualization software. Generally customers move from a physical machine to a VM at the time of a major version upgrade. At the next major version upgrade, we will offer a recommendation for VirtualBox.

Oh. Right. I had forgotten about that multiplier. Selling enterprise virtualization to our customers represents many more opportunities than any in-house usage at our development outfit. Nevermind my misgivings – offering free software to developers is about the best advertising Oracle could buy.

Does Oracle know this?

One surprise in the experience was the difficulty in getting the domain controller (and DHCP setup) properly working. Admittedly, setting up a domain controller is a somewhat unusual task. As a rare task perhaps not so important to Microsoft as a process that needs the user experience optimized. Still, given I have long been well-acquainted with how this all works at a network level, I ended up wasting a surprising amount of time trying to find the right knobs to turn. (Much searching with Google to get a key Microsoft product to work … the irony.)

Oddly, all the nice GUI interfaces to administering a domain server actually make the process more obscure than the equivalent activity on Unix (where I am accustomed to configuration kept in text files). I do now better understand my past conversations with network administrators who did not know how to do (what I considered) simple tasks with Microsoft’s domain controller … and why they often got it wrong.

Since my aim was testing, I wrote programs to populate Active Directory with large/variable numbers of users and groups, and to cleanup after. Though you could make direct Active Directory or LDAP calls, I find Microsoft’s ldifde utility is quite useful for this purpose. Note that Active Directory uses tombstones to support distributed replication (which is good), and that scripted changes will leave behind many tombstones – which may perturb your testing. You can limit the scope of the impact by changing the tombstone lifetime – the period of time tombstones are kept before cleanup – from the default down to 2(?) days. (Finding and changing that single setting ate up the large part of day.)

(Had a flashback to the old original text-based Adventure game. “You are in a maze of twisty passages, all alike.”)

Another approach is to take a snapshot of the VM when Active Directory is in a “clean” state, and restore the snapshot later. In principle this could be simpler/faster than the scripted restore (using delete operations fed via ldifde). Found a quite thorough article on virtualizing Active Directory, and ran across the USN ROLLBACK problem.

To aid in distributed replication, Active Directory keeps a count of local changes (the USN number) to track replication of remote changes. All quite usual and normal for this problem domain. Given that Active Directory has been around for a decade, you would expect the obvious use-cases to be covered, but not quite….

Turns out if you distribute changes among more than one Active Directory controller, you have a problem in the not-unusual case of taking a backup (or a snapshot) and restoring it later. Thus the many items returned by the above web search, including developers at Microsoft.

Um … what?!? Seems to me this is a pretty common case. That Microsoft would have got this wrong in the first version back in 2000 is only slightly disappointing. That this is still a problem a decade later is very surprising indeed.

Given that I may for future test purposes need to add a second domain controller VM, this looks to be another problem to avoid.

At first, there was no word from Oracle on a real fix … then I received this email.

Hi Preston,
Good to hear from you. Have I told you what a fine group of folks there is at Oracle Java? No? That’s because there isn’t.
[snip]

Anyway I ended up talking with some people in Oracle (our SES in DC called the president of Oracle) and several people later a VP of security called me. Anyway the fix will contain 3 implementations low, medium and high security and he claims it will take a significant amount of work in the application software to implement any of them. The release is expected by September. The gotcha is the VP told me that they will only release the JRE to licensed JRE users. I told him the JRE has always been a free download and he responded with “free download not free use”. In order to use the JRE you must have a support contract in place and it appeared to him the DoD was a couple million licenses out of compliance. Apparently it was only free until they came up with this Java for Business thing they have now. In my experience it’s a very Oracle response. Quite the opposite to my dealings with Microsoft.

I guess time will tell….
[snip]

(Emphasis is mine. All identifying information removed.)

Well, we have an answer. Yikes.

To be clear, I believe Oracle is entitled to do whatever they want to make money from Java. Oracle bought Sun, and now they own Java. Sun had a different approach, but Sun failed. Also what Oracle does in the long term may or may not be different from what the above message indicates.

On the flip side, I have to re-consider whether it makes sense to write code for the Java platform. If Oracle is fragmenting the Java platform, and changing the cost to customers … using Java may no longer make sense. This may be the point where I start planning to move all future work off Java and away from the JVM.

Of course, there is open-source Java, but I am not convinced there is the critical mass of interest required to make open-source Java a viable option. I would very much like to be wrong on this point, but for now, I am skeptical.

Is Google’s Dalvik VM potentially a viable option? Might IBM’s investment in Java translate into backing for open-source Java? Is it time to move off the Sun/Oracle JVM entirely?

Time to re-visit choices made long ago….

This item is not for sale in Catalina Island

What?!?

Note that Catalina Island is a modest-sized inhabited island off the coast of the most populated part of southern California. About the only industry on the island is catering to tourists. There is nothing special about Catalina. My measure of a clear day is when I can see Catalina when coming down the hill from my house.

Turns out, there are quite a lot of mentions, all in advertisements and product descriptions, and no apparent explanation.

Something smells funny.

As a guess, it looks as though this could all have a common source. All the references I found are roughly similar structure – as though somewhere the odd item were getting inserted into a stream of product information, which was in turn transformed and piped to a variety of destinations.

Someone trying to be clever?

When was the last time you used a computer with a floppy disk drive? When did you last save a file to a floppy disk?

Seems rather past time for a new icon.

Funny bit is that the square-with-notched-corner shape of a floppy looks rather like an SD memory card. Still not a good choice for file-save, as more likely use is to copy files from the memory card.

Checked to see if Microsoft had any good ideas. Office 2007 uses a image of a floppy for “Save…” and “Save As…” – adding a pencil writing on the floppy label (an antique behavior) for the “Save As…” icon. Internet Explorer changes “Save As…” so the pencil is writing on the sliding metal cover that protected the magnetic film (not really a good idea).

Went searching through GUI Style Guides looking to see if Microsoft (nope) or Apple (nope) had an updated replacement. Along the way ran across the style guide sections on designing for differing display resolutions. Display resolution is measured in DPI (dots-per-inch) and display sizes are measured in inches. Because both Microsoft Windows and Apple Macintosh software were originally developed on the west coast of the United States, entire generations of programmers and users have learned to think about screen sizes in inches.

Have you noticed that all the HDTVs sold have sizes measured in inches, not in Metric System units?

This is odd, as only the United States still holds firmly to the old “British Imperial System” of measures, and HDTVs are not made here. Even odder, in countries that do use the Metric measures, the HDTV sizes are still expressed in inches. (Don’t believe me? Check out the Amazon site in France – home of the Metric System.)

If only the French had developed Windows … then the Metric System would have had a chance.

I expect WordPress to be insecure. Looked at the source code early on. Like most PHP applications, the potential attack surface is very large.

Will be a bit before things are entirely in order. (Ick. Using a stock WordPress theme.)

If you type “model quality introductions” as a search string into Google or Bing (but not Yahoo – oddly), you get the site of the original business, followed immediately by my old weblog article.

This week I got an email (which I promised not to publish without consent) claiming to be from someone personally connected to the business, with the claim I was causing harm to their business. The writer was asking that I remove my old weblog article.

On reading the email there were two apparent possibilities. First, the email could be honest, and an attempt to rectify the unfair harm done by my article. Second, the email could be a lie – and an attempt by a marketeer (of some form) to eliminate unfavorable mention.

If the first were true, then I very much regret any harm I may have done. I offered to add the sender’s story, and hopefully offer some balance to readers who might be interested in their business.

If the later were true, then the email is lie, and a simple attempt to suppress any highly visible unfavorable information.

I should add that one of my very first jobs was with a guy who considered himself a super-salesman (Hello, Eric!), and would regularly offer bullshit persuasion on one topic or another. As a result, I tend to detect and reject very early the pattern of expression offered by sales/marketing critters. I should also add that while a teenager – and as a personal interest – I read rather a lot about Semantics and Psychology.

You can often tell a lot about a writer’s background from phrasing and how they express ideas. The initial email sounded rather more like the expressions of a marketeer, and rather less like the claimed source. Still … this was an uncertain judgement. In my response, I offered to encourage and protect the sender – if honest (and meant exactly what I wrote). I was – very carefully – honest in everything I said. At the same time I dropped in cues that I expected a marketeer would try to interpret as in their interest.

The subsequent responses fit exactly with what I would expect if the writer were a marketeer, and a third-rate mind. With each exchange, my conviction increased that the writer was a marketeer (and a liar).

Still … I could be wrong. The writer could be exactly as claimed. On that chance, I will honor my promises. On the other hand, my current bet is 10:1 that the writer is a marketeer.

As the exchange aroused my curiosity, I went searching, and found:

• The online dating business must be hugely profitable, as the number of consultants and related businesses indicate.
• The dating-service-with-rich-guys model must be very profitable, as “Model Quality Introductions” expanded to more locations – and there are many imitators.

The odds of a hired marketeer – outside the content of the exchange – seem rather high.

By analogy, I was playing chess, and the marketeer was playing checkers.
Bit amusing, this.

Not that you likely have any shortage of suggestions – but if in your shoes I would take a close look at the zoo that includes RepRap, CupcakeCNC, and a host of related devices. Feels very much like the experimentation with small microprocessor-based computers of the late 1970′s. Maker’s Faire is next month (in the SF area), so there will be lots to see first-hand. Whether you are interested in building small fabricators, or in the software (and these devices are very software-intense) … this is very, very interesting stuff. Just as small computers transformed the world over a couple decades, I expect this smart, small-scale, on-demand fabrication will transform manufacture.

The automated response:

Your comment was marked as spam and will not be displayed.
Comment contains blacklisted/ignored words

What exactly in the above comment is likely to be taken as spam, and/or blacklisted??

After a bit of experimentation … it seems Gosling does not like me. Guess that is an achievement of sorts.

When one of your dependents hits 17 years old, they no longer count as an exemption at tax time. Best name I can think of for this is the “Fuck-You Middle Class Parents of Teenagers” change in the tax law. If you are poor enough, or rich enough, this change in exemptions probably does not make a noticeable difference.

A 17-year old is in reality still very much a dependent. As parent, I can assure you that the expenses for 17-year olds are higher (much!) than at any other time – so there is no logical sense to this.

There is another even nastier interpretation – the aim was to put economic stress on families so as to pump more kids into the military. Need more bodies to fight those post-9/11 wars! (But not kids of politicians or rich Republicans.) When my oldest (now 21) hit 17, the change was new enough that the IRS website was only partly-updated, and I got very confused trying to figure out what was right.

What a nice turd parting gift left behind by the Republicans. Either the Republicans did not care about pissing off that part of the demographic, or they hoped it would be forgotten by election time.

Time to prune.

Twitter never really made sense. Too much idle chatter. Clearly some like the short-snippet format. Clearly some like a noisy, distraction-filled environment. Not me.

Google Buzz seems an improved variant of Twitter. Good for Google. Good for chatty folk. Not for me.

Getting rid of MySpace was a relief. Always found both the quality of construction, and the (unstated) premise … revolting. Good riddance.

Friendster was never useful. There are likely a bunch of other tertiary “social” sites where I poked around, but found no use. Will kill when they bug me.

Facebook is the most recently-popular. Had an account there since it was mostly college kids. A year or so back, a bunch of folk I knew suddenly started showing up. Spent a lot of time there. Did not accomplish much.

Will keep Flickr (for sharing photos) – at least for now. Google Reader is immensely useful for tracking many things, and finding longer more thoughtful communications.

Sun was always a bit of a mixed bag. Some of their work was absolutely brilliant, and some – especially with software – was amazingly dumb. Never did really understand why this was the case. (Lots of theories, of course, but no certain grasp.)

Oracle is a very different mixed bag. The Oracle database is both a very solid piece of work, and old not-quite-irrelevant to the present and future (an intentional over-simplification.) The Oracle database is core to Oracle the company. On the other hand, Oracle (the company) is also heavily invested in applications and software that cluster around the Oracle database. Much of this software is written in Java (for very logical reasons). This would seem to guarantee the future of Java (as a platform).

But … the Oracle-core has a diminishing (if very fat) future. That fat future can stretch out quite a while – beyond obvious reason – as the IBM mainframe market proved.

As a developer, I find that I crossed a threshold some time back, and am impatient with Java when I can express more elegant solutions (to complex problems) in Javascript. (Yes, really – Javascript. May have something to do with early heavy exposure to Lisp in school.) Hearing that C# incorporates lambdas … is C# a better base than Java (with appropriate wariness of stealth Microsoft patents), or rather an extraneous step before bundling a Javascript interpreter?

Do you trust Oracle?

Use of multi-column lists

One of my favorite annoyances is multi-column lists. When they first appeared in the early 1990′s, multi-column lists were cool. Add in user-sortable and re-sizable columns, and your application could offer better eye-candy than the competitors.

The usual problem with multi-column lists was that the columns came up the wrong size. Some applications were smart enough to auto-size columns – but not all – and auto-sizing was not always what the user wanted. The next logical enhancement was to save user preferences for column sizes, so that the next time the application came up, the user’s prior manual adjustments to column width were preserved. If the most-useful column widths change with different data, the users spend a lot of time manually fiddling with column widths.

As an example, we have the (really lame) Services view in Microsoft Windows. When this first appeared (in the mid-1990′s) the columns both (1) came up the wrong size, (2) in the wrong order (another topic), and (3) did not save user preferences. At the time, I figured this was a work in progress, and the Microsoft folk would fix this in a couple years.

The screenshot below is from Windows 7, so … apparently not.

The Eclipse IDE uses multi-column lists for compiler error messages (and the like). Eclipse might be my current-favorite IDE, but this is one of my least-favorite aspects.

Note that basically all the columns are the wrong size. The error message text is most important, and is too small, with only a fraction visible. That makes any less-important column too-large, by definition. The column with the most excess whitespace – “Type” – is one of the least important. The “Path” column occupies a lot of space, but is of at best secondary importance.

The most important bit of the “Problems” panel is tiny.

Note also that in the above screenshot, the overhead elements in the panel eat up about half the vertical space – far more space than the most-meaningful message texts.

This is not efficient … on a large 1920×1200 panel. On smaller panels, things are much worse.

Presenting program texts

The main payload in an IDE is the display of program text. An IDE may put up supplemental panels to help discover structure and aid navigation within the program, but the central bit is the program text. In Eclipse only a fraction of the screen is usable for the display of program text.

Current screens are more than wide enough for pretty much all program texts, but those same texts are far taller than they are wide. As screens are far shorter than program texts, you want to make the best use of the vertical space available. Only about 2/3 of the screen height is text, in the typical example below.

With Eclipse you can “maximize” the text panel, but there is still substantial vertical space lost to overhead elements.

Note also that while there is a shortage of vertical space, there is an excess of unused horizontal space.

This is not efficient … on a large 1920×1200 panel. On smaller panels, things are much worse.

Alternate presentations

To make the best use of scarce vertical space – especially on smaller panels – you want to excise vertical overhead. Program texts tend to be dense on the left edge, and sparse on the right. For the display of error messages or navigation aids, if you place panels on the right, there is good chance you will overlap little or no program text. If you use fly-away panels, you can quickly view even very long lines of program text on the smallest-current laptop displays.

There is a simple/clever way around the presentation of variable-sized columns – concatenate all the variable size bits into one column. You can keep the fixed sized bits as columns (or not). If the resulting text is too wide for the available column space, allow the text to wrap. Excise the less-useful fields, and present on explicit user action (like hovering or clicking on the row).

Note that too-wide message texts can be hard (or slow) to read. (Ever lose track of the line you are on when looking across a very wide display?) There is a good argument for limiting the width of message texts to about the width of a typical piece of paper. (Funny how that works out – common paper width tends to be about the efficient maximum for text width.)

Combine the notions for maximal program text and complete-but-constrained message texts, and you might end up with something like this simple IDE mock-up. (Note this is very much a quick-and-dirty first iteration.)

At 1920×1200 – on a 24″ desktop panel.

(Best to pull this up in Google Chrome, as only Chrome does a really good maximum size full-screen. Works in Safari. Renders in Firefox, though event handling is wrong. IE is pretty much hopeless. As this is a mockup, my aim was not to make it work across web browsers.)

Note that as the screen size drops, the use of screen space becomes radically better than conventional IDEs.

At 1440×900 – the resolution of my 17″ laptop panel.

(Imagine the vertical overhead of Chrome as a part of the text-space.)

At 1280×800 – the resolution of common basic laptop panels.

Note that on this smallish laptop we are getting both more vertical program text, and more complete message texts, than on my big 1920×1200 panel using Eclipse.

At 1024×600 – the resolution of a netbook panel.

With the fly-away message panel toggled off, even a netbook screen shows a good amount of program text.

Topic for later – navigation

After using Eclipse for several years, I have become pretty comfortable. Oddly, I have come to realize that in one aspect – navigation – Eclipse is inferior to my old use of Emacs with ctags/mkid/calls/grep. Under the hood, the functions for introspecting program texts are a dream compared to what I had twenty years back … but navigation still feels a bit clunky.

The next exercise will be to try and map out what is different, and take a stab at how to do better.

Simple basic facts about aeroplanes: long thin wings tend to more efficient (aerodynamically, not structurally) than wider/thicker/shorter wings. For much the same reason – propellers are more efficient than jets. Ducted fans are less efficient than propellers, but more efficient than pure-jet engines. Turbofan engines are basically ducted fan turboprops. Gains in jet efficiency over than past few decades are in part due to higher bypass turbofans (basically moving from pure jets closer to propellers).

Even propellers are not ideal. Swirling a couple curved sticks of metal through the airstream at high speeds is going to chew up energy without adding to propulsion. Many-bladed turbofans chewing through the airstream have got to be worse. Lots of energy wasted – could there be a more efficient way? Nothing especially obvious … or something better would be in practice.

Ideally we would like a way to throw back the bulk of an airstream without lots of extraneous physical churning. The only certain way we know is to use propellers – like oars in water. Could there be another way?

There is a well-known phenomena in Physics known as “electric wind”, that moves air without physical contact, but is by no measure efficient. Is there any way this could be used?

Is there any way to efficiently push an airstream without physical contact?

The “electric wind” is a stream of charged particles. A magnetic field deflects an charged particle moving through. The deflection exerts a force on the magnet (assuming the force is not sufficient to capture the charge). Movement against that force consumes energy. That energy presumably could accelerate the charged particles.

The mean free path of a charged particle at normal atmospheric pressures is short. Any accelerated ion would give up about half it’s energy at each collision. Short mean free paths mean many collisions. The net result would be (presumably) to accelerate a bulk of the airstream. Maybe.

Magnetic fields deflect charged particles. Strong magnets rotating on opposite directions could deflect and accelerate, then re-deflect and further accelerate charged particles – maybe. Would the result be significant? Would the result be efficient? I have no idea.

This might be an approach only possible if the “controller” is sufficiently smart, and with quite intense magnets (superconducting?). Matching the acceleration and deflection of charged particles through alternating magnetic fields through changing atmospheric conditions may not be possible with simpler control.

Is a “magnetic propeller” is practical possibility?

#!/bin/sh
gnome-open "https://mail.google.com/mail?extsrc=mailto&url=$*"

Make the file executable.

On Ubuntu Linux (using the Gnome desktop), go to:

System > Preferences > Preferred Applications

Under Internet / Mail Reader select “Custom” and enter the command:

/home/preston/bin/mailto %s

(Replace “/home/preston” with your $HOME.)

This should open GMail in your default web browser, composing a new message, with the recipient set.

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

Yep. Windows 7 is in fact Windows version 6. Gotta love those Marketing folk.

Most FastCGI backends seems to be written for non-multiplexed connections. (Much simpler, so understandable.) The IIS FastCGI connector apparently does not support multiplexed connections.

Writing a FastCGI backend that allows for multiplexed connections. Would be a waste of time if not supported by a frontend, or if existing frontends are buggy.

(Not really expecting a response, but have to state the question.)

The HTML Working Group at W3C is … far too much noise. The HTML5 “standard” is going to be a bloated monster, and there is no chance I can change that. Time to stop the pretense of trying.

Not that all the work is bad, or that there is any shortage of well-intentioned folk. What the group lacks is any sense of minimalism, and enough strong voices able to say “no”. What we will get is going to be even harder to digest than HTML4. This is sad. Future developers are going to have an even harder time, for no good reason.

The wildcard here is that the mainstream browser implementations may not follow all the half-thought ideas thrown in by the HTML5 Working Group. No idea how this will work out.

At the core HTML is pretty damn simple – or could be. HTML4 got stuffed with a bunch of half-thought notions, most of which have since proved of no value, and were ignored by developers. Ideally we would learn from experience, omit the fuzzy disused bits, and trim HTML down to the useful core. There are well-known (though not universally known) means to achieve this aim. This is not going to happen.

I cannot keep up with the herd of Energizer-bunnies eager to make their mark, and with too-limited experience.
Time to stop pretending.

The use of UTF-8 on the web is common and increasing. Lots of data comes in as UTF-8, and inefficiency in UTF-8 data handling is going to have pretty pervasive impact.

On the flip side, the creators of UTF-8 did a good job. There is nothing really complicated about the UTF-8 format, and processing is simple.

So I was surprised (or rather shocked) to find in an earlier experiment that Java performed UTF-8 conversion slowly. In fact, I was able to write a faster UTF-8 decoder in Java than the stock decoder. This is just plain wrong. Conversion between encodings is a primitive/simple operation best written in C/C++ and run as native code (and this is the sort of processing where C/C++ is probably always going to be much faster than Java).

There is a problem in that malicious external parties can send oddly-encoded UTF-8, and bypass simple-minded malware detection software. Ordinary ASCII characters can coded as an alternate multi-byte sequences, and simple scanners miss the alternate encoding.

This is a problem. There is a simple solution. One of a set of principles I adopted a long time ago is “convert at the edges”. If you have data coming in from an untrusted source, then you perform conversion and validation at the “edge” where the data is first received.

In the case of UTF-8 coming from an untrusted source, to make all later processing simpler, you must recode to eliminate any alternate encodings. This is quite simple, as recoded UTF-8 will always be the same size or smaller, and so can be done in-place. The prior experiment measured the cost of UTF-8 recoding. Looks like we can drive a 1-Gbit network link at full speed (with efficient code), while recoding the entire contents. Since UTF-8 data usually represents a smaller portion of traffic, and since other processing tends to take the larger part of the load, there is no reason to not perform recoding on any UTF-8 data coming from an untrusted source.

Combine “convert at the edges” with UTF-8 recoding, and we lose the basis for the requirement in RFC 3629 for detection of “illegal” UTF-8 code sequences. In addition we allow all downstream processing to be simpler and more efficient … and we also can be tolerant of imperfect upstream software. (Yes, I am going to invoke Postel, again.)

The basis is good (secure processing with untrusted sources), but the requirement for detection of “illegal” sequences is not necessary and (most definitely!) not optimal.

Example of UTF-8 recoding (from String.cpp).

void UTF8::String::recode() {
    // Iterate until all UTF8 characters are normalized.
    // UTF8 in canonical form can only be smaller, so work in-place.
    char* p1 = pBuffer;
    char* p2 = pBuffer;
    char* pEOS = pBuffer + nContent;
    while (p1 < pEOS) {
        int c = 255 & *p1++;
        if (c < 0x80) {
            *p2++ = c;
            continue;
        }
        if (c < 0xE0) {
            c = (31 & c) << 6;
            c |= 63 & *p1++;
        } else if (c < 0xF0) {
            c = (15 & c) << 12;
            c |= (63 & *p1++) << 6;
            c |= 63 & *p1++;
        } else if (c < 0xF8) {
            c = (7 & c) << 18;
            c |= (63 & *p1++) << 12;
            c |= (63 & *p1++) << 6;
            c |= 63 & *p1++;
        } else if (c < 0xFC) {
            c = (3 & c) << 24;
            c |= (63 & *p1++) << 18;
            c |= (63 & *p1++) << 12;
            c |= (63 & *p1++) << 6;
            c |= 63 & *p1++;
        } else {
            c = (1 & c) << 30;
            c |= (63 & *p1++) << 24;
            c |= (63 & *p1++) << 18;
            c |= (63 & *p1++) << 12;
            c |= (63 & *p1++) << 6;
            c |= 63 & *p1++;
        }
        if (c < 0x80) {
            *p2++ = c;
        } else if (c < 0x800) {
            *p2++ = 0xC0 | (c >> 6);
            *p2++ = 0x80 | (63 & c);
        } else if (c < 0x10000) {
            *p2++ = 0xE0 | (c >> 12);
            *p2++ = 0x80 | (63 & (c >> 6));
            *p2++ = 0x80 | (63 & c);
        } else if (c < 0x200000) {
            *p2++ = 0xF0 | (c >> 18);
            *p2++ = 0x80 | (63 & (c >> 12));
            *p2++ = 0x80 | (63 & (c >> 6));
            *p2++ = 0x80 | (63 & c);
        } else if (c < 0x4000000) {
            *p2++ = 0xF8 | (c >> 24);
            *p2++ = 0x80 | (63 & (c >> 18));
            *p2++ = 0x80 | (63 & (c >> 12));
            *p2++ = 0x80 | (63 & (c >> 6));
            *p2++ = 0x80 | (63 & c);
        } else {
            *p2++ = 0xFC | (1 & (c >> 30));
            *p2++ = 0x80 | (63 & (c >> 24));
            *p2++ = 0x80 | (63 & (c >> 18));
            *p2++ = 0x80 | (63 & (c >> 12));
            *p2++ = 0x80 | (63 & (c >> 6));
            *p2++ = 0x80 | (63 & c);
        }
    }
    nContent = (int) (p2 - pBuffer);
}

UCS (Unicode) to UTF8 conversion, and the reverse, when efficiently coded in C++ clocks in well above 100MB/s on current generation CPUs. If you are getting something much less – enough to be a problem – then there are questions you should ask. The following run spans 1 to 6 byte UTF8 encodings.

Recoding a UTF8 string to a normalized form is a bit slower at 122-87 MB/s.

preston@athena:~/workspace/json-c$ time Release/json
base 00000010 :  229.9 MB/s UCS to UTF8 conversion
base 00000010 :  253.7 MB/s UTF8 to UCS conversion
base 00000010 :  122.0 MB/s UTF8 recode
base 02000010 :  196.8 MB/s UCS to UTF8 conversion
base 02000010 :  177.1 MB/s UTF8 to UCS conversion
base 02000010 :  100.6 MB/s UTF8 recode
base 04000010 :  173.8 MB/s UCS to UTF8 conversion
base 04000010 :  157.1 MB/s UTF8 to UCS conversion
base 04000010 :   89.2 MB/s UTF8 recode
base 06000010 :  174.0 MB/s UCS to UTF8 conversion
base 06000010 :  158.5 MB/s UTF8 to UCS conversion
base 06000010 :   89.2 MB/s UTF8 recode
base 08000010 :  174.0 MB/s UCS to UTF8 conversion
base 08000010 :  154.5 MB/s UTF8 to UCS conversion
base 08000010 :   89.6 MB/s UTF8 recode
base 0a000010 :  174.1 MB/s UCS to UTF8 conversion
base 0a000010 :  156.5 MB/s UTF8 to UCS conversion
base 0a000010 :   89.8 MB/s UTF8 recode
base 0c000010 :  174.0 MB/s UCS to UTF8 conversion
base 0c000010 :  155.1 MB/s UTF8 to UCS conversion
base 0c000010 :   89.6 MB/s UTF8 recode
base 0e000010 :  174.0 MB/s UCS to UTF8 conversion
base 0e000010 :  158.8 MB/s UTF8 to UCS conversion
base 0e000010 :   87.4 MB/s UTF8 recode
base 10000010 :  170.8 MB/s UCS to UTF8 conversion
base 10000010 :  158.2 MB/s UTF8 to UCS conversion
base 10000010 :   89.7 MB/s UTF8 recode
base 12000010 :  174.0 MB/s UCS to UTF8 conversion
base 12000010 :  158.8 MB/s UTF8 to UCS conversion
base 12000010 :   86.5 MB/s UTF8 recode
base 14000010 :  171.5 MB/s UCS to UTF8 conversion
base 14000010 :  153.9 MB/s UTF8 to UCS conversion
base 14000010 :   87.1 MB/s UTF8 recode
base 16000010 :  172.1 MB/s UCS to UTF8 conversion
base 16000010 :  158.1 MB/s UTF8 to UCS conversion
base 16000010 :   87.5 MB/s UTF8 recode
base 18000010 :  172.1 MB/s UCS to UTF8 conversion
base 18000010 :  158.2 MB/s UTF8 to UCS conversion
base 18000010 :   86.9 MB/s UTF8 recode
base 1a000010 :  171.3 MB/s UCS to UTF8 conversion
base 1a000010 :  158.2 MB/s UTF8 to UCS conversion
base 1a000010 :   86.5 MB/s UTF8 recode
base 1c000010 :  169.5 MB/s UCS to UTF8 conversion
base 1c000010 :  158.5 MB/s UTF8 to UCS conversion
base 1c000010 :   86.5 MB/s UTF8 recode
base 1e000010 :  173.0 MB/s UCS to UTF8 conversion
base 1e000010 :  157.8 MB/s UTF8 to UCS conversion
base 1e000010 :   86.1 MB/s UTF8 recode
base 20000010 :  173.1 MB/s UCS to UTF8 conversion
base 20000010 :  158.4 MB/s UTF8 to UCS conversion
base 20000010 :   87.2 MB/s UTF8 recode
base 22000010 :  173.1 MB/s UCS to UTF8 conversion
base 22000010 :  158.2 MB/s UTF8 to UCS conversion
base 22000010 :   88.2 MB/s UTF8 recode
base 24000010 :  173.3 MB/s UCS to UTF8 conversion
base 24000010 :  158.8 MB/s UTF8 to UCS conversion
base 24000010 :   87.6 MB/s UTF8 recode
base 26000010 :  173.8 MB/s UCS to UTF8 conversion
base 26000010 :  158.1 MB/s UTF8 to UCS conversion
base 26000010 :   87.9 MB/s UTF8 recode
base 28000010 :  172.0 MB/s UCS to UTF8 conversion
base 28000010 :  158.7 MB/s UTF8 to UCS conversion
base 28000010 :   87.6 MB/s UTF8 recode
base 2a000010 :  172.0 MB/s UCS to UTF8 conversion
base 2a000010 :  158.1 MB/s UTF8 to UCS conversion
base 2a000010 :   86.1 MB/s UTF8 recode
base 2c000010 :  172.1 MB/s UCS to UTF8 conversion
base 2c000010 :  158.5 MB/s UTF8 to UCS conversion
base 2c000010 :   86.3 MB/s UTF8 recode
base 2e000010 :  173.8 MB/s UCS to UTF8 conversion
base 2e000010 :  158.2 MB/s UTF8 to UCS conversion
base 2e000010 :   89.3 MB/s UTF8 recode
base 30000010 :  170.0 MB/s UCS to UTF8 conversion
base 30000010 :  158.7 MB/s UTF8 to UCS conversion
base 30000010 :   87.2 MB/s UTF8 recode
base 32000010 :  173.8 MB/s UCS to UTF8 conversion
base 32000010 :  158.4 MB/s UTF8 to UCS conversion
base 32000010 :   87.8 MB/s UTF8 recode
base 34000010 :  173.5 MB/s UCS to UTF8 conversion
base 34000010 :  158.5 MB/s UTF8 to UCS conversion
base 34000010 :   88.0 MB/s UTF8 recode
base 36000010 :  173.1 MB/s UCS to UTF8 conversion
base 36000010 :  158.4 MB/s UTF8 to UCS conversion
base 36000010 :   88.4 MB/s UTF8 recode
base 38000010 :  172.5 MB/s UCS to UTF8 conversion
base 38000010 :  158.5 MB/s UTF8 to UCS conversion
base 38000010 :   88.7 MB/s UTF8 recode
base 3a000010 :  170.5 MB/s UCS to UTF8 conversion
base 3a000010 :  158.4 MB/s UTF8 to UCS conversion
base 3a000010 :   87.4 MB/s UTF8 recode
base 3c000010 :  169.3 MB/s UCS to UTF8 conversion
base 3c000010 :  154.7 MB/s UTF8 to UCS conversion
base 3c000010 :   86.3 MB/s UTF8 recode
base 3e000010 :  172.0 MB/s UCS to UTF8 conversion
base 3e000010 :  156.9 MB/s UTF8 to UCS conversion
base 3e000010 :   87.9 MB/s UTF8 recode
base 40000010 :  171.8 MB/s UCS to UTF8 conversion
base 40000010 :  148.9 MB/s UTF8 to UCS conversion
base 40000010 :   83.7 MB/s UTF8 recode
base 42000010 :  173.1 MB/s UCS to UTF8 conversion
base 42000010 :  149.3 MB/s UTF8 to UCS conversion
base 42000010 :   88.1 MB/s UTF8 recode
base 44000010 :  173.7 MB/s UCS to UTF8 conversion
base 44000010 :  157.7 MB/s UTF8 to UCS conversion
base 44000010 :   87.4 MB/s UTF8 recode
base 46000010 :  172.1 MB/s UCS to UTF8 conversion
base 46000010 :  152.2 MB/s UTF8 to UCS conversion
base 46000010 :   87.3 MB/s UTF8 recode
base 48000010 :  174.0 MB/s UCS to UTF8 conversion
base 48000010 :  142.7 MB/s UTF8 to UCS conversion
base 48000010 :   87.8 MB/s UTF8 recode
base 4a000010 :  173.5 MB/s UCS to UTF8 conversion
base 4a000010 :  150.2 MB/s UTF8 to UCS conversion
base 4a000010 :   87.7 MB/s UTF8 recode
base 4c000010 :  174.0 MB/s UCS to UTF8 conversion
base 4c000010 :  158.4 MB/s UTF8 to UCS conversion
base 4c000010 :   88.4 MB/s UTF8 recode
base 4e000010 :  174.0 MB/s UCS to UTF8 conversion
base 4e000010 :  156.9 MB/s UTF8 to UCS conversion
base 4e000010 :   88.4 MB/s UTF8 recode

real	2m0.751s
user	2m0.540s
sys	0m0.190s

The above run is on a AMD Phenom(tm) II X4 955 Processor running at the stock clock rate.

Code is in: http://svn.bannister.us/public/json-c/
(A start on an experiment in fastest-possible JSON conversion.)

Note that I intentionally allow proper conversion of “invalid” UTF8 code strings. I completely understand the reason for the disallowed conversions, and I disagree.

Update: Converted to use pointer arithmetic, rather than array and index. Was not sure pointer math was still a win on current CPUs and compilers. Got a big boost in throughput, so it is!

The lightweight string class is simple enough to be reproduced from memory (nothing over-complicated), but can make quite a difference in performance. I had hoped to make a point.

Since posting, after the initial burst, the articles have collected what looks like a few new hits, every day, for the past five years. Counting hits is a pretty foggy indicator, but it seems possible quite a few folk have read the article (though a relatively small fraction of the programming community). Some may have copied the C++ string class in their code. (What I hoped!)

But there is no sure way of knowing. What is the cumulative impact of this steady trickle of readers?

There is almost no chance I can move the working group is a useful direction. Time to disconnect.

This is all rather discouraging. The HTML working group will proceed. Some of the work is worthwhile. Much (measured by volume of email list traffic) is not. What mix will make it into the generated proposed “standard” is sure to be a mess. Not sure how to change any of this.

My status as an “Invited Expert” is up for renewal. With extreme reluctance … my judgement is that I cannot make a useful contribution, and should disassociate from the HTML working group. Of course, they will continue on the present course, in my absence. My withdrawal makes no difference of significance. There is a fair chance the body of work from this working group will be adopted, imperfect as it is. The existing body of work is … badly skewed by an imperfect process.

Nothing meaningful I can do. The result will be a mess, and will create a mess for years after. Time to disengage.

Funny bit – I do not see a way to force a disconnect.

Three distinct usages I’d like to cover: optimal performance,Windows web server (IIS) interoperable, and webhosting.

In addition, there are three interesting aspects of optimal performance: throughput, scalability, and stability.

For serving static content, I really like the model of a single-threaded non-blocking web server, of which thttpd was an early example, and for which the C10K question clarified the need. A small/simple web server has a much better chance to being very reliable. With the single-threaded non-blocking model, massive scalability is possible.

For serving dynamic content, I really like the isolation and load distribution possible with the FastCGI model (or the like). Dynamic code tends to be complex. Javascript interpreters are complex. Complex code tends to fail more often. Complex code can use more compute throughput than possible on a single box. For intranet applications, a single front-end web server is often preferable, and load distribution via FastCGI offers more headroom. All of which tends to argue for the FastCGI model, with isolation from the front-end web service, and potential distribution of load across more than one machine.

For the widest possible usage, in additional to optimal deployments (when there is no restriction on the front-end web server), the engine on which the application runs should be deployable behind IIS (for Windows-only organizations), and at common web-hosting services (like Dreamhost). Microsoft’s recent support of FastCGI with IIS is a big help.

At that time (three years back), none of the solutions were really optimal – and in fact were pretty far from optimal. The Java-based RhinoJavascript interpreter was easiest to embed, but failed the webhosting case. The C++ based JavaScript interpreters were a pain to embed, and offered good (but not great) performance.

Fast forward to the present, and Google offers the V8 JavaScript Engine that offers great performance, and is easy to embed. (Google as the good guys, riding to the rescue once again … you’d think they have white hats superglued to their brains.) Suddenly we have lots of projects embedding the V8 engine. In addition, seems most all the single-threaded non-blocking web servers have picked up support for FastCGI.

Oh … and I am pretty much fed up with the Java Servlet model. After considerable time with the problem, I am of the opinion that the servlet model chose the wrong abstractions, and this makes for awkward solutions. (Of course, the servlet model appeared very early in the history of web applications, so the mistake is easy to understand.)

Which means the model offered by node.js makes a lot of sense. I like the notion of a naked node (running JavaScript on the V8 engine) performing request dispatch without any extra layers or abstractions. The main lack with node.js is the ability to work via FastCGI (and thus no means to be deployed behind IIS on Windows).

But there are as yet items unresolved and/or unclear.

• Projects like v8cgi offer the V8 JavaScript Engine connected via FastCGI.
• The node.js project offers a single-threaded non-blocking web server … but can it work behind FastCGI?
• Is the environment for server-side JavaScript the same (or sufficiently similar) between node.js and v8cgi?
• Comet is still a question. Can FastCGI work well with long-outstanding requests from applications?

The good news is that we seem a lot closer to attractive and well-supported server-side JavaScript for web applications … but it seems we are not quite fully there, as yet.

Went with the Intel adaptor when I ordered this notebook, in part as Intel seems to be actively supporting the development of Linux drivers. In practice, my laptop wireless connection is mostly unreliable, and often near-useless.

I used to think the problem was interference from other nearby wireless routers, but my daughter’s laptop seems to work well when mine does not. I have a simple “bounce” test for network performance. My daughter’s cheap Toshiba, on the same “bounce” page, gets steady/fast performance when side-by-side to my laptop … which gets slower/unsteady/unreliable performance.

Not specific to the wireless router or crowded neighborhood either – I get similar poor performance when at my father’s place in Colorado (different brand and generation of router, and fewer/further neighbors).

After the prior results, I wanted to see if a less general-purpose implementation would perform better. Given enough advantages, could a Trie out-perform a hash table? (Again, this visits to some extent the question asked in a prior discussion.)

For the current exercise I built two Trie implementations. The LinkedTrie is cheap to build, minimal in use of memory, but not especially fast to access. The FixedTrie implementation should be pretty close to optimal in access time (for a Trie), but expensive to build (in fact the FixedTrieBuilder transforms a LinkedTrie into an optimized FixedTrie).

As before, the sources are in (as an Eclipse project): http://svn.bannister.us/public/Trie/.

The performance numbers make sense. The older TallTrie and WideTrie implementations (that traded increased memory use for speed) are indeed faster, though the LinkedTrie uses much(!) less memory. The new FixedTrie is fastest (hurrah!) and uses the least memory (a shade less than LinkedTrie).

But even the FixedTrie is slower than a generic hash table, with read-only access about 3 times as expensive.

Sample measurements…

=== words
21 ms - 98569 words loaded

=== hash
3935/second - hash map loaded {4007 ms, 15771040 operations = 254 ns/op}
8554/second - hash map re-loaded {4010 ms, 34302012 operations = 116 ns/op}
10768/second - access each item in hash map {4000 ms, 43074653 operations = 92 ns/op}

=== trie (linked)
870/second - loaded trie (linked) {4078 ms, 3548484 operations = 1149 ns/op}
1204/second - re-loaded trie (linked) {4010 ms, 4829881 operations = 830 ns/op}
1431/second - access each item in trie (linked) {4063 ms, 5815571 operations = 698 ns/op}
98569 slots of trie (linked)
225791 nodes of trie (linked)

=== build fixed trie

=== trie (fixed)
85779/second - loaded trie (fixed) {4000 ms, 343118689 operations = 11 ns/op}
87652/second - re-loaded trie (fixed) {4000 ms, 350609933 operations = 11 ns/op}
3438/second - access each item in trie (fixed) {4013 ms, 13799660 operations = 290 ns/op}
98569 slots of trie (fixed)
225791 nodes of trie (fixed)

=== trie (wide)
65/second - loaded trie (wide) {4502 ms, 295707 operations = 15224 ns/op}
1659/second - re-loaded trie (wide) {4040 ms, 6702692 operations = 602 ns/op}
1672/second - access each item in trie (wide) {4007 ms, 6702692 operations = 597 ns/op}
41533124 slots of trie (wide)
225890 nodes of trie (wide)

=== trie (tall)
682/second - loaded trie (tall) {4043 ms, 2759932 operations = 1464 ns/op}
1733/second - re-loaded trie (tall) {4036 ms, 6998399 operations = 576 ns/op}
1913/second - access each item in trie (tall) {4019 ms, 7688382 operations = 522 ns/op}
6982694 slots of trie (tall)
446075 nodes of trie (tall)

=== string to UTF8 conversion
1393/second - word to UTF8 (stock) {4033 ms, 5618433 operations = 717 ns/op}
9585/second - word to UTF8 (fast) {4000 ms, 38343341 operations = 104 ns/op}

(Note that the FixedTrie ignores re-load, so the times for load and re-load are bogus.)

I suspect a C++ Trie implementation could do a bit better … but not necessarily outperform … compared to hash tables.

Looks very much like even a specialized read-only Trie cannot match the performance of a generic hash table (at least in Java).

After getting stuck on the latest problem, I finally built a “PrintExplorer” test program. The sources are up on Kenai, and on my website. In both cases the sources are in a Mercurial repository, and I “push” changes to both (one of the nice features of Mercurial). The project was built using NetBeans 6.7, and you can download the built PrintExplorer.jar file.

Run as usual:

java -jar PrintExplorer.jar

(On Windows you can just double-click the JAR file.)

I filed two reports (so far, with review ids: 1660182 and 1660234) with bugs.sun.com, though for all I know that may be a black hole (or not … I have no idea). With the OpenJDK and JDK7 work there may be better places to report/record this … I do not know. (If you know a better path, please let me know.)

Found so far:

• PrinterJob.getPageDialog() makes invalid change to page size. (log)
• Toolkit.getPrintJob() changes JobAttributes.getPrinter() to default(?) printer. (log)
• Quarter-sized pages sometimes printed on an HP DeskJet 5550 on Windows 7. (log)

I am sure there are more. :/

Note that PrintExplorer always generates a log file. The log file contains the parameter values and print operations performed. (You need to be acquainted with Java Printing, otherwise the test program will be entirely obscure.) If you can reproduce a problem, then you can see in the log file exactly what you did, and what was returned. (Useful when you lost track of the exact sequence of steps.) When you send a log file, anyone can verify the exact sequence of steps you followed (a mistake in the problem description can be caught).

At this point the DocPrinterJob works for an empty Doc and a Pageable Doc, but not for a Reader Doc. (Unclear if the last is my bug, or a Java bug.)

Update – 2010.02.01
Today the filed reports became public-visible bugs 6921596 and 6921597.

My original query as represented in Canon’s support system (and trimmed):

Original Message Follows:
————————-
Email Support Form Message
Product Type: PIXMA MX860
Product Serial Number: absx22394
Date of Purchase: 11/18/2009
First Name: Preston
Last Name: Bannister
Address: 14 Vallecito
City: Foothill Ranch
State: CA
Zip: 92610
Phone Number: 9495880872
Email Address: preston@bannister.us
INQUIRY: Need drivers for 64-bit Ubuntu (Debian Linux).
Found the 32-bit drivers and sources on the Canon European site, but the sources are incomplete, so I cannot compile a 64-bit version.
Either compiled 64-bit drivers, or a complete set of sources would be acceptable.

The response was pretty predictable, if not exactly addressing the question as posed.

On Fri, Nov 20, 2009 at 5:09 AM, Canon Support – MFP <mfp @cits.canon.com> wrote:

Dear Preston Bannister:

Thank you for contacting Canon product support. We value you as a Canon customer and appreciate the opportunity to assist you. We regret the issues experienced with locating Linux drivers for the MX860.

Canon USA does not support Linux with its consumer products. The only recommendation I can offer would be to check the various Linux enthusiast websites to see if anyone has successfully attempted writing a third party driver.

We apologize for any inconvenience. Please let us know if we can be of any further assistance with your MX860.

Sincerely,

James
Technical Support Representative

Since there is always a chance (if small) of a constructive outcome, I tried again.

What I did was find that the Canon Europe and Canon Australia sites distribute Canon-developed Linux drivers for the MX860 (and other printers).

The software is pretty well done, has an active documented development history that goes at least to the year 2000, and is distributed as both binaries and source.

Odd that Canon USA does not distribute Canon-developed drivers (in which Canon has made a significant investment).

This is a case of so-close-but-not-quite there. The effort required for Canon to have a complete offering is pretty small, if someone can find the right branch of the company hierarchy to nudge.

Any chance of finding who within Canon sponsored development of the existing drivers for Linux?

The last response, which sounds dangerously close to “Fuck you. Go away.” (Though I would gladly be proven wrong.)

Dear Preston Bannister:

We appreciate your continued correspondence regarding Linux drivers for your PIXMA MX860.

I have forwarded your comments to Canon USA through our Customer Feedback process. This process allows us to capture important feedback from our valued customers. As we constantly strive to improve our products and services, your comments are vital to our continued success.

We hope this information is helpful to you. Please let us know if we can be of any further assistance with your PIXMA MX860.

Thank you for choosing Canon.

Sincerely,

George
Technical Support Representative

Had to try….

Billed To:
preston@bannister.us
Preston Bannister
14 Vallecito
Foothill Ranch, CA 92610

Order Number: MGLN2G836G
Receipt Date: 11/19/09
Order Total: $4.99
Billed To: MasterCard …. 1234

Item Number
1

Description
변환기, v1.0, Seller: Wang Xi|299225365 (4+)

Unit Price
$4.99

Write a Review
Report a Problem

Subtotal: $4.99
Tax: $0.00
Order Total: $4.99

Please retain for your records.
Please See Below For Terms And Conditions Pertaining To This Order.

Apple Inc.
You can find the iTunes Store Terms of Sale and Sales Policies by launching your iTunes application and clicking on Terms of Sale or Sales Policies

Answers to frequently asked questions regarding the iTunes Store can be found at http://www.apple.com/support/itunes/store/

I do not have iTunes installed. Both my desktop and laptop are running Ubuntu Linux. Also I cannot read Chinese Korean, so I do not know what was ordered. The “Report a Problem” link asks me to install iTunes.

I am guessing the iTunes store has a (big!) problem.

Update:
No charges have appeared on my credit card (so far).

I eventually found a “support” link for the iTunes site where you fill out a form to send email. Had to lie a bit as you cannot complete the form without indicating the OS on which you are running iTunes. Linux is not offered as a choice. The first email from Apple was not useful (“run iTunes to check on your account”, grrr). I replied with a copy of the receipt, and they did … something.

If no credit card charge appears, my guess is the Apple caught the problem part-way through.

Got an email from another iTunes customer who also got a receipt with the exact same mysterious item. Interesting.

Update (2):
The charge just appeared on my credit card, so it is not clear the exchange with iTunes Support addressed the problem.

Update (3):
Shortly after my last email (where I again described the problem, and added the other guy’s receipt), Apple Support sent an unexpected email refunding the charge to the other guy! Right. Half-successful?

China to investigate U.S. car subsidies
China is preparing to launch a trade investigation into whether US carmakers are being unfairly subsidised by the US government, according to people familiar with the matter.

The move comes at a time of heightened trade tensions between the two countries after the US imposed duties on Chinese tyres last month. Many warned this would prompt Beijing to retaliate.

Interesting. Is this shallow, or deep?

China is using extraordinary measures to keep the dollar high and the yuan low. This has huge effect on trade. Words from the US to China have no leverage, at least compared to China’s near-term benefit from continued exports.

What has leverage? An import duty on tires … really … why tires? Is there some shallow US domestic political motivation? Or is this an opening move with enough lead time prior to Obama visiting China?

The Chinese government used to subsidize all sorts of industries, and they probably still do to some degree. At least some of that goes into exports. Tagging the US for subsidizing an industry – one that likely does more imports from China than exports to China – is odd. If the yuan rises relative to the dollar, then US exports to China become more competitive. Bit of a doubled impact on the Chinese economy (less exports and more imports). Discouraging imports from the US through import duties could lessen that impact. Is this a pretext for the logical counter-move (but not a strong move). Again, is this shallow or deep?

In this game, China is on the wrong end of the lever. Less trade between the US and China is likely to hurt China far more than the US. There may be some point in the future when they can absorb the loss of exports to the US. Is China at that point now? Or more relevant, do the Chinese leadership believe China is at that point?

My bet is that the Chinese leadership is too conservative, too cautious to want to risk too much change. China is a country of “rule by men”, not “rule by law”. Obama meeting with the Chinese leaders would be the point where – privately – the game is made plain … if this the deeper game. If China allows the yuan to rise after the meeting – that would be a big clue. (Likely China will keep buying dollars, but at a lesser rate – a sort of compromise.)

Is this shallow, or deep?

The problem is buttons. A change in GTK+ 2.18 broke Eclipse, and anything based on Eclipse.

There are a number of bug reports tracking this problem. On the Eclipse site look at:
Bug 291257 – [Widgets] Buttons functionality problem with GTK+ 2.18

There is a work-around offered on the above ticket (setting the GTK_NATIVE_WINDOWS environment variable). It seems to work for some. For others – myself included – it causes another severe problem (for which there are other bug reports).

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f30eb1e7f7a, pid=4766, tid=139848919398672
#
# JRE version: 6.0_16-b01
# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libpango-1.0.so.0+0x24f7a]  pango_layout_new+0x2a
#

Strictly speaking, this is a bug in GTK. A new version of GTK shipped that was meant to be compatible, but instead broke existing applications (at the very least those based on Eclipse). The application use of GTK might (or might not) be considered in some way improper, but practical considerations should be dominant.

Fixing this in GTK will in one step remove the problem in all currently impacted applications. Fixing this in Eclipse will only help new and newly updated Eclipse installations. Existing Eclipse-based applications will continue to be impacted until (and if) updated by their respective developers.

Other related/duplicate bug reports:
Buttons in Eclipse not working correctly with GTK+ 2.18.1-1
Eclipse loses button events with swt-gtk in karmic
GDK_NATIVE_WINDOWS=1 still required for some dialogs
Breaks Default button in some eclipse dialogs (and rcp apps)

What’s Wrong with the Culture of Wall Street? – TIME
Before this crazy crash of 2008, bankers always landed on their feet, almost always. Job insecurity isn't the same thing for the average American worker. They often experience downward mobility or don't land on their feet.

New Device Extracts Oxygen from Moon Rocks | Universe Today
Fray anticipates that three reactors, each a meter high, would be enough to generate a ton of oxygen per year on the Moon. Three tons of rock are needed to produce a ton of oxygen, and in tests the team saw almost 100% recovery of oxygen, he says.

The comments got the math badly wrong. Poking around the web I got some rough numbers:

• ~0.25 liters / minute – oxygen consumed for human at rest.
• ~5 liters / minute – oxygen consumed for athlete / heavy exercise.
• ~1.43 grams / liter – oxygen gas at Earth sea level pressure.

Barring any dumb math errors on my part, that translates to….

• ~636,000 liters / year – the ton of oxygen produced by the above-mentioned generator.
• ~131,500 liters / year – oxygen consumed by a person at rest.
• ~4.8 person-at-rest-years of oxygen generated every year.

Assuming that a “real” space habitat would also have the means to recycle oxygen from exhaled CO², that is a lot of oxygen!

What is the real need for that much oxygen? Outside use as rocket fuel (or other manufacturing process), the main need would be to supply air for expanded living space. Assuming ~3 meter ceilings, we need ~3000 liters air for each square meter of living space. So one generator would allow expanding the living space by ~2000 square meters / year.

Not bad. And that is just one reactor (and essentially a prototype, at that).

Only one part of the equation (for habitat supply needs), of course. For practical purpose, would be nice to also have some nitrogen in the air. Not sure where that would come from. Do any moon rocks contain nitrogen? Water is also needed. Barring any fortunate discovery of water on the moon, we could synthesize water given a source of hydrogen. Do any moon rocks contain hydrogen?

• Conversations are a first-class concept, and each message is a part of a conversation.
• Each message can be presented by a different application.

Because Wave is on the web, the set of applications is come from all applications in the world (on the web).

E-mail clients can present “threads” that are something like conversations, but this requires a bit of guessing and fakery on the part of the client, and is easily confused. IM clients can present a log of messages – that looks something like a conversation – but does not work when you switch between devices. Given that email and IM servers know nothing about conversations, the clients do pretty much the best they can. If you know about conversations on the server, you can do a better job on the client.

When conversations are a first-class concept, there are lots of new possible functions.

On the flip side, the Google Wave folk have made some (common) missteps.

• Using new names for existing notions is sometimes useful, but more often not. Calling a conversation a “wave”, calling items (messages) within a conversation “blits” … I find this more confusing than useful. This reminds me more of talk from marketing folk than engineers.
• It’s Talk, on the web. As a programmer, I get it – this is kewl! Writing a web application that allows all participants to see each character typed – is tricky. As a demo of tricky programming, this is sure to impress. As a useful feature in a general use application – not so much. Character-at-a-time message traffic is a huge load increase on the server, and a disservice to the users. Think about:
  • Locus of attention – the on-screen activity from updates to other users’ incomplete messages is sure to distract.
  • Quality of expression – how often have you started writing a message, then revised what you said and how you said it, before sending? Do you really want everyone to see your missteps … always?

  Note that both are a disservice to all participants – cool in a programming demo, not cool in a deployed application.

Other bits….

• It looks like Outlook. This was new and cool in 1997, but not so much now. Not at all sure this layout is efficient, especially given the changes in most-common screen layouts. In 1997 you would target 640×480, 800×600, and 1024×768 on 14″ to 17″ as most common screen sizes. Today you are looking 1280×800 on 14″ or 15″ as your minimum size, and 1920×1080 at 20″ or bigger as common. Present and future screens have not so much new height, with lots of new width – in fact too much new width for text content. Seems a different presentation is in order.
• Mixed applications in one web page is cool and exciting as ActiveX was in the web browser in 1996. Viewed in terms of the good things it can do, ActiveX allows an open-ended mix of applications to present within a web page – very cool. The Microsoft folk were blindsided (as constructive, well-intentioned folk) as they did not consider all the bad things that could be done (and with ActiveX, very bad things are possible). The Google folk have lots of history to make them wary, and the scope for harm is much smaller, so likely little bad stuff get through. Still – do not underestimate the black hats. There are bound to be some exploits (though likely few and quickly suppressed).

In sum, a decent start on a great idea. I’d want to be pretty ruthless about overhead that takes height away from the conversation view. I’d drop the per-character updates. I’d drop the use of funky names for existing ideas. None are fundamental problems, and all are easy to address.

Fixing up an old Java desktop application for viewing old “green-bar” reports. Should be pretty simple – the reports are all fixed pitch text. Given I like to do things that “just work” (from the users perspective), I’d added live “smart” definition of controllable font-auto-sizing, empty page suppression, and margins. Given that old printed reports may be formatted for paper sizes different than in your desktop (or network) printer, this was slightly tricky, but only that.

Of course, many of those old reports – meant for old line-printer green-bar paper – are best printed in landscape orientation. To my surprise, I find when in landscape orientation, the return from PrinterJob.pageDialog(PageFormat) consistently alters the margins, even when no changes were made, and the error is cumulative.

Um, what?

At first I assumed the error was in my code, but this bit of logged output caught the problem:

PageFormat validate::
	paper dx: 792.0 dy: 612.0
	margins top: 72 left: 72 bottom: 72 right: 72
	orientation : LANDSCAPE
PageFormat after pageDialog::
	paper dx: 794.0 dy: 613.0
	margins top: 72 left: 74 bottom: 73 right: 72
	orientation : LANDSCAPE

The first is the contents of the PageFormat before calling PrinterJob.pageDialog(PageFormat), and the second is the new PageFormat returned. Note the +1 dy bump and +2 dx bump to the page extents (and the matching errors introduced to the left/bottom margins).

The code I used to remove the error:

            controller.pageFormat = job.pageDialog(controller.pageFormat);

            // Work around an interesting(?) bug in PrinterJob.pageDialog().
            // For some reason the returned Paper is over-sized. (Why? Dunno.)
            // Use the Paper returned from PrinterJob.defaultPage() to get a proper size.
            // If the Java folk remove the bug, this code should change nothing.
            PageFormat pf = job.defaultPage();
            Paper paper1 = controller.pageFormat.getPaper();
            Paper paper2 = pf.getPaper();
            double x = paper1.getImageableX();
            double y = paper1.getImageableY();
            double dx = paper1.getImageableWidth();
            double dy = paper1.getImageableHeight();
            paper2.setImageableArea(x,y,dx,dy);
            controller.pageFormat.setPaper(paper2);

The bug was encountered using Java 1.6.0_15 (64-bit) on Ubuntu (Linux 2.6.28-15-generic #52-Ubuntu SMP Wed Sep 9 10:48:52 UTC 2009 x86_64 GNU/Linux).