random memes }

Guess I'm a little slow...

Self-judgment is always tricky. Some bits took me a long time to figure out.

Whenever I go into a new situation, I expect to find someone much smarter. It never seems to work out that way. Often I find folk as-smart, but never (yet) anyone I thought clearly smarter. (Worse, I tend to over-estimate the abilities of folk I know.)

Yet my expectation persists.

I have had some occasional odd interactions with folk in the past, that I did not understand at the time. Easy to forget that other folk have insecurities of their own. Years after one episode, I figured out the most likely and unexpected explanation.

My career-path meant I got heavy exposure to client-server programming very early. I tend to think through the issues fairly throughly (perhaps I have an odd notion of "fun"), when presented with an interesting problem. A decade later, I started at a company that had a client-server product. The product had clearly got some of the principles I had arrived at, badly wrong. After starting at Quest, and seeing the current-generation product, I heard there was a guy working on the next-generation of that same product. So I went to the guy and outlined all the things I thought he should do for the next generation product. On one point he said something like "we thought about doing that, but it was too hard", for which my answer was "we did that on my last project, and it worked out very well". I think the guy's name was Jeremy.

I was just trying to be helpful.

What I did not know at the time was that what I had seen demonstrated was in fact the next-generation product, the result of two years of Jeremy's work. So instead of criticizing the old generation to offer advice for the new generation ... it must have sounded to Jeremy like this new guy was tearing apart his best work. The product was called "Vista".

Then things got much worse.

Not many folk are intimately familiar with both Windows and Unix programming. Vista was a Unix-only product. Most folk in that outfit were Unix-only. Since I knew both platforms, I got tasked with porting Vista to Windows. Another guy - a long-term hand at the company - was tasked to work with me. The other guy - John? - I knew and liked. He was pretty sharp, but had other obligations. After a few weeks he bowed out, claiming that he was only going to slow me down.

I am a sucker for challenging problems - and always have been.

There were about a half-dozen programmers working on the Vista product. I had to keep up with their changes, while building a port to Windows, against an ever-diverging source-base. After six months of very intense work, I had a dual port. The same source base could be compiled to generate either a Windows or Unix version of the product. To keep up the necessary pace, I had isolated myself from most of the daily workings of the group. Along the way I found some perfectly astonishing bad code. I had assumed that programmers with equivalent levels of experience, but who had spent their entire careers doing only Unix programming, would be far better than I in that domain. Working through the Vista source code, I found out that I was wrong - and very badly wrong. Since I was under huge time pressure, I took an extremely pragmatic approach. When I found a malfunctioning section of code, I first tried to read through the code. If I was unclear what the code was meant to do, I re-wrote the section of code for clarity. At this point I usually realized the original code contained a number of errors. Code-level errors do not necessarily translate to customer-level errors, and the pressure on my time was enormous, so I made no effort to report my findings.

When I had finished, the group cut-over to using my source-base. They were a bit surprised to find that I had kept current with all their changes. (I had a secret weapon. They were using RCS-based source code control. I used CVS to track their changes.) What was worse was that I had almost-accidentally killed off a large part of their bug list. The code inspection and re-coding for clarity was just to make the port work, but had the side-effect of eliminating a lot of existing bugs. By this time Jeremy had been made manager for the group, and he called me in and yelled(!) at me, while admitting this. I was confused.

What I did not realize - until years later - is how this looked from Jeremy's point of view.

Imagine a new guy at your company walks into your office, and details all the things wrong with the project you have worked on for the past two years. Imagine he claims to have done something you thought was too hard to do. Now imagine that the new guy is tasked with something you cannot do. You expect him to fail (and I know he did). Instead of failing he delivers a result that not only works, but is much improved over your best work.

Did I mention that Jeremy was made manager for the Vista group, and that I reported to him?

Midway through the project, I realized each client instance of the server in the Unix implementation of the Vista server was started via inetd. Since the Vista server-side incorporated Ghostscript, and the Ghostscript initialization took 20-30 seconds (loading a large amount of font information, mainly), this put a serious crimp in per-client startup time, and a pretty serious load on the server when there was a lot of clients. Since the Ghostscript initialization was the same for all clients, if the Vista server was run as a server, the per-client startup could be reduced to milliseconds, and the total load placed on the server hugely reduced. I went to Jeremy to explain how this could be done. He angrily rejected my suggestion, for no logical reason that I could discern. I was puzzled, but went back to the all-consuming task of completing the port.

Imagine that annoying new-guy offers a suggestion (which he has already implemented and demonstrated) that improves the startup time of your pet project by a few-thousandfold. At this point Jeremy really did not like me, and I had no clue as to why.

Naturally I had taken the same approach with the Windows port, so the per-client startup time was almost instantaneous.

After finishing the Vista port (to the great pleasure of the Marketing folk, who could now sell Vista into the Windows market), I got tasked with another port. I had a conversation with a distinctly grumpy Jeremy, who had to admit that the Vista port worked, and since I was apparently some sort of legend (not to my knowledge), I was tasked to port another product.

The next product was called "Shareplex", and was a database replication product for Oracle databases.

I had used Oracle as a programmer/user a fair amount while at FileNet, so I was pretty familiar with Oracle at a user-level. Knowledge of Oracle at the reversed-engineering level needed for third-party database replication was entirely outside what I knew. Moreover the guy in charge of the group had written a book on Oracle performance tuning, so I felt entirely out-classed on both fronts.

Again, I was one guy trying to keep up with an active development group of about a half-dozen, while performing a port from Unix to Windows. It quickly became plain that I could not perform testing in the same labor-intensive manner as the existing group. (It also struck me as odd that there were no comprehensive test suites for a potentially mission-critical product.) So I came up with a series of Perl scripts to perform a series of automated tests against the database-replication product. The tests created, exercised, and destroyed a series of progressive, then semi-random, series of structures, data, and operations.

At the end of the port - another six months of very intense work - I had another dual-port. The same sources could be compiled for either Windows or Unix. The tests I had built proved that the port worked just as well as the original Unix code - and also proved that the original code did not work.

I was stunned. I had no idea how to proceed from that point.

What I did not realize at the time was that I had given two important group managers reason to hate me. A few weeks later, Charles - the Shareplex group manager - escorted me to a meeting, where I was fired.

Much later, I learned that the Shareplex product was withdrawn from the market, and took another 2-3 years of rewrites before it became a marketable product. The conclusions I derived from my tests were right, but I did not, at the time, know what to do with the information.

Without knowing, I had intimidated co-workers, by doing things that they could not.

This morning I made a different connection along the same lines....

When I worked at FileNet, the company hired in a new graduate from CalTech. Management treated him as if he was a living legend. Dan was a fairly smart guy, but I kind of doubted his ability to walk on water. Fairly early on, he did a performance analysis in the area in which I was working, and made a couple fairly elementary mistakes (for a first-year Physics student), and circulated some bogus results. I sent out a correction, but did nothing else.

What I did not take into account was his likely insecurity as a guy promoted well past his peers. Any sort of criticism was likely seen as a serious threat.

Fast-forward a bit over a decade. A marketing guy at my then-current employer wants to go talk to the folk at FileNet. I don't see a likely gain to the exercise, but went along to support the marketing guy. We end up waiting in the entrance lobby for an oddly long time. Dan walks through the lobby. Dan claims not to know me, with a hint of anger. Odd. To put this in context - the lobby is not on a natural path between any two points within the building (I used to work there). That Dan would walk through the lobby at the time I was there was ... odd. That Dan would not remember me (we worked together for about seven years), is ... odd.

Since the episode was of no importance, the odd bit stuck, but was otherwise forgotten. In retrospect, if I had indeed made Dan feel threatened, then the combination of oddities add up to a probability. Is it possible I had made Dan feel threatened? Is it possible that we were made to wait until Dan could walk through the lobby? At some point Dan was promoted to CTO. Was the odd delay, odd diversion, and odd lack of recognition ... not odd at all?

This morning's amusement - of no further import. :)

Now ... I could be wrong about any of the above. On the other hand, even if right, it seems unlikely that anyone involved would confirm anything. Oh well.