random memes }

Long-term compatible protocols

Prompted by this article: Lightweight versioning for lightweight protocols

Yes, this is a many-times-solved problem - though not always well solved. Seems like most everything I've done in the last twenty-odd years (or more) has had a network in the middle. From that experience, a few guidelines...

Guidelines for network protocols used in distributed applications

  1. Design for differing versions on the client and server.

    In some limited cases you can keep the client and server versions in sync. For the most part - over time and over growing deployments - you cannot. Assume that the client version will often differ from the server (in either direction).

  2. Convert at the edges.

    The types inside your application may change many times (for good reason). The structures that cross the network should change slowly (if at all). In your application, convert from internal to protocol structures at the edge.

  3. Version your protocol.

    This could be as simple as a single version number offered by the client to the server, and the server to the client, at the beginning of the conversation. You could do finer-grained versioning, but this seems rarely needed.

  4. Do not be over-specific in your protocol types.

    When passing a number across the net, use a general number type. You application might internally use a small number type (8-bit or 16-bit), and you may think that is all you will ever need. Some of those numbers will grow. At the edge, when ingesting incoming data, check for values outside the range you can handle.

    The same argument applies to strings, arrays, and the like.

  5. Design for upwards and downwards compatibility.

    When the client asks a question of the server, older servers may not understand the question, or return a smaller answer. When the client is older, the server may have a larger answer, or the question may be smaller. Most of the time, changes are incremental, and both server and client can handle the difference without explicit reference to the protocol version. Sooner or later someone will goof, and you will need to add a special case for a specific version.

This all might sound complex or difficult - but is simpler than the alternative.