random memes }

Coming up to speed with OpenStack

Was asked to look at OpenStack a few months back. Had read a bit prior, and was impressed with the velocity and vigor of this open source cloud platform and project.

Made the mistake of starting with OpenStack. Read a large chunk of the OpenStack documentation. Stood up instances of devstack both on virtual and physical hosts. Not everything made sense.

OpenStack exists because of Amazon Web Services (AWS). Amazon does an excellent job hosting cloud applications, and is the largest player. As a developer, building a cloud application for AWS makes good sense.

Some organizations want to host cloud applications in-house. Hosting an in-house cloud can make sense for a variety of reasons, ranging from performance through cost, control, and legal requirements. If you want to host your own cloud, you want a cloud that works for both existing and newly developed cloud applications. Since cloud application developers are most likely to build for AWS first, you want a cloud as close as practical to AWS.

So OpenStack must be close to AWS from a developers perspective … and is, mostly.

To understand OpenStack, you really need to understand AWS, first. The OpenStack documentation is not bad. The AWS documentation is much better. You can play with AWS very easily, and at very low cost. (Part of the reason AWS is a success is the ease of starting.) Once you are acquainted with AWS, you will find OpenStack easier to understand.

That said, the OpenStack community does at times lose focus.

To be clear, the AWS API can best be viewed as a thin wrapper around Amazon’s cloud services. As AWS evolves, the AWS API evolves. The API is not a set-in-stone independent entity. The Amazon folk are thoughtful in their choices. They change the services and the API carefully, so as to minimize the impact on developers. But the AWS API does change over time.

A few years ago, Mark Shuttleworth (of Ubuntu) made an argument about OpenStack and AWS compatibility. While in general I strongly agree … Mark seems to be arguing for an exact emulation of AWS. I think this is a mistake … but not far from right.

The OpenStack API is a wrapper around services offered by a particular instantiation of cloud services. The services offered might vary somewhat. Different projects and vendors want to plug in differing implementations behind the service APIs. I think this is a good thing. An organization might want to stand up a cloud with emphasis on different aspects of performance, to suit their usage of applications in the cloud.

As a cloud application developer, I like AWS quite a lot. If considering an alternate cloud, the effort on my part to move my application should be as small as practical.

Small but not zero.

If a cloud service has differing characteristics, my use of that cloud may need to be different.

The AWS API is exactly designed for AWS. The AWS API is not designed for cloud services of differing character. As such, the AWS API is not well suited for differing clouds.

There are parts of the AWS API that should be exactly emulated - the S3 service and API is an obvious candidate. As a cloud application developer I am almost certain to make extensive use of S3 (as simply an excellent web-scalable object store).

The other AWS APIs … not sure if I care. For operations I use much less, all I want is well-defined and well-documented simple mappings from AWS APIs to OpenStack usage. If the services are of differing character, I might need to make differing usage. If the differences are small, the work on my part may be small enough I do not care.

So I do not care about exact emulation in all cases. I do care about minimal difference.

If you are trying to understand OpenStack, start by reading up and playing with AWS.