Marrying Ephemeral Docker Containers to Persistent Data


Docker containers are ephemeral by design. They come and they go like a herd of hyperactive squirrels, which is great for high availability, but not so great for preserving your data. Kendrick Coleman of EMC {code} demonstrated how to have both ephemeral containers and persistent data in his talk called “Highly Available & Distributed Containers” at ContainerCon North America.

As container technologies become more complex, using them becomes easier. Coleman gave a wonderful presentation using a Minecraft game to demonstrate persistent data storage with ephemeral containers, and did it all live. This setup requires two technologies that were not available as recently as a year ago: Docker SwarmKit and REX-Ray.

SwarmKit is an easy-to-set-up cluster manager, in contrast with the older Kubernetes which is not easy to set up. SwarmKit is a major upgrade to Swarm, the native Docker cluster manager. Coleman said, “SwarmKit is where life is becoming more and more simplified. It’s integrated directly into the Docker engine. It’s an optional feature so you don’t have to start Swarm if you don’t want to. It has desired state, you’d be able to say, “I want this many types of containers running for this particular service”, as well as reconciliation, “If I lose a host” it’s going to restart those containers on a different host as well. It’s decentralized by having concepts of managers and workers, so now you can delineate work out to different things.”

SwarmKit provides load balancing, service discovery, rolling updates, and secure TLS networking with the option of using your existing certificates.

REX-Ray provides storage orchestration, a vendor- and storage-agnostic tool that links your persistent storage to your now-you-see-them-now-you-don’t containers. “REX-Ray is our Docker volume driver that we had written at EMC, and it’s still under heavy development, and it’s a completely open source project…REX-Ray is a super-simple installation and a super-simple architecture. It’s a stateless application that only needs a configuration file, and it’s a simple cURL-Bash command that installs a binary,” Coleman said.

Container technologies are improving so quickly that “Clustering, networking and failover with persistent applications has gotten exponentially easier than it was even four months ago”, said Coleman. See for yourself how easy in the video of Coleman’s presentation below.


You won’t want to miss the stellar lineup of keynotes, 185+ sessions and plenty of extracurricular events for networking at LinuxCon + ContainerCon Europe, Oct. 4-6 in Berlin. Secure your spot before it’s too late! Register now.