Last updated at Fri, 03 Nov 2017 20:44:16 GMT
Wow, this cloud OS scene is really speeding up. Without going all Ray Kurweil on you, I can’t keep up – literally! In my last blog post, I said I’d cover Kubernetes in more detail, explore what plans Mesosphere might have for Mesos and explore how Linux containers would play with clustering.
In the intervening six weeks, Google launched the Kubernetes-based Google Container Engine, so I started writing about that. But then CoreOS came out to put containers in their place; and then Mesosphere announced a $36M raise and confirmed that their vision is to build an operating system for the data center AND then they called it DCOS just to avoid any confusion!
The CoreOS announcement is important not so much for what they launched (Rocket – a new simplified container management system) but for the suggestion that Docker is evolving into a “platform”… so is this another contender / pretender to the Data Center OS throne? CoreOS called Docker fundamentally flawed – ouch! No wonder there was such a strong reaction. Particularly interesting is that as Docker’s ambition increases with their valuation, the vision now includes “systems for clustering.” So maybe this deserves a future blog post all of its own.
The nice thing about the new positioning of Mesos is the concept of killer apps. Once you come out as a fully-fledged OS you need killer apps, right? And we have all the buzzy killer apps. Hadoop and Spark are a no-brainer. Massive scale data crunching is where most people will start when it comes to building their own clusters (that is what we do at Logentries after all). But name-checking Kafka, Kubernetes and YARN is very interesting.
I spoke before about Jay Kreps vision of the future of enterprise computing being built around the log. Kafka is the technology that helps scale the ingestion of events (including log entries) at cluster-scale, providing a solid foundation plartform for real-time event processing. And, of course, Jay and his colleagues have spun out Kafka from LinkedIn as Confluent (and taken an investment from LI at the same time). We must assume that a partnership announcement between Mesosphere and Confluent is imminent.
Then there is the positioning of Kubernetes and YARN as killer apps. I’ve spoken before about this positioning of Kubernetes being at odds with the vision presented by the Kubernetes team. It is clear that these two technologies (and YARN for that matter) compete. Yes, Kubernetes can bring some nice features to Mesos – pods (a group of co-located containers), pod labels for service discovery and load-blancing etc. But with Google’s launch of the Container Engine it’s clear that Kubernetes will evolve into a very critical piece of Google’s public cloud. But even more important is that this is not just about the public cloud. It’s about private and hybrid clouds and Google is very much chasing after the big enterprise market where public cloud brings convenience but private / hybrid clouds keep CFOs, Chief Legal Officers and risk-averse CIOs happier. If Mesos is the data center OS then Kubernetes is evolving into the platform for roaming microservices that live everywhere and nowhere.
Containers facilitate this migration but, as Microsoft has shown in the past, the big prize goes to the platform winners. So it’s game on. Docker Inc. as a contender? Sure. Kubernetes – definitely here to stay. And if Mesosphere can genuinely add value as a platform for YARN and Kubernetes, as microservices leap from data center to data center (public, private and hybrid) then it really will justify the title of world’s most disruptive startup.
