Facebook

I've come to the conclusion that Docker is actively harmful to organizations. Not the underlying technology...I think LXC is fantastic as are cgroups. And Kubernetes can solve real problems around improving efficiency in production.

No, the problem is with the Docker community and the hype around it. At its core, Docker allows organizations to paper over very real problems. It doesn't solve those problems, they are still there festering...just given more time to really build up pressure before exploding. But make no mistake, they will still explode.

Two of the main benefits I hear touted for Docker is it gets Operations/Production Engineering/SREs out of the way of Engineers and it gives Engineers a consistent and reproducible environment. Docker solves neither of these.

Organizationally, if your Engineering organization feels like it has to go around your Ops/PE/SRE teams, you've got something fundamentally wrong with your organization. Focus on fixing that first. Either your Ops team is behaving too much like gatekeepers instead of enablers OR Engineering is sidestepping their very real concerns about running the latest and greatest version of a year old project in Production. Likely there's some combination of both happening. Those problems _still exist_! You still have to fix that organizational divide! And guess what...now that Engineering and Ops are talking less the resentment and divide will only increase. No, the answer can't just be dumping all of production onto engineers or just renaming your "Ops" team to "DevOps" and calling it good.

Technically, Docker promises reproducible environments. Which is true once the image is produced. You can ship that image all over the place and it will come up the same. I have no problem with this...as I said before I think there’s a ton of value in that by enabling technologies like Kubernetes that actually solve problems. The problem is in, however, how those images get produced. Take https://github.com/CentOS/CentOS-Dockerfiles/blob/master/wordpress/centos7/Dockerfile#L17 for example, from the official CentOS Dockerfile repository. What’s wrong with this? IT’S DOWNLOADING ARBITRARY CODE OVER HTTP!!! This is the same issue that I railed against Maven for for years. See http://blog.ontoillogical.com/blog/2014/07/28/how-to-take-over-any-java-developer/ for a POC attack. That same attack now works against anyone building Docker containers. Also, which version of Wordpress was installed? Say I need to do a security patch on the parent image...how do I get that same version as I had in production before?

There’s nothing technically forcing teams to do this, but this appears to be the prevalent way the community has decided to build containers. Grab random, unversioned code from the Internet and execute it without checking anything about it. Now those same attacks that worked against Maven are still there...there is just a smaller window. The core problem, however, hasn’t been solved and the developer has been lulled into a sense of safety. The developer has been fooled into believing they have a repeatable build when really they’ve just left a time bomb for the next person to deal with.

This is to say nothing about the Docker project itself, which seems to think LTS is for suckers and everything should be bleeding edge. Bleeding edge FS. Bleeding edge Networking. Glibc? Fuck GNU as a staff, open source organization, and as a fucking crew. Let’s switch to Musl. The that really sucks about this is there is so much promise. Containers have the ability to radically improve efficiency and deployments, but in no way is it a cure all. Google has been doing this for decades with Borg(https://research.google.com/pubs/pub43438.html), but they also solved repeatable builds with Blaze (https://www.bazel.io/). Facebook has done similar work and built a fantastic Production Engineering organization that worked extremely closely with Engineering such that no one felt like the best answer was go to around them (Thanks Pedro and bethanye!!).

Sometimes you have to do the hard work to solve the core problem. Sometimes there are no silver bullets, only lead bullets (http://www.bhorowitz.com/lead_bullets). These problems will spectacularly explode. I just hope they don’t take the future of running containers with them.