Mark cavage on agentic AI, sandboxing & enterprise security | ep 10

Hello and welcome to another episode of future of data and AI. I’m Raja Iqbal. My guest today is Mark Cavage. Mark is the chief operating officer at Docker. Welcome to the show, Mark.

Thanks, Raja. It’s great to be here.

Great to have you. So, Mark, you started your career at AWS.

I’m even older than that, actually. It was once upon a time at Nortel Networks, who doesn’t exist anymore. But I spent my early formative years at IBM, and then in the early 2000s — 2005 or 2006, I always get confused — I went to AWS.

And then you transitioned to Stripe, and you were one of the early folks at Heroku.

Yeah, so I spent years at AWS and built out what I will call the first one, maybe V1 to maybe V2, V1.5, somewhere in there — the first couple versions of AWS. And believe me, it’s all been rewritten probably 10 times over by now. But really, through that early wave of AWS turning into the financial growth hockey stick, I actually left there and went to Joyent, which was the Betamax of cloud computing and big data. But we also did Node.js — I think that’s what a lot of people remember the company for. And then I left there and founded what’s now Oracle Cloud Infrastructure. And after that, I found myself at Heroku, after Salesforce had acquired Heroku years later. I worked on transition to serverless and the rest, then Stripe, then a small healthcare startup, before coming to Docker. So that’s my arc in 60 seconds or less.

What was the pitch like internally at Oracle to take on AWS and Azure? Do you think you guys were a bit late?

Well, we were a decade late. It was very frenzied and very much a constant sprint to go as fast as we possibly could. But the funny thing is, from the day we got there, we knew we were doing the right architecture. There was a small set of us from AWS that had founded it. A lot of it was built on what we called the Gen 2 cloud. The real innovation was building it as a bare metal first cloud — that was the base primitive that unlocked everything. True L2, L3 network virtualization, something like a VPC, but the ability to get ephemeral bare metal instances of any shape, size, and configuration on that kind of network in a cloud-like way. Beyond normal virtualization things you’d expect in VMs, you could get very high-scale computers, run very large databases, run ML workloads on them. That was the genesis of it.

To answer your question on the pace — yeah, we were in market and everyone was laughing at us for at least 3 or 4 years. I would enter countless meetings where the press would basically say, “Oh, you guys are kind of fools. I don’t know why you’re even here.” But I’d walk into customers and they’d say, “Everybody needs to be here.” So it was both a great experience and a grind of perseverance, as well as a pace to catch up. There were a lot of sleepless nights.

I imagine. So I joined Bing fresh out of college, and the internal joke was that they were hiring people so they could have more users, right? But when you’re working on such an important initiative within a company, you get to work with the smartest talent in the industry, right?

Yeah, well, I definitely don’t consider myself one of the smartest people, but we definitely got to work with a lot of the smartest people while there. Really, at all these different arcs — Amazon, Oracle, Stripe, and Docker — when you’re onto something and you have conviction of mission and belief in what you’re doing, and it’s actually something ambitious, that attracts a lot of smart people. Smart people want to work on hard problems. Those are some of the most rewarding points in my career — not just the technology we accomplished, but working with the people, learning from them and the problems. Smart people motivate you to want to be better yourself. It’s a virtuous flywheel.

And you get to meet people who are not just smart — they work incredibly hard, right? You’re always trying to catch up with them.

Yeah, it’s healthy competition.

And you met Don Johnson there?

Actually, Don and I were office mates along with Tushar, who’s also at Docker now, back in 2005 at AWS. That’s where we met and have known each other and been lifelong friends since then.

What are some of the lessons you’d like to share? You’ve built a hyperscaler cloud platform twice — you were one of the early people at AWS, and then again at OCI. What are some lessons you learned about building from scratch?

Well, the interesting thing about the first time around is kind of like now in the AI era — the cloud was very green field. When I left IBM for AWS in 2005, everyone told me, “You’re going to go work for the bookseller? And the bookseller is going to sell computers? What?” Nobody really got it. But if you paid attention, it was obvious this was going to be transformational. It was completely green field — we didn’t know what we were doing or solving.

The thing is, when you’re doing something green field, you have to learn everything you can from people who came before you. I’m a big fan of history. I spent my time at AWS reading mainframe papers from the 1960s and ‘70s, and Sun Microsystems papers on grid computing from the 1990s. The art of AWS was taking these ideas that existed in history and making them work on commodity infrastructure at scale. So lesson number one: do your homework on history — there are valuable things in there, even if you throw away 50% of the assumptions.

The OCI experience was different. When you show up knowing what you got wrong before, you want to do it totally different this time. By then, I’d transitioned from being an individual contributor to a manager and leader at Oracle. The hardest part was the cultural amalgamation of different peoples. You have this cohort from AWS who know what to do, but you’re injecting that with people not from that domain — and that was the hardest part to work through. The biggest lesson I learned was: don’t underestimate how important it is to get people on the same page with principles and cultural way of working, because that dictates and drives your success more than anything.

If you got a third chance at building something similar from the ground up, what would you get right? And what would you do differently?

In terms of cloud now, with Docker, or looking at AI — AI gives you the latitude and mandate, and necessity to rethink first principles because everything about the user you’re serving changes. What we get right, I’ll liken back to the Unix philosophy. Unix has served us for 40 years. It’s a beautiful system with elegance in how you compose things — small tasks that do simple things. When you look at AWS, the best SaaS architectures, the best big data architectures, they all figured out the same thing: build microservices or agents that are purpose-built, do a small thing, don’t do surprising things, and can be composed with others. That’s how you get to something that looks like a platform.

When you think about platforms, you have one when the next three things on it have surprised you that you never thought of. When I look back at AWS, Docker containers, or Oracle, the rewarding thing is when people start figuring out what this unlocks for them. I’m most proud that these things have stood the test of time — sure, they’ve been rewritten internally many times, but the constructs and primitives from 2005-2007 are with us now and will last.

The most exciting thing to me is that we get the opportunity to build a new platform again for the agentic world. Platforms are like playing nine-dimensional chess. You have to think through your problems, your users’ problems, your users’ users’ problems — layering top to bottom and unlocking compounding use cases. That’s what’s exciting about now.

You joined Docker about a year and a half ago, along with Don. At that point, it must have been clear that containerization and Docker would be important in agentic AI — that separation, sandboxing, secure isolation.

Yeah, absolutely. The question is really: why come here? What do you see? And so, let me explain why Docker is so uniquely positioned in AI. If I go back to the very beginning — I was there when I built the first couple clouds — I actually think Docker is what unlocked the mass move to the cloud. It solved this fundamental problem. Before Docker, everybody had their own cloud. Microsoft had Azure. Everybody had something. The cloud was already fairly mature, and AWS servers alone had bleeding edge companies doing amazing things. We literally could not keep up building data centers and racking servers fast enough. But it still felt like less than 10% of the market could move to the cloud because it required specialists and experts that understood distributed systems and operations. The huge part of the problem was: developers would write code on their laptop, but AWS wasn’t like their laptop. Docker solved that simple problem — I need to get my code from my laptop to production. Build once, run anywhere, with isolation and ease of use. That was original Docker.

Since then, the container world has blossomed. I read a stat from Gartner or IDC saying 93% of the world now runs containers in production. Containers are now much like Unix — ubiquitous and lingua franca. Docker was the company, technology, and ecosystem that made the transition to cloud possible. It unlocked this massive wave of developers being able to more easily and accessibly use the cloud.

The giant opportunity and exciting thing is the agentic world needs a lot of the same problems. Portability, agnosticity, isolation, ease of use, developer friendliness. We want to help everybody access not just using agents, but building agents. I can use any model, any harness, any cloud, any device. Docker is both the ethos of the technology that allows that, as well as the technology people already use and trust. The core ethos of Docker — build once with anything, run anywhere — that’s what’s compelling and will be compelling.

Now, agents have non-determinism and autonomy, which brings security issues — permissions, authorization. How do you ensure that anything published and distributed by Docker meets customers’ expectations and doesn’t create problems?

Great question. There are two things: how do I ensure anything built by Docker meets customer expectations, and how do agents change that? One is about containers and images; the other is about agents.

From the image piece, Docker has been a core element of the supply chain for a decade. The big investment we’ve been making is Docker Hardened Images. We run on a pretty hardened supply chain with build processes that guarantee provenance of any source, making sure what we produce is secure, tested, and high quality. We give customers something they can build on that improves their own security posture.

We’ve advanced from Docker Official Images to Hardened Images because vulnerability counts in open source are going up. Developer laptops are the new gold mine. Recent vulnerabilities are from breaching into that space. It’s insufficient to just have official images. We need to help everyone have a more secure foundation. DHI is a free, open-source primitive with enterprise features. Every developer has a baseline that gives them a minimized, secure foundation with least privilege, guaranteed with no malware, CVEs mitigated faster, etc.

The agent side is a little different because agents require mutability. Now you get into containers versus sandboxing — a very complicated topic. Sandboxing in computer science is running untrusted code. Docker containers already give you OS virtualization through kernel-level isolation. But the entire container ecosystem was built around immutability. Kubernetes watches if containers drift and kills them. Scanners watch for any change to the file system. The entire ecosystem is built around the container being an immutable unit.

Agents, by definition, want to mutate their environment. When you run Claude or Copilot, it says, “I’m going to run package install, download from the internet,” and immediately breaks that contract of containers. Agents are a vicious adversary — if you set them loose, they’ll find a way to break your machine. You need the hardest isolation boundary you can possibly have.

For us, sandboxing is that core primitive. We took core bits from the Docker engine and rewrote them with micro VM technology so they run with the properties you expect on Mac, Windows, Linux, cloud, etc. They fundamentally look, feel, and act like Docker, but give hardware isolation. Now nothing is shared. You can put the agent in a box, let it do whatever it wants, but you have god-like control. You can see file systems, network calls, keep secrets out, proxy. You can control anything about the outside and let the agent run, but kill switch or stop it any way you want.

We have deep conviction this is the right bottom layer. Sandboxing is the base turtle — the unit of execution you can run anything in. You can put any agent in there. Then you can build the rest of your trust profiles on that. When you go with our agents and actually run productively, you need to run it in yellow mode where the agent can run with all permission checking off and just go wild. The problem is when it does whatever it wants, it can do very bad things. The real benefit is you can put any agent in yellow mode, let them cook, but have safety, control, and isolation boundaries around it.

Now, let’s say I’m building an agentic AI application and need an MCP server from the Docker catalog. Should I install from Docker, and how does that protect me?

Well, first, there’s a disclaimer somewhere my legal team would tell me to say. But seriously, saying you can protect against anything is a journey, not a destination. What I can tell you is why do this today, what can we do today, what are we working on, and what should you still be worried about? That’s how I’d answer, because we’re all figuring out agents and AI together week by week.

If you go to the official MCP registry, there’s tens of thousands of things. It’s pretty wild west. You have the same names with the same package nine times over. If you want Notion, there are seven or nine different entries. The real question is: what do you want and do you trust it? For one, I don’t know what I’m getting, so I tend not to run those things.

With Docker’s catalog, we have 300-some servers. We’ve either handmade them ourselves or vetted them. We’ve had Docker Hub for years with 20 billion pulls. There’s a program called Docker Verified Publishers where we vet our partners. Getting a check mark requires legal agreement, audits, and trust assurance. The Docker MCP catalog has hundreds of things, not tens of thousands, but those are safe. We protect you from namespacing collisions and bad software getting in.

But the interesting thing in the agentic world is you have this non-deterministic actor doing whatever it wants. We’re working on prompt injection protection, data loss protection, and more advanced servers with platform building blocks. If you’re building custom MCPs yourself, you have the same problems. The same functionality should apply — filtering and protections both built into the image and inline.

Will it ever be 100% secure? That’s like asking if we’ll reach AGI. I don’t think you can say 100% of anything and be credible, frankly.

Yeah, I wouldn’t expect a different answer. So Docker vets them, and the publisher is official. But with non-determinism of agents, there’s so much that can be done. So we use it that way.

Yeah, and then you get back to needing trusted MCPs. You need to be able to trust every facet of your stack. Sandboxing is the most root primitive you need — it puts everything in a bounded box you can control and reason about. You need tools to be secure and trusted. You need any code it generates to be secure and trusted. But even then, it’s necessary but insufficient. You really want something semantically aware where you can say, “I’m going to use this agent, but it can only access this part of my inbox, not these Slack channels.” The world is rich with lots of data, protocols, and things. You need root-level execution control. It’s expanding circles of complexity. You need to reason about and secure everything so you can trust your agent didn’t delete your database or send your Bitcoin wallet to scammers.

More of a philosophical question: is containerization as we know it the right primitive for agents, or will it evolve? What might Docker look like 3 years from now?

Well, I think this is a very fair question. Containers have been around for a decade now — they’re ubiquitous. That’s amazing, but it makes it very hard to change. It’s an entire ecosystem. 92% of the world runs containers in production. Containers are absolutely essential for the agentic world. But agents require a different first principle because the ecosystem matured around immutable containers, while agents want to mutate everything all the time.

It’s not that you can’t use containers for agents — of course you can. We run every MCP server wrapped in containers. When I build agents and want to share them, I put them in a container. Containerization gives you a useful way to build, package, and share software. That was true yesterday; it will be true tomorrow, whether for humans or agents. The runtime execution needs something that supports mutability, and that’s where sandboxing comes in.

You can run containers inside a sandbox. Containerization is always going to be the unit by which you ship software because it gives you a nice packaging artifact, a way to describe deterministically what gets put together. It’s repeatable and reproducible. The running of most software will continue to be containers. 92% will asymptotically approach 100%. But for agents — specifically the thinking part — that has to have a primitive rethought with sandboxing.

We’re trying to make sure sandboxing feels consistent with the Docker toolchain and ethos, but it has a different primitive because agents fundamentally require a new primitive. We’ll see how we evolve the specifications, the company, and naming.

So you have 20 million plus developers, and Docker is ubiquitous. But things are changing so quickly at every layer — models, infrastructure, tools, frameworks. How do you build a business on that kind of mindshare and trust while things are changing so rapidly?

Well, I have this running joke with Tushar about Saturday morning development. You wake up Saturday morning, read Twitter, Hacker News, Reddit, and suddenly you’re like, “Okay, I guess we’re throwing out everything I’ve done and starting over next week.” So it’s very chaotic trying to keep up.

But the thing is, many of these problems are actually evergreen problems. Tying back to what we talked about at the beginning — when building AWS, I read papers from the mainframes. As ancient as that sounds, the durable problems are here. And I can say this pretty confidently because with Docker, I get to talk to customers, upcoming developers, communities, and anybody. Since Docker is so ubiquitous, I can say the vast majority of the world right now is in a spectrum where most people are okay with accepting AI, Copilot, Cursor, tab complete, cloud code. Everyone’s adopted that.

But the real unlock everyone’s trying to get to now is true agentic development — where the agent can run autonomously and make decisions on your behalf. That’s where the market is. Most people are there. And what it takes to move from point A to point B — looking at early adopters — fundamentally comes down to trust and safety. Being able to have confidence that what you’re running is what you think it is and that it can’t do bad things. That’s unlock number one. Two is getting more of the population able to write code, which is 10 to 100X now because anybody can pick up an agent and learn to write code and solve problems.

The evergreen problems here are: you need hard isolation boundaries, observability, controls, and the ability to package and describe what you want. I take a lot of solace that these core primitives of Docker are still going to be the right things tomorrow. When building a business around it, it’s very hard to predict what features any given developer will use. But most people understand we’re going to live in a world where agents can’t be 100% trusted. You have to build a new layer of defense.

What we’re trying to do is balance Saturday-driven development with the reality that there are evergreen problems here. We need to help people scale, help more of the workforce adopt agents, help companies and CISOs sleep well at night knowing it’s not going to run amok. All of that feels like the right path forward.