Why Agent Runtimes Need Their Own Infrastructure

Most AI infrastructure today is built around the idea that an LLM generates a response and something else acts on it. That works fine for chatbots and content generation. It falls apart the moment you need an agent to actually do things — call APIs, write files, spin up processes, move money.

The gap isn’t in the model’s reasoning ability. It’s in execution.

The execution problem

When an agent interacts with the real world, you immediately run into questions that have nothing to do with language modeling:

• Isolation — What if the agent’s code has a bug? What if it loops? You need sandboxing, not just prompt engineering.
• Ephemerality — Each execution is a fresh environment. No leaking state between runs. No accumulated garbage.
• Observability — You need to know exactly what the agent did, what tools it called, what failed and why.
• Resource control — CPU, memory, network, disk. Agents can’t just run wild.

These are infrastructure problems, not AI problems. And they’re the reason I’ve been spending most of my time building on Firecracker microVMs.

Why microVMs

Containers are too fat for this. Full VMs are too slow to spin up. MicroVMs hit the sweet spot — sub-second boot times, strong isolation (actual hardware virtualization, not just namespace separation), and you can tightly control the resource envelope.

Each agent execution gets its own VM. It boots, runs, dies. Nothing persists unless you explicitly persist it. That’s the model.

The orchestration layer

But isolated execution is only half the problem. The other half is orchestration — coordinating multiple agents, managing tool calls, handling retries, maintaining durable state across long-running workflows.

This is where things like LangGraph and Temporal come in. You define workflows as graphs. Nodes are agent steps. Edges are transitions. The framework handles persistence, retries, and state management. You focus on the logic.

What this means for building products

If you’re building AI products that do more than generate text, you need to think about execution infrastructure from day one. Not as an afterthought. Not as “we’ll add sandboxing later.”

The agents that matter are the ones that take action. And taking action safely requires infrastructure designed for that purpose.

This is the stuff I’m building at Meshi. If you’re wrestling with similar problems, I’d love to compare notes.