Docker Compose for AI

Canonical version: Docker Compose for AI.

Docker Compose now treats AI models as first-class application components via a top-level models: element, so you can wire models, agents, and tools into one declarative file and bring them up with a single docker compose up. The required field is model: (the OCI artifact pulled and run by Docker Model Runner); optional fields are context_size: and runtime_flags:. Services reference models and Compose injects connection env vars (short syntax auto-generates LLM_URL and LLM_MODEL; long syntax uses endpoint_var / model_var).

models:
  llm:
    model: ai/smollm2
    context_size: 8192
services:
  app:
    image: my-app
    models:
      - llm

It is the assembly layer of Docker AI: it ties Docker Model Runner and the Docker MCP Gateway into agentic apps, and the spec is portable across Compose-compliant platforms.

References

• https://docs.docker.com/reference/compose-file/models/
• https://docs.docker.com/ai/compose/models-and-compose/
• https://docs.docker.com/guides/agentic-ai/

Related

• Docker AI
• Docker Compose
• Docker Model Runner
• Docker MCP Gateway
• Docker Agent

About Sébastien

I'm Sébastien Dubois, and I'm on a mission to help knowledge workers escape information overload. After 20+ years in IT and seeing too many brilliant minds drowning in digital chaos, I've decided to help people build systems that actually work. Through the Knowii Community, my courses, products & services and my Website/Newsletter, I share practical and battle-tested systems.

I write about Knowledge Work, Personal Knowledge Management, Note-taking, Lifelong Learning, Personal Organization, Productivity, and more. I also craft lovely digital products and tools.

If you want to follow my work, then become a member and join our community.

Ready to get to the next level?

If you're tired of information overwhelm and ready to build a reliable knowledge system:

• 📚 KM for Beginners — 10+ hours of structured video lessons
• 🚀 Obsidian Starter Kit — Ready-made vault with 40+ templates
• 💼 Knowledge Worker Kit — Complete guides + lifetime community
• 🦉 1-on-1 Coaching — Personalized guidance
• 🎯 Join Knowii — Community + ALL courses & tools