Docker Compose for AI

Canonical version: Docker Compose for AI.

Docker Compose now treats AI models as first-class application components via a top-level models: element, so you can wire models, agents, and tools into one declarative file and bring them up with a single docker compose up. The required field is model: (the OCI artifact pulled and run by Docker Model Runner); optional fields are context_size: and runtime_flags:. Services reference models and Compose injects connection env vars (short syntax auto-generates LLM_URL and LLM_MODEL; long syntax uses endpoint_var / model_var).

models:
  llm:
    model: ai/smollm2
    context_size: 8192
services:
  app:
    image: my-app
    models:
      - llm

It is the assembly layer of Docker AI: it ties Docker Model Runner and the Docker MCP Gateway into agentic apps, and the spec is portable across Compose-compliant platforms.

References

• https://docs.docker.com/reference/compose-file/models/
• https://docs.docker.com/ai/compose/models-and-compose/
• https://docs.docker.com/guides/agentic-ai/

Related

• Docker AI
• Docker Compose
• Docker Model Runner
• Docker MCP Gateway
• Docker Agent

About Sébastien

Ready to get to the next level?