Machine learning code usually starts in a local environment full of accidental conveniences: cached packages, mutable notebooks, and implicit system libraries. The migration to multi-node execution breaks those assumptions immediately. What worked on one laptop often fails on the first clean worker.
Docker solves that problem by making the container image the unit of execution. The important detail is not merely using Docker, but using it with enough restraint that build times, image sizes, and dependency drift stay under control.
Image source: Docker documentation.
What a Minimal ML Dockerfile Should Do
A useful ML image should do four things:
- install only runtime dependencies
- keep rebuild time low by separating slow-changing layers
- expose a single clear entrypoint
- behave the same on one node or many nodes
At scale, image efficiency matters because total pull cost grows with both image size and node count:
$$ T_{pull,total} \approx N_{nodes} \times T_{pull,image} $$
An extra gigabyte is tolerable on a laptop and expensive on a 40-node job.
A Clean Base Pattern
For Python-based inference or batch scoring, the simplest good pattern is a slim base image, deterministic dependency installation, and a narrow working directory.
FROM python:3.11-slim
WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/
COPY pyproject.toml .
CMD ["python", "-m", "src.main"]
This is not glamorous, but it is operationally sound. The dependency layer is cached independently of your application code, which keeps incremental rebuilds fast.
What to Avoid
Most oversized ML images share the same avoidable mistakes:
- copying the full project before installing dependencies
- bundling raw datasets into the image
- installing compilers and leaving them in the final layer
- using notebook servers as production entrypoints
A container should package code and runtime, not become a frozen home directory.
Multi-Node Readiness
The jump from local execution to multi-node orchestration changes the failure mode. Your code is no longer starting once. It may start dozens of times on clean hosts with no shared local state.
That means the image should assume:
- model artifacts arrive from object storage or mounted volumes
- logs go to stdout and stderr
- configuration arrives through environment variables
- the filesystem is disposable
One useful mental model is:
$$ Artifact = Image + Config + External\ Data $$
If the image contains everything, it becomes too large. If it contains too little, startup becomes fragile. The balance is to package the application and fetch mutable artifacts externally.
A Better Layout for ML Teams
Keep the repo structure boring and explicit:
project/
|-- Dockerfile
|-- requirements.txt
|-- pyproject.toml
`-- src/
`-- main.py
This layout keeps the build context small and makes it easier to reason about what actually enters the image.
Final Check Before Shipping
Before promoting an image into a cluster, confirm:
- it builds from a clean machine
- it starts without local developer files
- it downloads or mounts model assets explicitly
- it emits logs without interactive assumptions
- its size is small enough for repeated node pulls
That is the difference between “Dockerized” and genuinely portable.