110 - ML-Ops: multi-stage Dockerfile for an ML service, non-root user and healthcheck

A Dockerfile for an ML service: multi-stage, non-root user, healthcheck

A Dockerfile is the recipe for building an image: a sequence of instructions, each of which produces a layer. Here you containerize a churn classifier served with FastAPI, using UV to install the dependencies from the lockfile (see uv and package management). Every line of the Dockerfile has a reason, and we go through them one by one: the split into two stages, the ordering that exploits the cache, the unprivileged user and the health check.

A word first on what was verified. I really ran the Python service (app.py), with FastAPI's test client and a model trained for the occasion; I ran the UV commands (uv lock, uv sync --locked --no-install-project) in a scratch project. I did not run the Dockerfile or the docker commands: in the text they are declared schema to adapt or deduced output, not executed, and wherever a figure depends on the image you will build (sizes, times) I report none and tell you how to measure it.

The words of the Dockerfile

To read the rest you need a few terms, in addition to those of the article on containers (image, container, layer). The base image is the starting point given by FROM, like starting from ready-made dough instead of from flour. The build context is the folder that docker build sends to the engine: only the files in it can be copied into the image. A multi-stage build is a Dockerfile with several FROM lines, each of which opens a stage: the final image is the last one, and from the others you take only the files you need, like the prep kitchen and the dining room, where the big pots stay in the kitchen. The build cache is the reuse of a layer that has already been built, if the instruction and its input files have not changed.

A UID is the number that identifies a user on Linux: 0 is root, while here we use 10001, an ordinary user. The healthcheck is a command that Docker runs at intervals to find out whether the service is healthy. Finally, FastAPI is the library you use to write the web service's functions, uvicorn is the server that exposes them on the network, and ASGI is the standard they use to talk to each other.

The handoff between the two stages:

The Docker documentation describes multi-stage builds as the tool for optimizing a Dockerfile while keeping it readable. The diagram above is the typical case for Python: the build stage holds everything needed to produce the environment, and the final stage holds only the result.

The analogy: the prep kitchen and the plated dish

In a restaurant, the prep kitchen has knives, cutting boards, big pots and sacks of vegetables. Only the plated dish reaches the table. If you carried the big pots into the dining room too, the customer would find a room full of equipment. A multi-stage build does exactly this: the build stage holds the equipment (UV, compilation tools); the final image takes only the result (the libraries, already installed).

The why: from the install script to the image

Before containers, making a service "installable" meant writing an install script ("install Python, install these libraries, copy the files, configure the service") to run on the server. The script has two structural flaws. First, it starts from an unknown state: if the server already has some version of something, or a leftover file from a failed attempt, the result depends on that. Second, it is not repeatable over time: run today and a year from now, it downloads different versions. A Dockerfile fixes both, because the build always starts from a known base, and it is executed once, producing an artifact (the image) that is then moved anywhere without being rebuilt. The Dockerfile is the recipe; the image is the dish, already cooked and identical everywhere (the article on registries, tags and digests covers this: an image is identified by a digest).

The service to package

An app.py file with FastAPI: it loads the model at startup, exposes /health and /predict, and validates its input. To run it I used the data generator and the train function from the article on testing models with pytest: 800 synthetic customers with seed 0, two variables (tenure, months of tenure, and complaints) and a scikit-learn 1.8.0 logistic regression, saved with joblib.dump(model, "model.joblib"). The AUC on the test set is 0.7483, the same value as in that article. The data is invented and the model has no real predictive value: it only serves to make the service run.

import os
from contextlib import asynccontextmanager

import joblib
import pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel, Field

MODEL_PATH = os.environ.get("MODEL_PATH", "model.joblib")
state: dict = {}

@asynccontextmanager
async def lifespan(app: FastAPI):
    state["model"] = joblib.load(MODEL_PATH)      # fails right away if the file is missing
    yield

app = FastAPI(lifespan=lifespan)

class Customer(BaseModel):
    tenure: int = Field(ge=0, le=600)
    complaints: int = Field(ge=0, le=100)

@app.get("/health")
def health():
    return {"status": "ok", "model_loaded": "model" in state}

@app.post("/predict")
def predict(c: Customer):
    p = state["model"].predict_proba(pd.DataFrame([c.model_dump()]))[0, 1]
    return {"churn_probability": round(float(p), 4)}

Two choices worth noting. The model path comes from an environment variable (as in the article on environment variables and secrets), with a default. And the validation of the limits (ge, le) lives in the input schema: an absurd value is rejected with a 422 error before it reaches the model. The loading happens in the lifespan, that is, at startup: if the file is not there, the service does not start, and this is intended (see below, on the healthcheck).

I queried the service without opening any ports, using FastAPI's TestClient, which calls the application directly in memory:

from fastapi.testclient import TestClient
from app import app

with TestClient(app) as client:
    print(client.get("/health").json())
    print(client.post("/predict", json={"tenure": 3, "complaints": 4}).json())
    r = client.post("/predict", json={"tenure": -1, "complaints": 4})
    print(r.status_code, r.json()["detail"][0]["type"], "|", r.json()["detail"][0]["msg"])

# {'status': 'ok', 'model_loaded': True}
# {'churn_probability': 0.9524}
# 422 greater_than_equal | Input should be greater than or equal to 0

The with TestClient(app) also runs the lifespan, so the model is loaded before the first request. The customer with 3 months of tenure and 4 complaints has a probability of 0.9524 on my synthetic model: the number depends on the invented data and has no other meaning. The third request, with tenure equal to -1, never reaches the model: the schema rejects it with code 422. Run twice with Python 3.14.4, FastAPI 0.136.3 and pydantic 2.13.4, the outcome is identical. With MODEL_PATH=/nope.joblib the startup raises FileNotFoundError, which is the loud failure we want.

The Dockerfile, line by line

The Dockerfile below is a schema to adapt, not executed (it requires Docker): every line is motivated in the sections that follow, and I verified the UV commands it contains separately, as shown further on.

FROM python:3.12-slim AS build
COPY --from=ghcr.io/astral-sh/uv:0.12 /uv /usr/local/bin/uv
WORKDIR /app
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy
COPY pyproject.toml uv.lock ./
RUN uv sync --locked --no-install-project

FROM python:3.12-slim
RUN useradd --create-home --uid 10001 app
WORKDIR /app
COPY --from=build /app/.venv /app/.venv
COPY app.py model.joblib ./
ENV PATH="/app/.venv/bin:$PATH" MODEL_PATH=/app/model.joblib
USER app
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request as u; u.urlopen('http://127.0.0.1:8000/health')"
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

The tags python:3.12-slim (Docker Hub) and ghcr.io/astral-sh/uv:0.12 (ghcr.io) exist: I verified them by querying the respective registries on 2026-11-12. The UV guide for Docker confirms the {major}.{minor} tag format, which points to the latest patch of that series.

Stage 1: building the environment

The first line starts from python:3.12-slim, the same Python version as development, in the "slim" variant that avoids unnecessary system packages. The second takes the UV binary from an official image, with the version pinned to 0.12 and not to latest: why, the article on registries and tags explains. The two ENV variables are the ones the UV guide describes: UV_COMPILE_BYTECODE=1 precompiles the Python files into .pyc, while UV_LINK_MODE=copy avoids the warnings about file linking when the cache and the destination sit on different filesystems.

The line that exploits the cache is COPY pyproject.toml uv.lock ./ before the code: as long as these two files do not change, the next layer is reused. Then RUN uv sync --locked --no-install-project installs exactly the lockfile. The --locked flag makes the command fail if the lock is not aligned with pyproject.toml, and --no-install-project installs the dependencies but not the project itself, which changes more often (this too is described in the UV guide). I verified the two behaviors in a scratch project with UV 0.12.21:

# scratch pyproject.toml: dependencies = ["idna>=3"]
uv lock
uv sync --locked --no-install-project
#  + idna==3.20                      (the dependencies are installed, the project is not)

# I add "certifi" to the dependencies WITHOUT regenerating the lock, then:
uv sync --locked --no-install-project
# error: The lockfile at `uv.lock` needs to be updated, but `--locked` was provided.
# exit=1

In the first case UV resolves and installs only the dependency (idna), not the churn-api package itself. In the second, with pyproject.toml modified but the lock unchanged, the command ends with code 1 and the notice that the lock must be updated: in a Dockerfile this means the build stops instead of installing unrecorded versions. The idna version is the one available on 2026-11-12 and changes over time.

Stage 2: the final image

The final stage starts over with FROM python:3.12-slim: UV and the build cache do not end up in it. RUN useradd --create-home --uid 10001 app and USER app run the service without privileges: if a defect allowed an attacker to execute commands, they would do so as an ordinary user inside the container, not as root. The documentation of USER specifies that the instruction sets the default user and group for the rest of the stage. COPY --from=build /app/.venv /app/.venv brings over only the ready-made environment; COPY app.py model.joblib ./ adds code and model last, because they change most often. ENV PATH=... MODEL_PATH=... sets the default environment, which can be overridden with docker run -e.

The HEALTHCHECK asks /health whether the service responds. From the documentation: the default interval is 30 seconds, the default timeout is 30 seconds (here we lowered it to 3) and the default start period is 0. The command exits with 0 if all is well and with 1 if not: urlopen raises an exception if the connection fails or if the server answers with an HTTP error, so Python exits with a nonzero code and Docker marks the container as unhealthy. Finally, CMD [...] is the start command in list form, with no shell in between: the why is in "flaw 6" further down.

The .dockerignore file

When you launch the build, Docker sends the current folder to the engine (the build context). Without a filter you also send the virtual environment, the experiment databases and the .env files with the secrets. The .dockerignore prevents it:

.venv
__pycache__
mlruns
mlflow.db
.env*
.git

It is not cosmetic: a careless COPY . . without this file carries the secrets into the image layers, where whoever pulls the image can read them (see the article on secrets). The UV guide says the same thing about the .venv folder alone: it must be excluded because it depends on the local platform and in the image it has to be created from scratch.

Build, run, verify

With the service and the Dockerfile in the same folder, the sequence is the following. It is deduced output, not executed: the answers from /health and /predict are those the service gave in the TestClient test above, because the code is the same.

docker build -t churn-api:0.1.0 .
docker run -d --rm --name churn -p 18000:8000 churn-api:0.1.0
curl -s localhost:18000/health
curl -s -X POST localhost:18000/predict -H 'content-type: application/json' \
     -d '{"tenure":3,"complaints":4}'
# {"status":"ok","model_loaded":true}
# {"churn_probability":0.9524}      (the number depends on your model)
docker run --rm churn-api:0.1.0 id -u
# 10001

The build creates the image with the tag churn-api:0.1.0; the run starts it in the background with the container's port 8000 published on port 18000 of your computer; the two curl calls query the service from outside. The last command, id -u, returns 10001: the process is not root. To find out how much the image weighs and which layer weighs the most, use docker images churn-api and docker history churn-api:0.1.0: with scikit-learn, pandas and SciPy inside, the virtual environment layer is by far the largest, and there is little to save without changing dependencies.

The trap: model and service must speak the same version

A .joblib file is the serialization of the model's Python object (pickle): it reloads correctly only with compatible versions of the library that produced it. The scikit-learn page on persistence says two things that matter here: when you load an estimator saved with a different version, an InconsistentVersionWarning is raised; and pickle-based formats must be loaded only from sources you trust completely, because loading can execute arbitrary code. The second point is the reason the model enters the image from a path in your own build, not from just any address.

I reproduced the warning by faking a save made by a different version (changing the version that scikit-learn writes into the file, in a scratch listing):

import warnings
import joblib
import sklearn
import sklearn.base
from sklearn.exceptions import InconsistentVersionWarning

m = joblib.load("model.joblib")
real = sklearn.base.__version__
sklearn.base.__version__ = "1.7.0"            # pretend the model was saved by another version
joblib.dump(m, "old.joblib")
sklearn.base.__version__ = real

with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    joblib.load("old.joblib")
for x in w:
    if issubclass(x.category, InconsistentVersionWarning):
        print(x.category.__name__, "| saved with", x.message.original_sklearn_version, "| in use", x.message.current_sklearn_version)

# InconsistentVersionWarning | saved with 1.7.0 | in use 1.8.0

The listing is artificial: it deliberately writes "1.7.0" into the file to simulate a model trained elsewhere. The result shows that loading does not fail, but flags the inconsistency with a warning that the application can intercept. The real risk is that the warning ends up in the logs and nobody reads it. It happens when training and serving have two separate lockfiles, resolved on different days: the day a new version comes out, they diverge without anyone noticing.

The defenses, in order of robustness: train and serve in the same project with the same uv.lock; record the library versions together with the model (MLflow does this, see MLflow: tracking experiments); add a test that loads the saved model and checks a known prediction (see testing models with pytest). The most valuable integration test in ML-Ops is exactly this one: the artifact you release loads and answers. In addition, you can turn the warning into an error with warnings.simplefilter("error", InconsistentVersionWarning) around the loading, so that an inconsistency stops the service at startup instead of passing in silence.

Model inside the image or outside?

Here the model is copied inside the image (in my test the file weighs 1,135 bytes: a tiny example). It is simple and makes the image self-contained and reproducible: one image version = one model version. But if the model weighs gigabytes or changes often, rebuilding and republishing the whole image at every retraining is a cost. The alternative is to keep the model outside (a volume, an object store, the model registry covered in the article on the model registry) and pass its address through MODEL_PATH or a URI. The trade-off is between self-containment and traceability on one side, speed of updates on the other.

The Dockerfile instructions, in a table

Two distinctions not to confuse. RUN versus CMD: RUN acts once, at build time (it installs the libraries); CMD acts at every container start (it switches the service on). EXPOSE versus -p: according to the Dockerfile reference, EXPOSE works as documentation between whoever builds the image and whoever runs it and does not publish the port; to reach the service from outside you still need docker run -p 18000:8000.

Building the Dockerfile step by step: each line repairs a problem

The Dockerfile can look like a block of instructions to memorize. It works better to understand it as the outcome of six successive corrections to a naive first attempt. We start from the naive attempt (schema to adapt, not executed):

FROM python:3.12
COPY . .
RUN pip install -r requirements.txt
CMD uvicorn app:app --host 0.0.0.0 --port 8000

It works. And it has at least six flaws: the base is huge, every change reinstalls the libraries, the versions are not pinned, the build tools end up in production, the service runs as root and shutdown is abrupt. We repair them one at a time.

Flaw 1: the base image is huge

python:3.12 contains a complete operating system with compilers and tools. The -slim variant removes what is not needed. Fix: FROM python:3.12-slim. By how much the size changes you see by comparing the two tags with docker images after pulling them; it depends on the date of the tag.

Flaw 2: every code change reinstalls the libraries

COPY . . comes before the installation, so Docker's cache is invalidated by a change to any file and the libraries are downloaded again. Fix: copy the dependency files first, install, and only then copy the code. What rarely changes goes on top, what changes often goes below, as the documentation on the build cache describes. To measure it, rebuild the image after adding a line to app.py with docker build --progress=plain .: the installation steps must show up as CACHED.

Flaw 3: the versions are not pinned

pip install -r requirements.txt resolves the versions today. Fix: uv sync --locked from the lockfile, which, as you saw above, fails if the lock does not match pyproject.toml. With this, the image built today and the one built six months from now install the same versions.

Flaw 4: the build tools end up in the production image

An image that contains UV, the package cache and unneeded project files is bigger and has more attack surface. Fix: the multi-stage build, which builds in a first stage and brings into the final image only the ready environment. Beware of a promise you often hear, "multi-stage = smaller image": it is not a law. It depends on what you leave out; for example UV_COMPILE_BYTECODE=1 adds the .pyc files to the environment and therefore enlarges it, in exchange for a startup that does not compile the modules the first time. Compare the variants with and without that variable on your service with docker images before deciding. The sure benefit of multi-stage is the cleanliness of the final image, not its weight.

Flaw 5: the service runs as root

By default the process in the container is root. Fix: useradd and USER app. It is also a trap to know about: the app user cannot write to /app, which belongs to root, while it can write to its own home folder /home/app (created by --create-home). If the application needs to write (a model cache, temporary files), you must explicitly prepare a folder writable by that user (for example with RUN mkdir /cache && chown app /cache before USER app) or mount a volume. The "Permission denied" error at the startup of a service that "worked as root" is one of the most common after this fix. You can try it with docker run --rm churn-api:0.1.0 sh -c 'touch /app/x; touch /home/app/ok && echo writable': the first touch must give Permission denied.

Flaw 6: shutdown is slow and abrupt

The CMD uvicorn app:app ... form (without square brackets) is called the shell form: Docker runs it as /bin/sh -c "uvicorn ...". The Dockerfile reference warns that with the shell form the executable does not receive Unix signals: so it does not receive the SIGTERM that docker stop sends. If the process does not respond within the grace period (10 seconds by default), Docker kills it with SIGKILL. With the exec form (the list) uvicorn is process 1, receives SIGTERM, can gracefully close the requests in flight and exits immediately. For an ML service this means faster restarts and releases, and in-flight requests that are not lost. Fix: always the list form for CMD and ENTRYPOINT. I report no docker stop times: you measure them yourself in exercise 3.

Tracing where an image comes from: labels and build arguments

An image in production should say which commit it was born from. With a build argument and a standard label:

ARG GIT_SHA=unknown
LABEL org.opencontainers.image.revision=$GIT_SHA

# docker build --build-arg GIT_SHA=$(git rev-parse --short HEAD) -t churn-api:0.1.1 .
# docker image inspect churn-api:0.1.1 --format '{{index .Config.Labels "org.opencontainers.image.revision"}}'
# deduced output, not executed: the short commit identifier passed to the build

ARG declares a value that is passed to the build with --build-arg; LABEL writes the metadata into the image. The name org.opencontainers.image.revision is a standard key of the OCI annotations specification, defined as the identifier of the source control revision of the packaged software. The inspect command returns the commit, so from a running image you know which code it is, without guessing from the tag. Careful: ARG values appear in the image history; do not pass secrets through them.

The health check, designed well

The HEALTHCHECK asks /health at regular intervals. Two details to design. What it verifies: if /health always answers 200 even with the model not loaded, it hides the failure; ours reports model_loaded and startup fails if the file is missing, but in a service that loads the model lazily the check has to verify the loading. When it starts: a heavy model takes time to load, and a check that begins immediately flags as "unhealthy" a service that is merely starting. The --start-period option grants an initial period (0 seconds by default); during that period the checks run at the --start-interval frequency (5 seconds by default), as the Dockerfile reference says.

This site's ML service follows the same principle in a different way: its /health, called before startup is complete, answers with status "starting" instead of pretending to be ready (ml-service/app/main.py), and the systemd unit grants TimeoutStartSec=300 to start (infra/systemd/portfolio-ml.service). One detail to remember: a response with status "starting" but HTTP code 200 still passes the check of our HEALTHCHECK, which only looks at whether the request succeeds; if you want "not ready" to count as a failure, the check must read the response body or the endpoint must answer 503.

Why the application writes its logs to standard output

In a container you do not access an internal log file: you look at docker logs. That is why uvicorn writes to stdout/stderr by default, and your code should do the same (the logging library with a stream handler, not a file). On the VPS, where this site's services run under systemd, the same stream is collected into the journal (see the article on deployment): the principle is identical, what changes is who collects it.

The path to production: a checklist for an image

• Reproducible: base pinned (tag or digest), dependencies from the lock, no latest.
• As small as needed: slim base, multi-stage, no build tools in the final stage; measure, do not assume.
• Unprivileged: non-root user, writable folders declared, read-only filesystem if possible.
• Clean: .dockerignore, no secrets in the layers or in the ARG values, no development files.
• Shuts down well: CMD in exec form, the service handles SIGTERM.
• Declares what it is: label with the commit, model version visible in /health.
• Checks itself: a healthcheck that says something true, logs on stdout.
• Has been tried: started, queried, stopped, with tests that run on the real artifact and not only on the code.

The eight items have no order of importance, but three are foolproof and two require choices of yours. Reproducibility, non-root user and .dockerignore cost a few lines and apply to any service. Size and model inside or outside the image depend instead on the service and have to be measured. The last item, the test on the real artifact, is the one many skip and the one that catches more errors than the others.

When the Dockerfile is not enough: scanning and updating

Building the image does not finish the job: inside there are dozens of system and Python packages, each with a history of vulnerabilities. Image scanning tools exist that compare them with public databases, often integrated into the registry or the CI. The general rule is the same as in the article on licenses and open source software: know what is contained in what you release, rebuild the image periodically to pick up fixes, and rerun the tests on the new image before releasing it.

Try it yourself: four guided exercises

Exercise 1: build and test the image

With this article's service (app.py, pyproject.toml, uv.lock, model.joblib) build the image, start it and query it: docker build -t churn-api:0.1.0 ., docker run -d --rm --name churn -p 18000:8000 churn-api:0.1.0, then curl -s localhost:18000/health and a request to /predict. You should see {"status":"ok","model_loaded":true} and a probability. Read the healthcheck state with docker inspect churn --format '{{.State.Health.Status}}': first starting, then healthy. What you learn: that "built" is not "working" and that the health check has a startup time.

Exercise 2: measure the variants

Build the single-stage variant (COPY . ., UV in the final image) next to the multi-stage one, and possibly a third without UV_COMPILE_BYTECODE=1. Compare with docker images churn-api --format '{{.Tag}} {{.Size}}' and check the user with docker run --rm IMAGE id -u. You should see different sizes (note them down: they are your measurements) and 0 against 10001 as the user. What you learn: not to trust "multi-stage = smaller" without measuring, and to recognize the trade-offs (bytecode: more weight for a faster startup).

Exercise 3: the form of CMD

Start two containers, one with CMD ["uvicorn", ...] and one with the shell form CMD uvicorn .... For each, after a few seconds, measure the time of docker stop (you can use date +%s.%N before and after, and bc for the difference). With docker exec NAME sh -c 'tr "\0" " " < /proc/1/cmdline' look at which process is process 1. You should see uvicorn as process 1 in the exec form and /bin/sh -c ... in the shell form, with longer stop times in the second (up to Docker's grace period). What you learn: why the list form is not a matter of style but of signal handling.

Exercise 4: permission denied

Inside the image, as a non-root user, try to write to /app and to /home/app: docker run --rm churn-api:0.1.0 sh -c 'touch /app/x; touch /home/app/ok && echo writable'. You should see Permission denied for the first and "writable" for the second. Then modify the Dockerfile to create a /cache folder writable by the app user and verify that writing to it succeeds. What you learn: the most common error after dropping root privileges, and how to solve it without giving them back.

Cleanup: docker stop churn and docker image prune; test images are removed with docker rmi.

What you can do now

You can write a two-stage Dockerfile for an ML service: dependencies from the lockfile with uv sync --locked in the build stage, a final image with only the environment, the code and the model, a non-root user, a .dockerignore and a healthcheck that says something true. You know why the lock is copied before the code, why CMD wants the list form, and how to notice that the model and the library serving it do not share the same version. You also know which figures not to trust in an article: sizes and times are measured on your own image. The next step is where to publish it: registries, tags and digests.