101 - ML-Ops: from notebook to service

ML-Ops is the set of practices that lets a model work for someone else, at another time and without you next to it. This module exists for one reason only: it is the last one of the course, and it turns a notebook that works on your laptop into something that runs for other people, repeatably, with a version you can point to and a way back when something goes wrong. Up to here you have learned to build a model; here you learn to hand it over.

The words of the module

The module uses a few words that come back in every lesson, and it is worth fixing them right away. A notebook (usually Jupyter) is a document that mixes code, results and notes, run in cells in an order you choose: it is the scratch pad, very handy for exploring and dangerous as a source of truth. The model is the result of training, an object that returns a prediction when you feed it new data; production is the real environment where it serves users or other programs, the opposite of "on my laptop", like the difference between rehearsing a speech in front of the mirror and giving it in front of an audience.

A dependency is something your code needs and you did not write (a library, a file, a service); the environment is the set of Python, libraries and versions the code runs in, and the same recipe in a different kitchen can come out different. Reproducibility is the ability to get the same result by redoing the same work, you a month from now or someone else today. An artifact is a file produced by a process and worth keeping: the saved model, a chart, a configuration. An experiment is a group of trials on the same problem and a run is a single trial, that is, one training with certain parameters; we pick them up again in MLflow: tracking experiments. Finally a service is a program that stays on waiting for requests, in contrast with the script, which starts, works and ends.

The thread of the module at a glance: the notebook becomes code under version control, with a declared environment, protected by tests, packaged in a container, published in a registry, tracked with MLflow, released and observed.

The last box, monitoring, is yellow because it is the only one this module names without building it: the site's series on drift and retraining goes deeper into it. What follows in the module covers the other seven boxes.

The analogy: the home kitchen and the restaurant

A recipe that works in a home kitchen is not yet a restaurant dish. At home it works because you are there: you know the oven that heats badly, you know the salt is the coarse kind, you taste while you cook. In the restaurant the same recipe has to come out identical on Saturday night, with another cook, with ingredients that arrived from another supplier. You need a written recipe with quantities, known suppliers, procedures, checks before serving and someone who notices if the dish comes back.

The notebook is the home kitchen. It works because your memory is inside it: the order in which you ran the cells, the file you downloaded last week, the version of scikit-learn that happened to be installed. ML-Ops is the move to the professional kitchen.

What is implicit in a notebook

When a model "works on my laptop", five things are in fact working that nobody has written down anywhere. Each one is a hidden dependency, and each one has a lesson in this module that makes it explicit.

The last row is often missing from "best practices" lists, but it is the most specific to machine learning: in traditional software, if you have the code you have the program. In an ML project, behaviour depends on code, data, parameters and seed together. If you do not record them, the result is not reproducible even by you. The series MLOps: from experiment to production develops the theoretical framework (maturity levels, differences from DevOps); here we work on the practical side, command by command.

The map of the module

The lessons follow the order in which the problem shows up in the life of a project:

1. Git and GitHub: so you do not lose your work and can review it.
2. Open source and licences: because almost everything you will use is not yours.
3. UV: to have the same environment on every machine.
4. pytest: to know whether a change broke something, model included.
5. Environment variables: to keep configuration out of the code.
6. Containers, Dockerfile and registry: to package the service.
7. MLflow: to track experiments and manage model versions.
8. Deploy, and when an automatic pipeline is worth it or not.

The order matters here: there is no point writing tests if the code is not under version control, and no point packaging in a container an environment you do not know how to rebuild. Each step presupposes the ones above it, and that is why it pays to follow the lessons in the given sequence, starting from Git, instead of jumping straight to Docker or MLflow.

A real case: this site's ML service

This site has a machine learning microservice, ml-service/: a FastAPI application that does semantic search over the articles with RAG (embeddings stored in Redis with an HNSW index) and uses a local language model served by Ollama on the same server. It is not a notebook: it is the final form we are aiming at. Three details, all verifiable in the repository, show what "in production" means. It runs as a systemd unit (infra/systemd/portfolio-ml.service) with Restart=always, so if it stops it restarts by itself. Its configuration comes from an environment file loaded with EnvironmentFile=, not from the code, as we will see in the lesson on environment variables. And the unit declares TimeoutStartSec=300 with a comment explaining why: on a cold start the service downloads an embedding model of about 90 MB, loads it and indexes the articles, and can exceed systemd's default 90 seconds. It is a limit no notebook would ever have shown you.

The deploy is manual by choice: one command line that updates the code, installs the package, syncs the files and restarts the service. The project explicitly forbids automatic deploys. In the last lesson, manual deploy versus CI/CD, we will see why it is a sensible decision and what it teaches about the way of reasoning in ML-Ops.

How much ML-Ops you really need

The temptation, when studying this subject, is to build the complete platform right away: orchestrator, registry, pipelines, monitoring. It is an expensive mistake. Every practice has a maintenance cost, and the right criterion is: introduce the practice when the pain it solves is real. With two exceptions that always apply, because they cost little and protect a lot: Git from day one, even alone and even for a one-evening experiment, and a reproducible environment (UV), because rebuilding it after the fact is much more tiring than declaring it right away.

Everything else follows the needs. A model that only you use for one analysis does not need a container. A service that others call every day does.

The clean-kitchen test

Before declaring a project "ready", do this mental exercise (and, as soon as you can, do it for real on another machine or in an empty folder). Answer five questions:

1. Can someone who clones the repository regenerate the environment with a single command?
2. Is there a command that runs the tests and says yes or no?
3. Is there a single absolute path, a single key, a single password in the repository? (There must not be.)
4. If the service starts with a wrong configuration, does it stop at once with a clear message, or does it start and fail silently?
5. Can you say which version of the model is answering right now, and how to go back to the previous one?

Every "no" corresponds to a lesson in this module: the first to UV, the second to pytest, the third and the fourth to environment variables, the fifth to the model registry. You do not need all of them right away, but you must know which one you are postponing and why: a conscious "no" is a choice, a "no" discovered in production is an incident.

A trap to know from now on

The most insidious mistake in ML-Ops is not the one that makes noise, it is the one that looks like a success. In this site's repository, on 9 October 2026, a failed build passed for a successful one: the command was piped into tail, the script saw the exit status of tail (always zero) and not that of the build, so it published the previous version. The site answered 200 and everything seemed fine, but the new code was not there. We will take it apart in the lesson on manual deploy. For now keep its moral: "the service responds" does not prove that the release happened.

What you need to follow the module: tools and versions

The examples were run on Linux with these tools. Versions matter: the numbers and messages you will see in the lessons come from these, and newer or older tools may behave slightly differently. To find out what you have installed:

git --version       # used: git version 2.53.0
uv --version        # used: uv 0.12.21
docker --version    # used: Docker version 29.3.0

Each command prints the version and nothing else: it is the quickest way to notice a difference. On the test machine (measured on 2026-11-12) the three lines match the ones in the comments. The rest (Python 3.12, pytest, scikit-learn, MLflow, FastAPI) is installed by UV inside an isolated environment for each project, so you do not need to install it by hand on the system.

Of this table I verified on this machine only the first three version rows (Git, UV, Docker); the others come from the lessons in which each tool is installed, and there it is stated how they were obtained.

The through-line project: a churn classifier

So as not to change the example in every lesson, the module builds a single small project: churn. "Churn" is the term companies use for customers who stop using a service; a classifier is a model that assigns a case to a category, here "leaves" or "stays". The data is synthetic, generated by code (so it is always reproducible and contains nobody's data): each customer has the months of tenure (tenure), the number of complaints (complaints) and the churn label (1 = left). The model is a logistic regression, the simplest of the classifiers, chosen on purpose: the module is not about the model, it is about everything around it. At the end the project has this shape:

churn/
  pyproject.toml        # declared dependencies        (UV)
  uv.lock               # resolved dependencies        (UV)
  .python-version       # Python version               (UV)
  .gitignore            # what not to version          (Git)
  src/churn/
    data.py             # generates the synthetic data
    model.py            # trains and evaluates
  tests/
    test_data.py        # tests on the data            (pytest)
    test_model.py       # invariants and thresholds    (pytest)
  track.py              # MLflow tracking
  register.py           # registry and aliases
  app.py + Dockerfile   # service and container

Each file corresponds to a lesson: you can use the tree as an index. If you follow the module by writing the code, at the end you will have a project that is not a notebook. The tree is the final structure the lessons build, not a ready-made project to download: each lesson writes its own files, and says so.

The notebook that breaks: hidden state in an example

Before the five implicit dependencies, a concrete example of the most insidious one, the code. In a notebook the cells share state in memory: a variable defined in one cell stays available in all the others, even if the cell that defined it was edited or deleted. Let us simulate two cells with two blocks of code:

# Cell A (later deleted)
threshold = 0.7

# Cell B (still there)
def is_at_risk(probability):
    return probability >= threshold

print(is_at_risk(0.75))
# Output: True

I ran the block twice with Python 3.14.4: the output is True both times. The function in cell B depends on threshold, defined in cell A. As long as the kernel (the Python process that keeps the notebook's state in memory) stays on, everything works. If you delete cell A and restart the kernel, cell B fails with NameError: name 'threshold' is not defined: I reproduced exactly that message by running the function alone without the line threshold = 0.7. Worse: if you change the value to 0.9 in A and re-run only B, the result depends on which cells you ran last, not on what is written. The notebook has a hidden state: what the program does depends on the history of your actions, not only on the code. A .py file run from top to bottom does not have this problem, and that is why the module moves the code into ordinary files, under Git, with tests.

The immediate remedy, even before everything else, is the gesture "restart the kernel and run everything from the start": if the notebook gives the same results, it is at least consistent with itself. If not, you have found a hidden dependency before someone else did. The cost is that a notebook that is slow to re-run discourages the gesture, and it is a good reason to move the expensive parts into functions inside .py files.

The why before the how: five problems, five tools

Every tool in this module exists because someone, before you, got hurt in a precise way. If you learn the command first and the reason afterwards, the command feels like bureaucracy; if you learn the pain first, the command becomes obvious. Let us see the five pains one by one, with their "root cause" and what it costs to solve them.

1. "I no longer know what I changed" (the code)

Imagine working on a model for three weeks. One day the metric drops and you want to understand when. If the work is in a notebook, you have two paths: remember (you cannot) or compare files with names like final_model.ipynb, final_model2.ipynb, final_model_REALLY.ipynb. It is the symptom of the lack of a history: each version is a separate object, with no declared relation to the others. Git solves the problem at the root, not with a naming convention: a single history, in which every change has an author, a date, a message and a parent. The cost is learning a dozen commands and the discipline of making small commits. The gain is that the question "when did it change?" has a mechanical answer (git bisect, in the lesson on Git) instead of an investigation.

2. "It works on my machine" (the libraries)

Your code uses libraries, and libraries change. A function is renamed, a default value changes, an algorithm gets a different implementation and the numbers slip by a decimal digit. When you ask a colleague to run your project and they have a different version, the result can be an error (the lucky case, because it is visible) or a slightly different number (the unlucky case, because it is silent). The root cause is that the code declares what it uses only halfway: it says "scikit-learn", not "that precise version of scikit-learn, with those precise versions of NumPy and SciPy underneath". The UV lockfile (lesson on UV) writes down the second thing. The cost is one more file to commit and an update routine; the gain is that "it works on my machine" becomes "it works the same everywhere, until I change the lock".

3. "It broke and I did not notice" (the tests)

Changing working code is risky because the consequences are far from the cause: you change a data cleaning function and, three modules further on, the model gets two points worse without any error. The root cause is the absence of an automatic check that says, at every change, "the behaviour I expected is still there". Tests are that check. In machine learning they matter more than in ordinary software, because the typical failure is not an error but a silent degradation: the program runs, produces plausible numbers, and they are worse than before. A test with a threshold (testing a model) turns silent degradation into a visible red. The cost is time to write them and to keep them fast; the gain is the ability to change code without fear.

4. "It works on my laptop and not on the server" (the system)

Even with the same libraries, the program runs inside an operating system, with its system libraries, its environment variables, its permissions. A missing native library, a different compiler version, a path that exists only on your machine: these are all causes of failure that the code does not declare. The root cause is that the execution environment is not described. Two answers, of different weight: external configuration (environment variables), which separates what changes from what does not, and the container (container concepts), which packages the base system too. The container has a real cost (images to build and store, one more level in debugging) and this project, which has a single machine, chose not to pay it for its services: the systemd unit seen above runs directly on the server. This too is an ML-Ops decision.

5. "Which model is the right one?" (the memory of results)

You train twenty variants of a model changing parameters and data. Three days later you are asked "the one you delivered on Tuesday, what parameters did it have?". If you did not write it down, you redo the experiment hoping to get the same result; if you did not fix the seed, you do not get it. The root cause is that an experiment has no identity: it is an event that happens and vanishes. MLflow (tracking and registry) gives it an identity: a run with an identifier, its parameters, its metrics, its model. And the registry gives an identity to the model in service too: a numbered version, not "the file that is in that folder". The cost is getting used to going through tracking instead of launching scripts by hand; the gain is reproducibility and the ability to go back.

Why ML-Ops is not just DevOps applied to models

Whoever comes from software already knows DevOps: the set of practices that joins development and operations (versioning, automatic tests, repeatable release, monitoring). Everything we have seen so far derives from it. But three things make machine learning different, and explain why there is a separate name.

First, behaviour depends on data, not only on code. In an ordinary program, if the code is identical, the behaviour is identical. In a model, the same code trained on different data produces a different program. So versioning only the code is not enough: you also need the identity of the data and the parameters. That is why tracking records the dataset and seed along with the parameters, and why the registry ties each version to the run that produced it.

Second, development is experimental. Software is written to reach a known behaviour; a model is sought by trying many variants and measuring. Daily work generates a huge number of intermediate results, almost all to be discarded, and the hard part is not losing the good one. A tracking system is the "notebook" that this way of working requires and that ordinary software does not require with the same urgency.

Third, failure is often silent, and arrives with time. An ordinary service breaks with an error. A model can keep answering perfectly while the world changes around it: today's customers do not resemble the ones it was trained on, and quality drops without a single error line. This phenomenon (drift) requires monitoring of results and data, not only of the server's resources. We do not cover it in this module, but the series drift detection and automated retraining tackles it, and the series MLOps: from experiment to production describes the maturity levels with which a team moves from manual work to the automated cycle. The paper by Sculley and colleagues, "Hidden Technical Debt in Machine Learning Systems", is the classic reading on why ML systems accumulate technical debt differently from ordinary software.

The complete life cycle of a model

Putting the pieces together, the life of a model in production is not a line (train, release, end) but a cycle:

1. Train: from versioned code and data, in a reproducible environment, producing a model and its metrics.
2. Verify: tests on the data, on the behaviour and on the minimum quality. If a test fails, the model does not go forward.
3. Register: the model enters the catalogue with a version and its metrics.
4. Package and release: the service, with its configuration and its environment, arrives where it has to run.
5. Observe: the service and the model are monitored. When something degrades, you go back to point 1 with new data.

The module covers points 1 to 4. Point 5 closes the circle and is what distinguishes an experiment from a product: a product has someone who notices when it no longer works. Point 2 acts as a gate: no model gets registered unless it first passed the tests, and it is the only step in which a human decision can be replaced by a written rule.

The real project: what is behind "an ML service"

To see how concrete this list is, it is enough to count what keeps this site up. In the repository's infra/systemd/ folder there are 50 service units and 64 timers (counted on 2026-11-12 with ls infra/systemd/*.service | wc -l and the same for *.timer). They are not all machine learning services: many are periodic data updates, the weekly retraining of the sports model, the SEO checks. The point is not the number, it is the idea: a system that runs by itself, at night and without you, is made of many small programs with a schedule, a configuration, a way to retry if they fail and a record of what they did. That is "production". A notebook has none of these properties, and it is the distance this module teaches you to close.

Three deploy traps documented in the repository

The project's deploy runbook (docs/01_foundation/04_deploy/11_deploy_manuale.md, section 0) collects the mistakes that have already done damage, written down so they do not repeat. You will meet three of them in the lesson on manual deploy. The first is the piped command that hides a failure (build | tail): a failed build passed for a successful one and the deploy published the previous version, with the site answering as usual. The second is the interruption of a process that also hits its "wrapper": a pkill -f with a string also contained in the command that wraps the process interrupted a chain of steps that was meant to carry on even after an expected failure. The third is a check that queries production from the production machine: during compilation, the preparatory phase sends a HEAD request to each of the roughly 5,139 URLs of the site maps, against the site itself, for 9 minutes in the best case (runbook figures, §0.3, not re-measured here). The runbook prescribes skipping it in the deploy with SKIP_SITEMAP_AUDIT=1.

There is a fourth detail, in the same runbook: the same piped-command error had already masked, a few hours earlier, a failed git pull because some files generated by the preparatory phase had dirtied the working folder. The frontend's prebuild runs several generate:* scripts (series, blog manifest, metadata) that write files inside the repository: if those files turn out modified, a later code update may refuse to proceed. It is a perfect example of an implicit dependency: no file says "the build leaves the folder dirty", but if you do not know it, the next release fails in a way that seems unrelated.

Mindset mistakes worth dropping right away

The first is believing that the model is the product: the model is a part, the product is the model plus the service that exposes it, the data that feeds it, the checks that protect it and the people who maintain it. The second is "first I make it work, then I fix it": with experimental code it is a good approach, provided that "later" really arrives, because the risk is that the exploration notebook becomes, by inertia, the system in production. The third is thinking that automation is always better: automating a process you do not understand multiplies its defects, and the right order is understand, document, then possibly automate (manual deploy versus CI/CD).

The fourth is "tests are for programmers, I do data science": a model's tests are exactly the place where data science experience (which threshold makes sense? which behaviours are absurd?) becomes a check nobody can forget. The fifth is "we need a tool for everything": what is needed is the problem the tool solves, and if the problem is not there yet, the tool is a cost with no benefit.

What you can do now

You can say why a notebook that works is not yet a service, and you can name the five implicit things that hold it up: code, libraries, configuration, system, memory of results. You can use the clean-kitchen test on a project of your own and find out which lesson you need first. You can also resist the temptation of the complete platform, keeping only Git and a reproducible environment as fixed points from day one. We start from the base of everything: Git, the time machine of your project.

The same series also includes Running a Model Locally: Quantisation, Memory and Latency, Measured (previous article).