From notebook to production
A model that works in a notebook is where the real work begins. See why machine-learning systems are harder to run than ordinary software, walk through the full lifecycle, explore the hidden technical debt around the model, and assess your team’s MLOps maturity.
In this lecture you will
- Explain why many machine-learning projects stall between prototype and production
- Describe how ML systems differ from traditional software — code, data and model all change
- Map the stages of the ML lifecycle and the feedback loop that closes it
- Assess a team’s MLOps maturity and choose the next practice to adopt
A data scientist shares a notebook: the churn model scores 0.94 on the test set, and the business wants it live next month. Months later it is still not in production — or it is, and nobody has noticed that its predictions went stale. This story is so common that industry surveys keep finding that a large share of machine-learning projects never deliver value in production. MLOps — machine-learning operations — is the set of engineering practices that closes that gap.
Why ML systems are different
Ordinary software changes when engineers change the code. A machine-learning system has three things that change, and only one of them is under your direct control:
| Traditional software | Machine-learning system | |
|---|---|---|
| Behavior defined by | Code written by engineers | Code plus data — the model is learned |
| Changes when | Someone commits new code | Code changes, data changes, or the world changes |
| Testing | Unit and integration tests | Also data validation and model-quality checks |
| Typical silent failure | Rare — bugs usually crash | Common — a stale model keeps returning confident answers |
That last row is the one that hurts. A broken web service throws errors; a degraded model keeps answering, just worse.
The lifecycle — a loop, not a line
Scroll through the full lifecycle of a model, from the notebook to a system that improves itself.
- 1It works in a notebook
A churn model reaches 0.94 accuracy on a laptop. That proves the idea is possible — not that it will work for customers next month.
- 2Frame the problem
Agree what the model is for, which business metric it should move, and its constraints: how fast, how often, how explainable, at what cost.
- 3Data, most of the effort
Collect data reliably, validate it automatically and version every snapshot. Most production failures start here, not in the model.
- 4Train and evaluate as a pipeline
Training becomes a repeatable pipeline. Every run is tracked, and a quality gate stops models that are worse than the current one.
- 5Deploy safely
The approved model is packaged, served behind an API or a batch job, and rolled out gradually — first to a small share of traffic.
- 6Monitor and close the loop
Monitoring watches inputs, predictions and outcomes. When drift appears, fresh data flows back into retraining. Automating and watching this loop is MLOps.
- MLOps
- Practices and tools for deploying, operating and improving machine-learning systems reliably — DevOps extended to data and models.
- Training pipeline
- An automated sequence of steps that turns raw data into a validated, versioned model.
- Model serving
- Making a model’s predictions available to other systems, through an API, a batch job or a stream.
- Drift
- A change in production data or in the relationship between inputs and outcomes that makes a model less accurate.
- Continuous training
- Retraining models automatically on fresh data, on a schedule or when monitoring detects a problem.
The hidden technical debt
In 2015 a group of Google engineers published Hidden Technical Debt in Machine Learning Systems. Its most famous figure shows the model code as a tiny box surrounded by everything else a real system needs. Explore it below.
Switch to the notebook view to see what a prototype contains — then back to production to see what is missing. Box sizes are illustrative.
Each surrounding box is a place where things go wrong. Most of this course is about them: data pipelines and validation (lecture 2), tracking and reproducibility (lecture 3), serving (lecture 4) and monitoring (lecture 5).
How mature is your team?
A widely used maturity model describes three levels. At level 0, everything is manual: notebooks, hand-offs and one-off deployments. At level 1, training is an automated pipeline that can retrain on new data. At level 2, the pipelines themselves are tested and deployed through CI/CD, and monitoring triggers retraining. Tick what your team already does.
Try the example teams, then tick your own team’s practices. The next step is the first missing practice at the next level.
The next lecture starts where most production failures start: the data and feature pipelines that feed the model.
Key takeaways
- A working notebook is the start, not the finish; production also needs data pipelines, serving, monitoring and retraining.
- An ML system can break because its code changes, its data changes, or the world changes — so all three must be versioned, tested and watched.
- The lifecycle is a loop — monitoring feeds fresh data back into retraining — and MLOps is the practice of automating and observing that loop.
- Build maturity in steps — tracked experiments and automated pipelines first, then CI/CD and monitoring-driven retraining.
Check your understanding
4 questions · answers explainedQ1A fraud model’s accuracy drops over six months although nobody touched its code. What is the most likely reason?
Q2According to the well-known “hidden technical debt” paper, how much of a real ML system is the model code?
Q3What is the defining step from MLOps level 0 to level 1?
Q4Where does the machine-learning lifecycle loop close?