How to deploy a machine learning model: from notebook to API

A model in a notebook helps nobody. How to save a model, wrap it in a FastAPI service, validate inputs, containerise it, deploy it and monitor it in production, step by step.

To deploy a machine learning model, save the trained model and its preprocessing steps, wrap them in a small web API such as FastAPI that validates inputs and returns predictions, package everything in a Docker container, deploy it to a server or cloud service, and monitor accuracy, latency and data drift once real users depend on it.

Most learning projects stop at a notebook with a good score. In real work, a model only creates value once other systems can call it reliably. Deployment is also one of the clearest ways to stand out in interviews, because many candidates never do it.

What does the deployment pipeline look like?

How do you save a model properly?

Save the model and every preprocessing step together, for example as a single scikit-learn pipeline, so training and serving transform data identically. Record the library versions and the data version used. Many production bugs come from preprocessing that differs slightly between the notebook and the service.

What does a minimal API look like?

FastAPI is a popular Python framework for this, with automatic input validation and documentation:

from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI()
model = joblib.load("delivery_time_model.joblib")

class Order(BaseModel):
    distance_km: float
    hour: int
    restaurant_load: int

@app.post("/predict")
def predict(order: Order):
    minutes = model.predict([[order.distance_km, order.hour, order.restaurant_load]])[0]
    return {"estimated_minutes": round(float(minutes), 1)}

Run it locally, send test requests, and read the automatically generated documentation page. Our guide to REST vs GraphQL explains the API conventions.

Why containerise the model?

A Docker container packages the code, model file, Python version and libraries so the service runs the same everywhere. It removes the “works on my machine” problem and makes deployment to cloud services straightforward. Our guide to Docker and Kubernetes explained simply covers the basics.

Where should you deploy it?

What should you monitor after launch?

What about security?

Validate every input, require authentication for anything not public, rate-limit endpoints, keep secrets out of code, and log access. For models handling personal data, India’s DPDP Act applies; our DPDP guide for developers explains the obligations.

What does a deployment checklist look like?

How do you deploy large language models differently?

LLMs usually need GPUs, careful memory management and specialised inference servers that handle batching and the KV cache. Many teams start by calling a hosted model API and only self-host when cost, privacy or customisation requires it. Our explainer on how LLM inference works covers what changes when the model is a language model.

How do you test a deployed model?

Write automated tests for the API: valid inputs return sensible predictions, invalid inputs return clear errors, and edge cases such as missing values are handled. Keep a small set of known examples with expected outputs and run them after every deployment, so a broken model file or preprocessing change is caught before users notice.

How does this grow into MLOps?

As models multiply, teams add experiment tracking, model registries, automated testing and deployment pipelines, and scheduled retraining. That discipline is MLOps, explained in Program Zero’s guide to what MLOps is. Program Zero covers model serving with FastAPI, Docker, Kubernetes basics, CI/CD, experiment tracking and monitoring in its cloud, MLOps and deployment phase.

Program Zero runs live online, so you can learn this from anywhere in India: metros such as Kolkata, Bengaluru, Mumbai, Delhi NCR and Hyderabad, growing cities such as Pune, Ahmedabad, Jaipur and Lucknow, and smaller towns such as Asansol, Durgapur, Siliguri and Guwahati. Every learner joins the same evening classes, at the same fee.

Frequently asked questions

Related reading