The gap between a Jupyter notebook and a monitored production service is where most ML projects die. Here's the shortest path across it.
Track everything
MLflow or Weights & Biases — pick one, then log every training run: params, metrics, artifacts, data version. Non-negotiable.
Feature store as contract
Feast, Tecton, or a homegrown Delta table with strict SLAs. Training and serving must read the same features from the same code.
CI/CD for models
steps:
- test-data-schema
- train
- evaluate-vs-champion
- shadow-deploy
- promote-if-metrics-holdObserve, don't hope
Feature drift, prediction drift, latency, cost per inference. Alert on the leading indicators, not the outages.