Production models age even when the code does not change

Rabanser et al. widely evaluated methods that convert high-dimensional data into low-dimensional representations and compare reference and production distributions using two-sample tests. Under the conditions examined, representations from pre-trained classifiers were effective, and domain classifiers could also characterize features of changed examples.[1] However, this test answers 'is the data different?' not 'has judgment worsened?'.

Moreno-Torres et al. distinguish covariate shift (where input distribution changes but the input-to-result relationship holds), prior probability shift (where result ratios change), and concept shift (where the input-to-result relationship itself changes).[4] Monitoring only inputs cannot capture degradation where relationships change while input distributions remain similar. Conversely, if performance and loss are maintained despite large input changes, it is not a target for emergency response.

Therefore, read the initial screen on two axes: 'presence of input difference' and 'increase in business loss.' If both increase, identify the affected interval; if only input difference increases, suspect harm-free seasonality; if only loss increases, investigate concept changes or label/decision shifts.[1] [4] Treat distribution distance as an investigative hypothesis to narrow causes, not as a notification condition.

• No Input Difference / Increased Loss: Prioritize investigation of concept changes, label definitions, thresholds, and on-site behavior changes.
• Input Difference Present / No Loss: Record seasonality or customer composition changes and suppress notifications.
• Input Difference Present / Increased Loss: Localize the start point, affected population, and upstream changes.
• No Input Difference / No Loss: Update comparison periods and sample sizes as the baseline state.

Koh et al.'s WILDS collected 10 real-world datasets with varying hospitals, camera locations, regions, and time points, reporting that standard learning methods degrade out-of-distribution performance across all datasets compared to in-distribution performance.[2] However, which populations, periods, and acquisition paths changed varies by dataset. A single value of 'overall drift' obscures both intervals where impact concentrates and intervals where no change occurred.

Also isolate failures prior to the model. Breck et al.'s production readiness rubric presents 28 tests spanning models, data, infrastructure, and monitoring.[3] Upstream schema changes, feature generation mismatches, training-serving skew, silent missingness, and time/unit errors have recovery paths distinct from statistical drift. Upon detecting input differences, first verify that the observation system itself is not broken.

The monitoring contract connects pipelines, inputs, predictions, decisions, outcomes, and human behavior in order. Place metrics, baselines, update cycles, owners, and investigation deadlines on each layer, and record the time lag for changes in previous layers to reach the next. Specify important intervals for acquisition source, region, device, customer segment, period, and business path in addition to overall averages, and list sample sizes and uncertainty alongside.

• PIPELINE: Verify count, freshness, type, range, missingness, duplication, and feature generation consistency.
• INPUT: Track univariate and multivariate changes by important population and acquisition path.
• PREDICTION: Track changes in score distribution, uncertainty, rejection rate, and calibration.
• DECISION: Track threshold passage, allocation, overrides, manual backlog, and exception handling.
• OUTCOME: Track performance of matured outcomes, population-specific errors, and business loss.
• HUMAN: Track override reasons, complaints, avoidance behavior, and changes in model dependency.

In many businesses, ground truth labels do not arrive simultaneously with predictions. Csaba et al. demonstrated in an online continual learning framework separating current unlabeled data from delayed past labels that performance degrades as delay increases, and that simply increasing computation does not resolve this condition.[6] While this is not research determining monitoring methods for all businesses, it undermines the premise that 'seeing only recent data reveals latest performance.'

NIST AI 800-4 also notes that production ground truth data is delayed, high-cost, or sometimes unobtainable, and that baselines, thresholds, and long-term tracking are unresolved issues.[8] Mixing delayed results like repayments, cancellations, readmissions, or inaccuracies into weekly aggregations at prediction time makes unsettled cases appear normal. Separate event occurrence time from result confirmation time and compare matured cohorts.

Place two clocks on the dashboard. The immediate clock tracks inputs, predictions, overrides, processing time, and provisional proxy metrics; the confirmed clock tracks label arrival rates, matured performance, calibration, and business loss. Cross-check proxy metrics against confirmed values when the corresponding outcomes become available, making the discrepancy between proxies and true results a monitoring target. Do not hide intervals without labels as missing; display arrival rates and reasons.

• Immediate Clock: Use inputs, predictions, actions, processing time, and proxy outcomes for early fault detection.
• Confirmed Clock: Measure performance, calibration, population-specific errors, and loss using only matured cohorts.
• Arrival Management: Record label arrival rates, confirmation deadlines, unsettled reasons, and subsequent updates.
• Proxy Validation: Regularly cross-check differences between proxy and confirmed values to determine conditions for stopping proxy use.

A fixed one-time two-sample test does not assume production data that arrives sequentially, returns, or changes gradually. Jang et al. proposed sequential covariate shift detection using online classifiers, showing detection in multiple image and code datasets under conditions maintaining a specified false positive rate boundary.[5] However, the target is covariate shift where conditional result relationships hold despite input distribution changes, not direct measurement of performance degradation or loss.

Do not fix the comparison baseline to one value. Separate baselines for training, recent stable periods, same week last year, and pre-regulation change, noting known seasonality. Display effect size, sample size, confidence intervals, and direction of important intervals in addition to p-values, and manage increased false positives when iteratively testing many features. Extend periods for low traffic and include in reviews that small differences are more likely to be significant in high traffic.

Notifications fire only when differences persist for a certain period, spread to important intervals, and consume the tolerance for performance, judgment, or loss. Send detectors to investigation queues and do not page until business impact is confirmed. Define release conditions simultaneously to suppress alerts that continue after normalization and duplicate notifications from multiple metrics for the same cause.

• BASELINE: Fix whether to compare against training, recent stable period, seasonal baseline, or pre-change.
• EVIDENCE: Display difference magnitude, sample size, uncertainty, and important intervals simultaneously.
• PERSISTENCE: Distinguish transient noise using duration and multiple windows, not single values.
• IMPACT: Judge statistical difference and performance/judgment/loss thresholds separately.

When predictions are used for decisions, those judgments change future data. Perdomo et al. formalized performative prediction, where predictions influence future result distributions, showing that retraining does not necessarily return to the original problem.[7] In credit, recommendation, underwriting, and allocation, models change target behavior or observed populations, potentially making excluded people's results unobservable even if apparent performance improves.

NIST organizes production monitoring into six categories: function, operations, human factors, security, compliance, and large-scale impact, pointing out that human-AI feedback loops are understudied.[8] Even if model metrics are kept good, the system degrades if overrides increase, queues stagnate, burdens concentrate on specific populations, complaints rise, or off-purpose use increases.

Therefore, the outcome layer includes not just discrimination performance but calibration, threshold-specific costs, population-specific errors, and business loss. The decision layer includes override rates and reasons, processing time, target exclusion, allocation changes, and manual confirmation backlogs. Do not attribute changes solely to the model version; overlay policy, regulation, price, staffing, and competitor actions into change history, and choose responses after eliminating alternative explanations.

Google SRE's SLO alert design distinguishes notification severity by the speed of consuming the error budget and multiple time windows, rather than a single threshold.[9] While this is not research validating ML monitoring effectiveness, it can be adapted into practical design that separates paging and investigation tickets based on how quickly tolerance loss is incurred, rather than determining urgency solely by statistical distance magnitude.

Define severity from your company's loss tolerance. Immediate degradation or stoppage for safety, legal, irreversible loss, or pipeline breaks; target restriction and manual confirmation for rapid consumption of performance budgets; and time-bound investigation for small but persistent changes. Lower statistically large but harmless changes to logs, and elevate changes with large impact but small detection distance from the result side.

Include model and version, detection start, baseline, target interval, sample size, difference magnitude, estimated performance/judgment/loss, related changes, owner, confirmation deadline, and degradation method in the notification body. Notifications that cannot determine next actions without the owner checking another screen are incomplete. Place release conditions, duplicate suppression, and re-notification intervals in the runbook, and conduct post-reviews of false alarms and misses.

• P1 / STOP: Immediately degrade or stop for safety, legal, irreversible loss, or pipeline breaks.
• P2 / CONTAIN: Increase target restrictions and manual confirmation for rapid consumption of performance/loss tolerance.
• P3 / INVESTIGATE: Narrow causes as time-bound investigations for small but persistent changes.
• LOG / LEARN: Record harmless differences and reflect them in seasonality and false alarm suppression.

Retraining without knowing the cause fixes dirty data or incorrect labels into the new model. First verify observation system faults, change history, affected intervals, and label maturity, then choose reversible measures in order: upstream correction, threshold change, target restriction, manual confirmation, rollback to previous version, and feature disablement. As Breck et al.'s rubric indicates, the ability to rollback, reproduce training and serving, and track inputs and outputs are prerequisites for monitoring.[3]

Divide the runbook into stages: detection, confirmation, containment, cause diagnosis, recovery, and prevention of recurrence. Define start conditions, responsible person, required data, maximum wait time, and next decision for each stage. Do not end recovery just because metrics return to normal; confirm that important interval outcomes have matured, proxy and confirmed values align again, and recurrence does not occur upon reapplying the same change.

Also phase the introduction. In the initial phase, define monitoring targets, baseline periods, important populations, owners, and loss tolerance. Next, measure false alarms, label arrival, and investigation time in a silent mode with no notifications. Finally, enable severity-based notifications and degradation drills. Maturity is measured not by alert count but by time from detection to confirmation/containment/recovery, missed loss, workload from false alarms, and recurrence rate of the same fault.

• OBSERVE: Verify observation system, baselines, label maturity, and change history are correct.
• CONTAIN: Choose reversible measures from target restriction, manual confirmation, threshold change, rollback, or stoppage.
• DIAGNOSE: Verify start point, affected interval, cause candidates, alternative explanations, and retraining data.
• RECOVER: Confirm outcome maturity and recurrence tests, then update baselines and runbooks.