Model updates require defining what must be re-proven before the change

AI system behavior depends not only on the base model but also on system prompts, search data, embeddings, tools, thresholds, post-processing, permissions, and human verification procedures. Even if the vendor maintains the same product name, changes to any of these components can affect accuracy, refusal tendencies, explainability, information leakage, latency, and cost.

The change ticket must record the modified components, reason for change, target use cases, anticipated benefits, potentially affected users, known constraints, differences from the previous version, and the final point for rollback. A single line stating ‘Update model to the latest version’ fails to specify what to revert to or which tests to re-run in the event of an incident.

FDA’s 2025 Final Guidance structures the Predetermined Change Control Plan (PCCP) for AI-enabled medical devices around three core elements: Description of Modifications, Modification Protocol, and Impact Assessment. This approach links the change content, development/validation/implementation methods, and impact on benefits/risks in advance, allowing iterative improvement within the approved scope. [1]

Translating this structure to general business operations, define the allowable scope of model versions or data additions, the representative/boundary/failure cases for re-testing, baseline values for comparison, approvers, rollout rate, and monitoring period before the change. Purpose changes or removal of safety features that fall outside the planned scope should not be processed as minor changes but returned to a review process akin to new introductions. [1]

The planned change table should include not only modifiable components but also boundaries that must not be crossed, such as purpose, permissions, and human verification. Even within these boundaries, proceed to phased rollout only when all approval conditions are met, including fixed regression sets, recent failure cases, and impact on key users. Do not transpose the medical device framework as a legal obligation, but use it as a design principle for pre-linking changes, validation, and impact. [1]

NIST AI RMF 1.0 proposes continuously cycling through four functions: GOVERN, MAP, MEASURE, and MANAGE, to track performance and new risks during operation. In update evaluations, place differences in important scenarios, minority groups, refusal/false answers, security, privacy, human verification, and cost on the same comparative table, not just average quality. [2]

Divide test sets into a fixed regression set and a learning set added from recent failures and complaints. A fixed set alone cannot capture new usage environments, while changing the set each time prevents comparison with the previous version. Since model outputs are probabilistic, verify distributions through iterative testing rather than single examples, and determine if deterioration is within acceptable limits by use case. [2]

Performance differences in the new version mix the effects of model changes with changes in input distribution, search documents, and user composition during the same period. Separate comparative testing with identical inputs for old and new versions from time-series monitoring in actual operations to isolate the source of differences. [2] If deterioration occurs only in actual operations despite no difference in controlled comparisons, return to re-evaluating the usage environment rather than the model approval.

FDA’s predetermined change management proposes linking change content, validation methods, and impact assessment in advance. [1] Using this as inspiration, we propose an interaction table to identify approval targets when multiple components are changed simultaneously. Do not transpose this as a legal obligation for general business, but separate baseline configurations, single changes, and combined changes, and fix the comparison order.

In the interaction table, pass configurations with only the base model changed, only search changed, and both changed through the same inputs to check if result directions align. If a combination fails even though single components pass, do not attribute the failure to the quality of either component alone, but register it as a combination-specific approval target. Also perform restoration tests that disentangle the combination.

Do not test all combinations; prioritize important use cases, changes that increase permissions, and combinations that have failed in the past. For incomparable combinations, do not fill gaps with averages, but decide to halt simultaneous rollout, split changes, or limit target use cases. The approval ticket must distinguish between tested and untested combinations.

Change approval is not a permanent attribute but a judgment based on specific configurations, test samples, usage permissions, and monitorability. Map each approval reason to its necessary premises, evidence confirming those premises, confirmation date, expiration conditions, and the owner for re-review. When premises can no longer be confirmed, do not leave only the pass stamp. Also indicate dependencies between premises.

Do not limit expiration conditions to changes in approved components. Also treat as a state where part of the approval grounds has disappeared if the evaluation sample cannot be reproduced, specifications of dependent external functions cannot be confirmed, required logs cannot be obtained, or contingency measures cannot be demonstrated. Do not attribute causes to operational distribution changes; re-verify the validity of the evidence.

In re-review, update only the evidence related to expired premises, and do not pretend to re-validate unrelated pass results. Conversely, if the impact scope cannot be identified, suspend the entire approval unit. Specify in advance which premises must be restored for re-approval to prevent automatic reversion to old approvals based on operator judgment. Also save the version of the reversion judgment.

• BASELINE: Identify baseline configuration and single/combined changes
• INTERACTION: Record failures and untested areas arising only from combinations
• ASSUMPTION: Link necessary premises and confirmation evidence for each approval reason
• EXPIRY: Determine expiration conditions, impact scope, and re-review owner

The EU AI Act addresses new conformity assessment after substantial modification of covered high-risk AI in Article 43 and requires post-market monitoring throughout the system lifetime in Article 72. It is important to distinguish between continuous learning changes predicted and evaluated in the initial assessment and unplanned changes affecting conformity or intended purpose. [3]

In practice, prepare phased rollout to a small user group, parallel comparison of old and new versions, immediate monitoring of key metrics, stop thresholds, and fallback to the previous version or manual processes. Restart or full rollout is determined by whether the observation period for quality, safety, complaints, and cost has been met, not by usage volume. Track the change ticket, test evidence, approval, monitoring results, and rollback decisions with the same identifier. [2] [3]

Separate the rollback decision-maker from the executor; if quality or safety falls below the pre-defined minimum for a use case, or complaints or cost exceeds the pre-defined upper limit, halt full rollout. For external services that cannot be rolled back, test function suspension or fallback to manual processes as the same recovery measure. After rollback, save inputs and logs, and configure the system to not automatically proceed to the new version until restart conditions are met. [2] [3]

• Pre-change: Fix target configuration, allowable scope, baseline version, and test plan
• Pre-approval: Compare performance, safety, group differences, cost, and operational load
• During rollout: Roll out in phases, keeping stop thresholds and responsible parties constantly visible
• Post-rollout: Return complaints and failures to test assets, and reflect them in the next change plan
• Change Owner: Declare the planned scope and boundaries for returning to re-review
• Evaluation Owner: Report by separating controlled comparisons from actual operational changes
• Operations Owner: Demonstrate fallback to the previous version, stoppage, or manual processes