Recording a model decision: registration and reassessment

By Ivo Donker — compiled with AI support · Last updated: August 7, 2026

When developing software applications based on Large Language Models (LLMs), selecting a specific model is often treated as a one-off design decision. In a landscape where AI providers roll out model updates, retire API endpoints, or adjust pricing structures every few weeks on average (status: August 2026), that assumption turns out to be a continuity risk in practice. A model choice that is not formally recorded can no longer be reconstructed within three to six months. Developers and system architects forget the exact reasoning, the accepted constraints, and the alternatives tested in the original decision. When questions later arise about rising API costs, declining response quality, or changed processing terms, the basis for a rational analysis is missing.

Periodically reviewing a model choice, moreover, only makes sense when the original criteria were recorded quantitatively and transparently. Without predefined measurement points, a review degenerates into a subjective discussion about how a newly released model "feels". A structured model decision prevents this by anchoring choices in measurable performance, costs, and risk analyses.

Why record a model decision

A model decision forms the administrative and technical foundation under an LLM integration. It not only anchors which model was chosen at a specific moment, it explicitly records why alternative models were rejected. In complex production environments variables change continuously: prompt templates get refined, context windows grow, and providers adjust their hardware infrastructure. Without registration a historical vacuum appears in which decisions have to be evaluated all over again as soon as the team composition changes or a provider pushes an update.

Recording a decision forces a team to be explicit about constraints that would otherwise stay in the dark during the development pipeline. Think of the maximum acceptable latency for the end user, the ceiling for costs per thousand processed transactions, and the legal frameworks around data retention by the model provider. Where the initial process of choosing a model focuses mainly on the comparative analysis and first trade-off at the start of an application, the model decision records those outcomes durably so they serve as a benchmark for the entire software lifecycle.

A recorded model decision also prevents the phenomenon of update-driven detours. Developers sometimes tend to switch straight to a newly announced model as soon as it hits the market. If the decision register states, however, that the current task involves nothing more than simple entity extraction where cost and processing speed take priority over complex reasoning ability, a more expensive or slower model can be ruled out immediately without time-consuming re-evaluations.

What a model decision is

A model decision is an immutable or version-controlled document — comparable to an Architecture Decision Record (ADR) in traditional software architecture — recording the choice of a specific LLM, a specific hosting service, and a specific configuration for a bounded task. The model decision forms the hinge between the exploratory phase and the management phase of an LLM application.

To avoid confusion in the management process, a model decision has to be clearly distinguished from three other documents in the AI lifecycle:

• Model card: This is the document supplied by the model developer or provider (for instance by OpenAI, Anthropic, Meta, or Hugging Face). It describes the architecture, training data, general benchmarks, and known limitations of the model. This article does not cover reading model cards and licenses; see the article on reading model cards and licenses for analyzing this vendor documentation. The model card contains external facts; the model decision contains your internal trade-off.
• Migration plan: This is an operational playbook describing the steps needed to replace model A in production with model B (including prompt transformations, regression tests, and rollout strategies). For the concrete planning and execution of such a transition, consult the guide to model versions and deprecation; this article describes the substantive decision that precedes a migration.
• Monitoring log: This is the continuous stream of runtime telemetry (p95 latency, error rates, daily token consumption, drift measurements). The log supplies the factual data on which a model decision is later reviewed, but contains no decision-making or criteria itself.

A model decision brings these elements together in a clearly bounded framework. It links the external claims from the model card to the internal test data from the monitoring log, and formulates the constraints that determine when a migration plan has to come into effect.

What you record: the decision register

A structured decision register consists of a set of fields recording the context, the quantitative justification, and the constraints of the choice. Formatting the register as a fixed table or structured JSON/YAML file ensures that different teams within an organization apply the same standards.

A complete decision register contains at least the following twelve fields:

1. Date: The date on which the decision was officially approved.
2. Decision-maker / role: The name and role of the person ultimately responsible (for instance Lead Architect or Product Owner).
3. Model + exact version: The unique identifier of the model, including the specific snapshot version or commit hash (for instance gpt-4o-mini-2024-07-18 or meta-llama/Llama-3.1-70B-Instruct on a specific vLLM instance). Avoid using floating aliases such as latest without a version number.
4. Task / application: The specific functionality within the application this model is deployed for (for instance "Summarizing customer service conversations for CRM entry").
5. Rationale: A short qualitative description of the decisive arguments.
6. Measurable criteria: Hard threshold values for performance, cost, and speed.
7. Evaluation evidence: A reference to the test set, benchmark run, or evaluation report the decision is based on.
8. Cost estimate: Expected operational costs based on the estimated volume (for instance cost per 1,000 transactions or TCO of your own hardware).
9. License terms: Permitted purposes of use, data processing agreements (DPA), and guarantees around data retention.
10. Risks: Known weak points of the model for this specific task (for instance sensitivity to hallucinations in long contexts or strict rate limits from the API provider).
11. Review date: Planned date for the next periodic check.
12. Triggers: Events that make an interim review mandatory.

The table below gives a concrete overview of how a decision register is filled in practice for a specific processing pipeline (illustrative example, status: August 2026):

Criteria that make review possible

The heart of a usable model decision lies in the measurability of the criteria formulated. A criterion such as "the model must write high-quality texts" is useless for review, because perceived quality is subjective and shifts as teams get used to LLM outputs. Criteria have to be quantitative, reproducible, and directly linkable to business objectives.

Measurable criteria break down into four categories:

• Functional performance: Accuracy on a specific golden dataset, F1 score on classification tasks, or the percentage of valid output structures (such as JSON or SQL) across 1,000 consecutive requests.
• Operational properties: Time-To-First-Token (TTFT), processing speed in tokens per second, and the 95th percentile (p95) of total response time under normal production load.
• Financial limits: The maximum cost price per thousand tokens (input and output weighted) or the maximum total cost per successfully handled user transaction.
• Safety and robustness: The percentage of accepted prompt injections against a standard test suite, or the error rate on edge cases in the input.

To translate these criteria effectively into practice, raw measurements have to be converted into a clear decision structure. For carrying out and interpreting these measurements in a structured way we refer to turning evaluation results into decision cards, which explains in detail how to formulate quantitative test results into a clear decision framework.

A reliable review also requires that the evidence can be gathered continuously and automatically. In this process, the methodology for testing LLM integrations forms an indispensable pillar. Automated integration tests deliver the hard evidence showing whether an existing model still meets the stated criteria, or whether a new candidate model genuinely performs better on your specific workload.

Review moments

A model decision is not a static document that disappears into an archive once signed. It is a dynamic agreement with an explicit expiry date and a set of action conditions. Reviews take place in two ways: on a fixed calendar rhythm or on the basis of specific events (triggers).

Calendar-based review

For standard production applications a fixed cycle of three to six months is advisable. During this periodic review the team checks whether the assumptions in the decision register still match reality. Are the API costs still in line with the cost estimate? Has the volume changed? Have new models come to market in the meantime that deliver the same performance at a fraction of the price? A fixed cycle ensures the application does not quietly keep running on outdated or needlessly costly infrastructure.

Event-driven triggers

Alongside the calendar review, the decision register has to define specific situations that force an immediate interim evaluation. These triggers keep an organization from reacting only once a service goes down or an invoice comes in unexpectedly high. The most important triggers are:

• Announcement of a model deprecation: The provider indicates that an API endpoint will be discontinued within a certain period.
• Release of a new model version: The provider releases an update of the chosen model or of a directly competing model in the same performance class.
• Price changes: An adjustment of token rates by the current provider or a drastic price cut at a direct competitor.
• Quality regression in production: Continuous evaluation or user feedback shows that the number of incorrect answers, hallucinations, or format errors exceeds the agreed threshold.
• Legal or policy changes: The provider adjusts the terms of service, the privacy policy, or the data retention agreement, breaching the license terms in the decision register.

To spot these triggers in time without the team having to scan news sources manually every day, a structured process is required. For setting up such a signaling process, consult keeping track of model updates structurally. It describes how to capture provider changes automatically and filter them for relevance to your decision register.

Decision process and ownership

A common pitfall in managing LLM applications is the absence of a clear owner of the model decision. When nobody is explicitly responsible for the choice made, nobody feels called upon to act when a trigger fires. Ownership of a model decision has to sit with a specific role within the team — usually the Lead AI Architect, the Senior Backend Developer, or the Product Owner, depending on the size of the organization.

The decision process ideally follows a fixed escalation ladder:

1. Signaling: The monitoring tooling or the periodic calendar check flags that a criterion has been breached or a trigger has fired.
2. Analysis: The developer runs the standard evaluation test suite (as recorded in the evaluation evidence) against the current model and any alternative models.
3. Proposal: The developer proposes a revision of the decision (keep, migrate to a new version, or switch to another provider).
4. Approval (human in the loop): The assigned decision-maker reviews the test results and the financial impact, and approves the new decision.

One strict rule applies here for administrative handling: a changed decision is always a new decision, never an after-the-fact correction of the old document. The decision register works as an append-only log. If a model set to gpt-4o-2024-08-06 is replaced by gpt-4o-2024-11-20, Decision #001 is not overwritten. A new Decision #002 is created that refers explicitly to Decision #001 and documents the reason for the replacement. This preserves the full audit trail and makes it possible to fall straight back on the historical assumptions when problems appear in production.

For a detailed treatment of the roles and responsibilities in the operational phase we refer to the article on ownership of AI applications after go-live. It explains how the handover between development and operations is formally arranged.

Practical execution

Recording model decisions must not become a bureaucratic burden that slows development down. For small teams, indie developers, or homelab administrators, heavy enterprise software is superfluous. A sober and lightweight process works best in practice.

A simple and effective way to record model decisions is to include a DECISIONS.md file directly in the Git repository of the application concerned. In it a fixed Markdown template is used for every decision. Because the decision register is version-controlled alongside the source code, it is clear at a glance which model decision belonged to a specific release of the software.

An anonymized and sober example of a single decision line in a plain text or Markdown structure looks like this:

Decision ID: DEC-20260807-01 | Date: 07-08-2026 | Status: ACTIVE
Task: Summarizing internal incident reports (module: services/reporter)
Chosen model: mistral-large-2407 (hosted through an internal vLLM cluster on 2x RTX 5090)
Rationale: Full data locality required (no external API permitted given confidential incident data); performs on the internal test set on a par with commercial closed models on the summarization task.
Hard criterion: Entity hallucination rate < 0.5% on benchmark set incidents-v1; processing speed > 40 tokens/sec.
Owner: DevSecOps team (contact: T. Jansen)
Review: 2026-11-07 or as soon as a new local open-weights model with >70B parameters appears.

By keeping the template this compact, creating a decision line takes less than fifteen minutes while still containing all the information needed for future audit and migration moments.

Pitfalls

A number of persistent pitfalls come up in practice when setting up and maintaining a decision register. The overview below sets out the most critical points of attention:

• Recording but never reviewing: The decision register is filled in once during project rollout and then never looked at again. As a result, a year later the application is still running on an outdated, more expensive, or slower model version. A decision without a planned review date is administrative ballast.
• Formulating criteria that are not measurable: Using vague goals such as "the answers must sound natural" or "the model must be smart enough". If criteria cannot be tested quantitatively through a test script or benchmark, a review cannot objectively establish whether a model still qualifies.
• Not appointing an explicit owner: The decision is stored in a shared wiki without a specific individual or role being designated as owner. When a trigger fires nobody responds, because responsibility falls between the Development and Operations departments.
• Confusing the model decision with the model card: Relying on the vendor's documentation (model card) instead of recording your own specific application context. A model card states that a model "performs well on Python code", but does not guarantee that the model meets your internal latency requirements for a specific API endpoint.
• Not linking the decision to the application version: Storing the decision in an external location without recording which version of the source code (git commit hash) and which prompt templates belonged to the decision. That makes it impossible during regression testing to trace which prompt changes were made when the model was chosen.
• Forgetting specific event-driven triggers: Relying solely on a calendar review (for instance twice a year) and forgetting to include a rise in API error margins or a deprecation announcement as a trigger. This leads to acute emergency migrations at the moment a provider switches an endpoint off for good.

Avoiding these pitfalls requires no complicated processes, only discipline in applying the agreements made. A structured checklist at every release keeps decisions from lapsing unnoticed.

Registering and reviewing a model decision forms the missing link in a professional LLM lifecycle. Where the initial model selection determines what a project starts with, the decision register ensures that choices stay transparent, measurable, and manageable throughout its entire lifespan. It connects the first selection process directly to continuous monitoring and controlled migration planning. By recording choices explicitly in a sober register with hard criteria and clear owners, developers transform their AI integrations from ad hoc experiments into robust, traceable software components.