ai-platform-architecture: AI on governed data, every decision tied to a measurement

Context

The AI labs on this site each answer one question: does an agent publish wrong answers on its own, can an evaluation gate tell a regression from noise, does filtered vector search return what it was asked for, does a model learn from the future. An AI architect’s job is the layer above them: how those answers combine into features a company can put on its own data, and what they cost.

ai-platform-architecture is that layer for the same fictional SaaS business as data-stack-lab, sitting on the platform in data-platform-architecture. It covers five features: an analyst over governed data, a help-centre assistant, triage of short free text, scoring models, and category trends from classifier output.

Constraints

Everything is assembled from public labs on synthetic and public data, not from any employer’s or client’s system. The language-model labs ran small local models (Qwen3 1.7B and 8B through Ollama), so the methods transfer to other models while the rates would not. The cost model rests on list prices and assumed token counts; no feature’s bill and no inference server’s throughput were measured, and the decision that depends on them is marked Proposed.

Architecture

• Thirteen decision records: one typed boundary to data; outputs that become state only after checks and, for side effects, a person; abstention as a measured answer; a paired evaluation gate with latency and cost budgets; judges calibrated and audited; retrieval chosen by filtered recall; labelling distinct strings by cluster and margin; corrected counts from classifiers; point-in-time features; promotion on outcomes; one model gateway with replay and budgets; canary releases; a cost per answer before launch.
• Evaluation: answerable, unanswerable, adversarial and boundary cases; deterministic checks before judges; the gate; outcome monitoring after release.
• Threat model: the OWASP Top 10 for LLM Applications (2025), each risk mapped to the controls in this design and to what has been tested, with the untested gaps ranked.
• Quality targets with the measured value next to each, including the three that are not met.
• Runbooks: changing a prompt or model, changing the judge, a reported wrong answer, a provider outage, a spend spike, a review backlog, an injection.
• A model and cost guide pricing two features on nine current models, and the request rate at which a GPU instance of your own costs less than the API.

In the code

The cost model is deliberately small. Its one non-obvious output is the self-hosting break-even: an instance’s hourly price divided by the API’s price per request. That is the request rate a dedicated GPU must sustain before it is cheaper, before anyone asks whether it can.

break_even = {
    gpu: {
        "usd_per_hour": g["usd_per_hour"],
        "requests_per_hour": {m: round(g["usd_per_hour"] / answer_cost(models[m], feature)) for m in models},
    }
    for gpu, g in prices["gpus"].items()
}

Decisions and trade-offs

Results

Twelve of thirteen decisions are backed by measured results. Among them: 0 of 5 planted instructions led to an action; a 4.8-point drop that the paired gate correctly called noise at p = 0.39; and filtered vector search that returned 3.9 of 10 rows until the engine’s iterative scan was used. All 27 figures are in the repository’s EVIDENCE.md.

Three targets are not met. 3 of 37 answers that passed every check and published without review were wrong, against a target under 1%; 4 of 8 unanswerable questions were declined, against 7 of 8; accuracy was 48 of 54, just under 90%. The architecture keeps them as targets; they are what the next iteration of the checks is measured against.

What an answer costs. The same help-centre answer costs from $0.0005 to $0.0475 across the nine models priced. A single-L40S instance costs less than the API above 196 vs 3,918 answers an hour against Claude Sonnet 5 and GPT-6 Luna respectively. The reasoning is in the write-up.

What went wrong

The first README miscounted its own misses. It listed two unmet targets; 48 of 54 is 88.9%, under the 90% accuracy target, so there were three. The table and the text now say three.

A cover headline claimed more than the arithmetic. “A GPU pays off only against the bigger models” assumed that one instance cannot serve 3,918 answers an hour, which nobody measured. It now says what the numbers show: the cheaper the hosted model, the more a GPU of your own must serve.

A cost comparison mixed volume tiers. A draft contrasted $33 a month with $33,000; the second figure was a tier ten times larger. The same feature at the same volume ranges from $33 to $3,300.

What I’d do differently

Close the three gaps in the labs. The grounding check and abstention are where the governed agent is weakest; the next change to it should be judged against these targets.

Write the adversarial suites the threat model ranks first: tenant isolation, injection through retrieved documents, and spend caps under load.

Measure what the cost model assumes. Token counts per call from a running gateway, and the serving rate of an open model on one GPU instance, would turn the break-even from arithmetic into a decision.

The decisions, documents, cost model and checks are in ai-platform-architecture.