Build vs Buy for AI: A Decision Framework That Survives Contact

Why the cost comparison misleads you

The meeting usually looks the same. Two vendors have demoed. Someone on the engineering side says the team could build a version of this in a quarter. Finance wants a number for each column. And whoever runs the meeting is being asked to compare a published license price against an internal estimate, which is not a comparison at all — one figure is a contract term and the other is a guess whose accuracy depends on how honestly the estimator counted the parts nobody demos.

The more durable framing is this: build and buy are two different ways of distributing risk. Buying moves control to a vendor — their roadmap, their model versions, their outage, their pricing decision next renewal. Building moves maintenance to you — your on-call rotation, your evaluation harness, your engineer who understands the retrieval logic and might leave. Neither is safer in general. The framework below is a way of finding out which set of risks your organization can actually carry, and it works because the questions are answerable with facts you already have rather than forecasts you do not.

Question 1: Where is the data allowed to go?

This one is a gate, not a preference, and it belongs first because it can end the conversation before anyone argues about features. If the data is regulated — controlled unclassified information, protected health information, anything living inside an accredited boundary — then the vendor question narrows to a factual one: can this product legally process this data where it sits, today, with paperwork you can point at?

For federal buyers the check is concrete. The FedRAMP Marketplace describes itself as the authoritative place to confirm whether a cloud service offering has a FedRAMP designation, is working toward certification, or is connected to agency reuse. Those are three different states, and only the first is something a security team can rely on now. A vendor that is working toward a designation is telling you about their roadmap, not about your authorization.

Notice that this gate is usually narrower than people treat it. An agency team that wants document classification may face a hard "no" for case files containing personal data and a comfortable "yes" for the public rulemaking corpus. The gate applies per data set, not per project, and splitting the project along that line is often the whole solution. If you are working in an environment with no outbound connectivity at all, our guide to air-gapped AI deployment covers what remains possible when the network answer is no.

Question 2: Does your data change the answer?

This is the differentiation question, and it gets muddled because in a decision meeting everything sounds strategic. Replace the word "strategic" with a testable one: does our data or our process materially change the quality of the output?

A useful way to force the answer is to ask what you would have to teach a vendor for their product to reach the accuracy you need. If the honest reply is "our labeling conventions, our exception rules, and two years of examples of what our reviewers actually accept," then you are already doing the build — you are simply doing it inside someone else's product, on their release schedule, with your institutional knowledge encoded as configuration you cannot export. If the reply is "nothing, we would just turn on the connectors," buy it, and spend the saved engineering time on something only you can do.

The failure mode here runs in both directions. Teams build commodity capability because it feels more serious, and teams buy differentiated capability because a demo looked convincing on generic examples. The demo corpus is the tell. If a vendor cannot show performance on documents that look like yours, the demo established that the product works, not that it works for you.

Question 3: How deep does it reach into your systems?

Integration depth is the most consistently underestimated variable in this decision, and it is the one that makes "buy" quietly turn into "build" three months after signature.

A product that sits at the edge — a web application people log into, an API your service calls — is inexpensive to adopt and inexpensive to abandon. A product that must read from six systems of record, write back to two, respect your permission model, and reconcile when a nightly job fails is a build regardless of whose logo is on the login page. You will write the connectors, the identity mapping, the retry logic, and the reconciliation report, and you will maintain them.

The research literature named this a decade ago. In Hidden Technical Debt in Machine Learning Systems (Sculley et al., NeurIPS 2015), the paper's opening figure carries the caption that "only a small fraction of real-world ML systems is composed of the ML code," with the surrounding infrastructure described as vast and complex — configuration, data collection, feature extraction, serving, monitoring, and the glue code between them. Buying gets you the small box in the middle. The rest of the diagram is your work on either path, which is why the honest version of many buy decisions is buy the model, build the system.

So count the integration surface explicitly before deciding. List every system the thing must read from and write to, by name. That list is the real scope, and it is usually longer than the feature comparison anyone prepared.

Question 4: Who owns the result?

Ownership splits three ways and each is a separate contract question: the model artifacts, the data you send, and the pipeline code around it. Three specific things to ask, before the pilot rather than after it:

• If you leave, what do you take? Prompts, retrieval indexes, labeled data, evaluation sets, fine-tuned weights — each either exports or does not.
• What happens when the model version you validated against goes away? This is documented behavior, not a hypothetical. Anthropic, for example, publishes a model deprecations page with lifecycle states — active, legacy, deprecated, retired — and a commitment to at least 60 days' notice before retiring a publicly released model. Ask each vendor for its equivalent policy in writing. Your accreditation evidence was gathered against a specific version; plan for that version to be retired.
• Who can be told what your system does? Vendor terms differ on whether your usage informs their product, and the answer belongs in writing.

For federal work, ownership is decided by clause. FAR 52.227-14, Rights in Data—General, gives the Government unlimited rights in data first produced in performance of the contract, with defined carve-outs for limited rights data and restricted computer software. On the defense side, DFARS 252.227-7014 governs other-than-commercial computer software, and DFARS 227.7203-5 states that the clause provides a nominal five-year government purpose rights period, after which the Government has unlimited rights in the software. If who-owns-the-result matters to your program, it is settled in the contract file, and an architecture that assumes otherwise will not change the outcome.

The split most teams actually land on

Run the four questions and the answer is rarely a clean column. It is a seam, and the seam usually falls in the same place: buy the commodity layer, build the layer where your data and your rules live.

The commodity layer is model inference, vector storage, orchestration plumbing, observability, and identity. These are well-served by products, they improve without your effort, and switching costs stay manageable if you keep the interface thin. The differentiated layer is retrieval over your corpus with your access rules, evaluation against your ground truth, and the workflow your staff actually uses. That layer encodes decisions no vendor can make for you, and it is where your accuracy comes from.

Naming the seam out loud is most of the decision. It also converts an argument into a design: once the boundary is written down, procurement can move on the commodity side while engineering starts on the differentiated side, instead of both waiting on one all-or-nothing verdict. If model selection is part of your commodity layer, our guides on choosing a model tier and open-weight versus frontier API cover that trade in detail.

Six ways this decision goes wrong

1. Comparing a price to an estimate. If you are going to compare numbers, cost three years of ownership on both sides — license, integration, evaluation, monitoring, and the staff time to run it — or skip the numbers and decide on the four questions.

2. Buying to avoid hiring. A product does not relieve you of needing someone who can tell whether the output is right. Evaluation capability is the one thing you cannot outsource, because judging quality requires knowing your domain. Teams discover this when the pilot produces confident output nobody is qualified to check.

3. Building to avoid procurement. Slow paperwork is a real cost and a bad reason to accept a multi-year maintenance obligation. If the honest driver is that a contract takes months, fix the contract path.

4. Skipping the evaluation harness. Whether you build or buy, you need a held-out set drawn from your real data and a scoring method your subject-matter experts agree with. Write the acceptance criteria before the pilot; criteria written after the demo tend to describe the demo. Our guide on how serious teams test AI systems goes deeper on this.

5. Ignoring the operating half. The NIST AI Risk Management Framework (AI 100-1) organizes AI risk work into four functions — Govern, Map, Measure, and Manage — and Measure and Manage do not end at deployment. Building means you own them permanently. Buying means you own them anyway, with less visibility into the component you are measuring.

6. Deciding once. The right answer changes when a vendor ships the feature you were building, when your volume changes the unit economics, or when a policy shifts. Put a review date on the decision and record what would have to be true to revisit it. In federal environments, procurement conditions move too — the GSA OneGov arrangements and OMB's LLM procurement guidance both changed what buyers must ask vendors for.

A decision you can defend

Reduce all of it to one page anybody can read: the data gate and the evidence for it; the differentiation claim stated as a testable sentence about your data; the integration surface listed system by system; the ownership terms in writing, including what you take with you; the evaluation criteria and who signs off; and the date you will revisit the decision.

If you cannot fill in a box, you are not ready to decide — and the box that is empty is almost always evaluation. That is worth knowing before the contract, because a team that cannot measure quality will not be able to tell whether the build succeeded or the purchase was a mistake. Everything else in this framework is recoverable. That one is not.

Sources: Sculley et al., Hidden Technical Debt in Machine Learning Systems (NeurIPS 2015); FedRAMP — The FedRAMP Marketplace; Acquisition.gov — FAR 52.227-14 Rights in Data—General; Acquisition.gov — DFARS 227.7203-5 Government rights; Acquisition.gov — DFARS 252.227-7014; NIST — AI Risk Management Framework (AI 100-1); Anthropic — Model deprecations. Analysis and framing by Precision AI Academy.

Common questions

Is build vs buy for AI a cost decision? Not usefully. A license price is a known number and a build cost is an estimate, so comparing them directly favors whichever side you measured more carefully. The decision is better framed as which risks you are willing to own: buying moves control to the vendor, building moves maintenance to you.

When does buying an off-the-shelf AI product clearly win? When the work is not differentiated by your data, the product can legally run where your data lives, it touches your systems at the edge rather than deep inside them, and you can leave with your data if you need to. If all four hold, building is usually spending engineering time to reproduce something already available.

What do teams underestimate most when they build? The system around the model. Sculley and colleagues made the point in their 2015 NeurIPS paper on hidden technical debt: only a small fraction of a real-world ML system is the ML code, and the surrounding infrastructure is vast and complex. Data plumbing, evaluation, monitoring, and on-call ownership outlast the modeling work.

Who owns a model built under a federal contract? It is decided by the data rights clause, not the architecture. FAR 52.227-14 gives the Government unlimited rights in data first produced in performance of the contract, with carve-outs for limited rights data and restricted computer software. For DoD software under DFARS 252.227-7014, government purpose rights run a nominal five-year period per DFARS 227.7203-5, after which the Government has unlimited rights.