How AI Systems Actually Get Built: Why Version 1 Is Never the Final Product

By Amin Rabinia · Founder, Glissando AI

At some point in every AI build, the client sees the first version run on their actual data. And in almost every one of those moments, the first number that comes back is lower than what was hoped.

For the RFQ automation system we built with a B2B procurement team, that moment came early. V1 accuracy was low. Coverage was incomplete. Speed wasn't where it needed to be. The client looked at the numbers and then looked at us.

We weren't surprised. We'd told them this would happen.

Not because we planned to do poor work — but because we'd built a clear AI strategy and roadmap before writing a single line of code, and that roadmap sequenced V1's job correctly: prove the core works, not prove the finished product works. Those are different jobs, and confusing them is how AI projects fail.

Why You Can't Fully Specify an AI System Upfront

Business owners buying AI for the first time often assume the process looks like buying software: write down the requirements, a developer builds to spec, you get what you described. That model works well for rules-based systems where every case fits a rule you knew about in advance.

AI systems — especially the kind doing real interpretation, research, or classification — don't work that way. Not because the builders are guessing, but because the input data doesn't reveal its quirks until it's flowing through the system. Real requests look different from sample requests. Edge cases that didn't appear in planning appear in volume two. The language users actually use doesn't match what anyone imagined in a meeting room.

This isn't a failure of planning. It's the nature of working with language, context, and real-world messiness. The first version of any serious AI build will be shaped by real production data — and you only have that data once something is running.

The implication: budget for iteration from the start, not as a contingency. The roadmap should say "V1 proves X, V2 targets Y, V3 is production-ready" — not "we'll deliver the finished thing on date Z." If you're evaluating what a real AI strategy versus a project plan looks like, this distinction is usually where the difference shows up first.

The One-Goal Rule: What V1 Is Actually For

When we started the RFQ build, we deliberately made V1's scope narrow. One goal: can the system read an incoming quote request, search for relevant vendors, filter out the noise, and classify what's worth pursuing? That's it. No ranking. No historical scoring. No formatted output report. Just: does the core signal exist?

This is the one-goal rule, and it shapes every AI build we do. You start with the single most important thing and prove it works before you add anything else. The reason is simple: if the core doesn't work, nothing built on top of it matters. If the core does work — even imperfectly — you have something real to calibrate against.

When V1 ran on real RFQ data, it found vendors. Not always the right vendors. Not quickly. But the signal was there. That told us the approach was correct. It also told us exactly where accuracy broke down, which cases the system mishandled, and what the data actually looked like in production. That knowledge is worth more than any planning document, because it comes from the system doing the job.

V2 was built from what V1 taught us. Accuracy targets, edge case handling, coverage improvements — all calibrated against production inputs. V3 brought speed and scale. By production, the system was processing quote requests in under 60 seconds with accuracy above 90%, and the team's capacity had effectively tripled. The full story is in our RFQ automation case study.

Setting Expectations That Hold Up

The reason the client wasn't alarmed by V1's numbers wasn't luck. It was the work we did before the build started.

We built a full AI roadmap that sequenced the work quarter by quarter, with ROI as the organizing principle. What matters most comes first. What can be deferred gets deferred. Each phase has a clear definition of done — not a progress update, but a measurable result. When the roadmap says "V1's job is to prove feasibility," a low V1 accuracy score is confirmation the roadmap is working, not evidence it's failing.

That framing changes the conversation completely. Instead of "V1 is underperforming," the update is "V1 confirmed the approach is correct and showed us exactly what V2 needs to fix." Those are very different things to communicate to a team or a board.

We've written about the four pillars of a real AI strategy that underpin this kind of structured thinking. The point isn't to produce a document — it's to build shared understanding of what each phase is for before anyone sees a number that's lower than expected.

The Mistake: Building V1 Like It's V3

There's an understandable temptation, especially when a project has internal visibility, to try to make V1 impressive. To add features before the core is proven. To optimize before the approach is confirmed.

We've seen this fail. Not in our builds — but in conversations with companies that came to us after a previous AI investment didn't land. The pattern is almost always the same: the team tried to build everything at once, there was no clear "prove this one thing first" moment, and the result was a system that almost worked at many things rather than actually working at one thing. Scope without sequence is how AI projects spend a lot of money and deliver nothing in production.

Our rule: prove the model works before you optimize or add features. It always starts with basic software before advanced AI comes in. Once the core is confirmed, you earn the right to make it faster, smarter, and more complete. Not before.

On a different build — the AI design tool we built for a furniture company — there was a point mid-project where we'd planned a moodboard feature that users turned out not to care about. Because we were building iteratively and the core was already in users' hands, we caught that early. We redirected the effort toward what actually mattered. In a single-delivery model, that discovery comes after you've already built the wrong thing. The details are in our furniture design AI case study.

What This Means for Your AI Project

If you're scoping an AI build — whether it's an agent that does research, a tool that handles intake, or an automation that replaces a repetitive process — here's what to take from this:

Start with one thing. What is the most valuable single thing the system needs to do? That's V1's only job. Not the full workflow. Not the polished output. Just: can it do the core task at all?

Plan for data you haven't seen yet. Your production data will surprise you. Build in cycles to respond to those surprises — because it's not a failure when the data surprises you, it's information.

Sequence by ROI, not by technology. The question "what should we build next?" should be answered by "what closes the biggest gap between current performance and business value?" — not by what's technically interesting. A roadmap organized this way produces different decisions than one organized by feature completeness.

Give V1 a short timeline. We typically run first build cycles in four to six weeks. Long enough to build something real, short enough that if the approach is wrong, you find out before you've sunk six months into it. If you're trying to estimate how long and how much an AI project like this might cost, our AI MVP scope estimator is a practical starting point.

Communicate what V1 is for — before it runs. The number that comes back from V1 will look alarming if people expect V3. It will look like progress if people know what V1's job actually is. That framing conversation is part of the work, and it belongs before the first build, not after it.

What "Good Enough for V1" Actually Means

V1 is good enough when it answers the question it was built to answer. For the RFQ system, that question was: can the agent find relevant vendors from a real quote request? V1 answered yes — imperfectly, slowly, with gaps. But yes.

That's the milestone. Not a polished product. Not an impressive demo. A confirmed answer to a specific question about whether the core approach works.

If V1 answers no — if the core doesn't work even imperfectly — that's valuable too. You learn that in weeks, not months, and you either pivot the approach or stop the project before more resources go in. Either outcome is better than discovering the same thing after a long single delivery.

Most AI projects that fail don't fail because the technology was wrong. They fail because nobody defined what V1 was supposed to prove, so there was no early checkpoint to catch the problems. By the time the problems were visible, too much had been built on top of them.

If you're still figuring out whether an AI agent or automation is the right fit for a workflow in your business, this post on how we structured the full RFQ agent walks through the anatomy in detail — intake, research, ranking, human approval — and the pattern transfers to most research-and-recommend workflows.

The Takeaway

Our first version of the RFQ system wasn't good enough. We told the client that before we started. We defined exactly what "good enough for V1" meant, built a roadmap that sequenced what came next, and used real production data from V1 to make V2 better. V3 ran in production at the performance level the business needed.

That sequence — narrow scope, confirmed core, calibrate on real data, expand — is the only reliable way to build AI that actually works. Anyone promising you a finished AI system, fully specified upfront, delivered on a fixed date, is describing a process that doesn't account for what production data does to a plan. Be skeptical.

The right question to ask any AI partner isn't "can you build this?" It's "how do you sequence the work so we learn fast and build on what's real?"

Scoping an AI build and want a realistic plan? AI Strategy Consulting →