Redefining Technology

Artificial Intelligence

What is AI Product Development?

AI product development is the process of taking an AI idea from concept to production through staged evidence gates: a feasibility audit, a proof of concept (POC), a minimum viable product (MVP), and a production release. It differs from classic software development because data risk, model uncertainty, and evaluation-driven iteration decide the outcome — not feature completeness.

What is AI product development?

AI product development is the process of turning an AI idea into a working product through staged evidence gates: a feasibility and data audit, a proof of concept (POC) that settles one falsifiable question, a minimum viable product (MVP) that real users touch, and a production release hardened for scale. Each gate produces evidence — a written verdict, pilot telemetry, an evaluation score — and each gate decides whether the next stage is funded.

The staging exists because most AI product bets are decided on conviction: months of build before the first user, feasibility discovered after the budget is spent, and technical due diligence answered from memory. Gates invert that order. The audit precedes product code, the time-boxed POC precedes the MVP, and pilot data precedes scale-up — so the spend at risk before validation drops from the full build cost to roughly a quarter of it.

How an AI idea moves through its evidence gates

The top row is what gets built; the bottom row is the evidence each stage produces. The audit's verdict authorises the POC, the POC's eval scores authorise the MVP, and pilot telemetry authorises the scale-up. A stage that produced no evidence has not finished.

Read this diagram as a list
  1. Feasibility & data audit — weeks 1–2
  2. POC sprint — 3–4 weeks · one question
  3. Written verdict — kill criteria agreed
  4. MVP & design-partner pilot — 8–12 weeks
  5. Eval scores vs thresholds — accuracy · latency · cost
  6. Production & scale-up — monitored and retrained
  7. Pilot telemetry — activation · retention
After last year's hype, executives are impatient to see returns on GenAI investments, yet organizations are struggling to prove and realize value.
Rita Sallam, Distinguished VP Analyst, Gartner (July 2024) (opens in a new tab)

The abandonment figure is not an argument against POCs — it is the gate doing its job. A weak idea killed in four weeks at a fixed price is the cheapest outcome that idea can have. The expensive failure is the reverse: an idea that skips the gate and dies after a year of engineering.

How does AI product development differ from classic software development?

AI product development differs from classic software development in three ways: data risk replaces requirements risk, model uncertainty replaces deterministic behaviour, and evaluation-driven iteration replaces test-driven completion. A classic build fails when the specification was wrong. An AI build can fail when the specification was right but the data cannot support it — or the model cannot reach the accuracy the workflow demands at a cost the business can carry.

The practical consequence: feasibility is a discovery, not an assumption. The data the idea assumes often does not exist yet, exists in the wrong form, or belongs to an owner who has not committed access. A disciplined AI build therefore starts with a data audit, not a sprint backlog.

Classic software development vs AI product development
DimensionClassic softwareAI product development
Primary riskBuilding the wrong featuresData quality and model feasibility
RequirementsSpecifiable up frontDiscovered through data audits and POCs
BehaviourDeterministic — same input, same outputProbabilistic — outputs vary and drift
TestingUnit and integration tests on code pathsEvaluation suites scored on model behaviour
Unit economicsRoughly fixed per user after launchCost per inference moves with model, prompt, and volume
Definition of doneFeatures complete, bugs closedMetrics hold on live data at acceptable cost
Post-launchMaintenance and feature workMonitoring, drift detection, retraining

RAND's interviews with 65 experienced data scientists trace those failures to misaligned objectives, underestimated data engineering, and the distance between a working demo and a reliable production system. None of the top causes is a modelling problem. All of them are process problems — which is precisely what a gated development model exists to manage.

Why do AI products stall between pilot and production?

AI products stall between pilot and production because a pilot proves capability while production demands reliability, unit economics, and an owner — none of which a demo is asked to prove. Adoption is now close to universal; attributable financial impact is not, and that gap is where most AI budgets are lost.

Adoption is near-universal; measured impact is rare

McKinsey's 2025 global survey drew 1,993 responses across 105 countries. Almost every organisation runs AI somewhere, but fewer than two in five can attribute enterprise-level EBIT impact to it, and only about one in sixteen attributes more than 5%.

Source: McKinsey, The State of AI: Global Survey (2025) (opens in a new tab)

View the data
ItemShare of organisationsNote
Use AI in at least one business function88%up from 78% a year earlier
Say AI is enabling their innovation64%
Report enterprise-level EBIT impact39%most of them put the figure below 5%
Attribute more than 5% of EBIT to AI6%McKinsey's 'AI high performers'

Independent surveys land in the same place. S&P Global Market Intelligence, polling more than 1,000 enterprises across North America and Europe, found abandonment rising sharply year on year, with cost, data privacy, and security the obstacles named most often.

  • Data access was assumed, not committed The pilot ran on an export someone pulled by hand. Production needs a governed pipeline and a named data owner — and that conversation starts months late.
  • The pilot had no acceptance bar Without thresholds agreed in advance, "good enough" is renegotiated at every demo. Nobody can say the pilot passed, so nobody funds it.
  • Unit economics were never modelled A cost per inference invisible across fifty pilot users becomes the largest line item across fifty thousand. Cost belongs in the POC's evaluation.
  • Nobody owned the system after launch Behaviour drifts, providers deprecate endpoints, volumes change. Without an operations owner the product degrades quietly until someone switches it off.

How do you go from idea to POC to MVP to production?

An AI idea reaches production through five stages, each gated on evidence: feasibility audit, POC sprint, MVP build, instrumented pilot, and production scale-up. Working to this sequence, a POC lands in 3–4 weeks, an MVP in 8–12 weeks, and a first production release inside 90 days is the default delivery goal rather than an aspiration.

  1. Run the feasibility and data audit (weeks 1–2)

    Before any product code, audit the data the idea assumes: does it exist, in what form, and who must commit access? The output is a written feasibility verdict a board can act on, including an agreement on what evidence would kill the idea. Time to that verdict is the first KPI: weeks, not quarters.

  2. Build a POC that answers one falsifiable question (weeks 2–6)

    Time-box a proof of concept to a single question — usually whether the model reaches the accuracy the workflow needs on the data that exists, at a cost per inference the business can carry. Keep it rough: a POC is judged on the question answered, not the UI. A fixed price caps the downside of a negative answer.

  3. Gate the MVP decision on the verdict (week 6)

    Kill, pivot, or fund. An idea killed here has cost a feasibility report and one fixed-price sprint — roughly a quarter of what an ungated build burns before its first validation. An idea funded here proceeds with the feasibility risk retired.

  4. Build the MVP real users touch (weeks 6–16)

    Rebuild on production architecture — product, model, infrastructure, analytics — rather than extending POC shortcuts. Recruit design partners before anything is polished, and instrument telemetry that separates novelty from habit. An 8–12 week window is realistic for a gated scope.

  5. Pilot, evaluate, and harden for production (weeks 16 onward)

    Run an instrumented pilot and track activation: the share of invited users who complete the core workflow unaided. The evaluation suite becomes the regression harness, and security, tenancy, and cost move from footnotes to release blockers.

What the evidence says about AI product bets

≥30%

of generative AI projects abandoned after proof of concept

Source: Gartner

46%

of AI proofs of concept scrapped before reaching production

Source: S&P Global Market Intelligence

39%

of organisations report AI impact on enterprise EBIT

Source: McKinsey State of AI 2025

Why is evaluation-driven iteration central to AI products?

Evaluation-driven iteration is central to AI products because model behaviour is probabilistic: the same input can produce different outputs, quality varies across the input distribution, and performance drifts as live data diverges from training data. A pass/fail test suite cannot certify that behaviour. An evaluation suite — a scored set of real cases the product must handle, with acceptance thresholds agreed before the build — can.

The eval suite takes over the role the specification played in classic software. Stakeholders agree what acceptable output means for a non-deterministic system; every model change, prompt revision, and data update is re-scored; and a regression blocks the release exactly as a failing test would. Without it, model debates are settled by whoever saw the most recent impressive demo.

  • A golden set of real cases One hundred to three hundred examples from live data, labelled by the people who own the decision — not by the build team, and not by the model being tested.
  • Thresholds agreed before the build The score at which the product ships, written down while nobody is under deadline pressure. It is what stops "good enough" being renegotiated at each demo.
  • Adversarial and edge cases The inputs that embarrass the system: missing fields, contradictory records, hostile prompts, out-of-distribution documents. Production supplies these whether or not the suite does.
  • Cost and latency budgets A correct answer that is slow and expensive fails the workflow as surely as a wrong one. Score cost per inference and p95 latency alongside accuracy.

Evaluation does not stop at launch — drift starts the moment real users arrive, which is why a shipped AI product needs an MLOps function monitoring, retraining, and re-scoring it in production. The product, its data pipelines, and that operational layer together form an end-to-end AI system — the thing the development process ultimately delivers.

POC vs MVP: which should you build first?

Build the POC first whenever the core technical question is unsettled — and for most AI ideas it is. The POC answers whether it can work: model capability on real data, at a workable cost per inference. The MVP answers whether anyone will use it: workflow fit, retention, willingness to pay. Reversing that order is how twelve-month builds die on a feasibility problem a four-week sprint would have found.

Which to build first, by the risk that is actually open

HighTechnical uncertaintyLow

POC first

  • Accuracy, latency, or cost in doubt
  • Buyers are already asking for it
  • 3–4 week falsifiable sprint

Sequence both risks

  • Novel capability, unproven buyer
  • POC first, then a design-partner MVP
  • Never both risks in one build

Go straight to product

  • Known pattern, known buyer
  • Risk is delivery, not discovery
  • Scope the MVP and start

MVP first

  • Document extraction, forecasting on clean data
  • Only real users retire adoption risk
  • Instrument the pilot from day one

Demand is knownMarket uncertaintyDemand unproven

Place the idea by the two uncertainties that can kill it. Only one quadrant justifies going straight to product; the rest buy their answer with a time-boxed POC first, because feasibility is cheaper to retire than demand.

Whichever comes first, a POC is not a small MVP. The shortcuts that make a POC fast — hardcoded flows, single tenancy, no eval harness — make it misleading at scale, so the MVP is a rebuild, not a polish pass.

  • Fundraising ahead POC plus verdict. A working proof of concept and a written feasibility verdict give the deck evidence instead of claims, and give a technical due-diligence reviewer something real to review.
  • Corporate venture constraints POC outside the core stack. A kill-or-scale verdict inside a quarter, built where the experiment cannot destabilise production systems.
  • Enterprise procurement ahead POC with the security review attached. Tenancy, residency, and audit logging decide whether an enterprise buyer can sign.

What is at risk before an AI build is validated?

Before validation, an ungated build puts the entire programme budget at risk, because nothing is proven until the whole thing ships. Gating changes the exposure: the audit and POC are priced separately, so roughly a quarter of build spend is committed before the first evidence arrives.

4 weeks

to a written feasibility verdict, against 12 on an ungated build

8 weeks

to the first design-partner session on the real product

25%

of build spend at risk before the first validation gate

The arithmetic is concrete. Take a €400,000 AI product programme. Run as one continuous build, all €400,000 is committed before anyone knows whether the data supports the idea. Run through gates, about €100,000 — audit, POC, and MVP scoping — is exposed before the first evidence lands, and the remaining €300,000 is released only against a written verdict and a scored evaluation. Where one idea in three fails its POC, that is €300,000 preserved per failure.

What sits inside that quarter changes by sector. In regulated finance and healthcare, most of the audit budget goes to data access and consent, and the POC must demonstrate auditability alongside accuracy. In industry the constraint is instrumentation — a predictive maintenance programme cannot start until failure history is queryable. In consumer and SaaS products the technical risk is lower and adoption risk higher, moving spend toward design-partner recruitment.

What happens after the POC verdict?

After a funded verdict the question changes from "can this work?" to "will it still work when the build team leaves?" That is an operations problem, and the eighteen months that follow decide whether the product becomes an operated system or another abandoned pilot.

From a funded verdict to an operated product
  1. Months 2–4

    Funded MVP build

    Production architecture from the first commit: real tenancy, real auth, the eval suite wired into CI. Design partners are recruited now, before anything is polished.

    Decision: can a design partner complete the core workflow unaided?

  2. Months 4–6

    Instrumented pilot

    A closed group of real users on real data. Telemetry separates novelty from habit, and cost per active user is tracked from the first session rather than reconstructed later.

    Decision: activation and unit cost against the agreed targets.

  3. Months 6–9

    Production hardening

    Security review, multi-tenancy, rate and cost controls, failure-mode runbooks, support training. The eval suite is promoted from build tool to release gate that can block a deploy.

    Decision: can operations run this without the build team?

  4. Months 9–18

    Operate, monitor, retrain

    Drift monitoring, scheduled re-scoring against the golden set, and a documented path for provider or model changes. New capability ships against the same thresholds.

    Steady state: eval scores and unit cost reviewed on a fixed cadence.

The windows assume a POC that closed at week six. Phases widen only when the previous one produced numbers — eval scores, activation, unit cost — against thresholds agreed before the build.

How do you evaluate an AI product development partner?

Evaluate a partner on what they will put in writing before the contract starts: the kill criteria, the evaluation thresholds, the unit-cost model, and who owns the system after launch. A team that will not name the conditions under which it would tell you to stop is selling build hours, not evidence.

  • A polished POC Effort spent on the interface is effort not spent on the question. A POC that looks finished usually answered nothing falsifiable.
  • Accuracy quoted without a denominator "95% accurate" means nothing without the test set, the class balance, and the cost of the remaining errors. Ask what the 5% looks like in the workflow.
  • A model choice made before the data audit Architecture decided in the pitch is a decision made without evidence.

Key terms

Proof of concept (POC)
A deliberately rough, time-boxed build whose only job is to answer one falsifiable question — usually whether a model reaches the accuracy the workflow needs, on the data that exists, at an affordable cost per inference. Its deliverable is a verdict, not a codebase.
Minimum viable product (MVP)
The smallest version of the product that real users can complete a real workflow in, built on production architecture and instrumented so the pilot produces decision-grade data on activation, retention, and cost.
Evaluation suite
A scored set of real cases an AI product must handle, with acceptance thresholds agreed before the build. It replaces the specification in classic software: every model, prompt, or data change is re-scored, and a regression blocks the release.
Kill criteria
The evidence, agreed in writing before work starts, that would end the project. Naming it in advance is what converts a negative POC result from an argument into a decision the board can act on.
Activation rate
The share of invited pilot users who complete the core workflow unaided. It is the clearest early signal that an AI MVP fits the job it was built for, because it cannot be inflated by curiosity traffic.
Model drift
The gradual decay of model accuracy as live data, user behaviour, or provider models change away from the conditions the system was validated in. Detecting and correcting it is the core of post-launch MLOps.

Frequently asked questions

The questions founders, product leaders, and corporate venture teams ask before committing budget to an AI product.

How long does it take to build an AI product MVP?

8–12 weeks for a gated scope, after a 3–4 week POC has settled feasibility — with a first production release inside 90 days as the default goal. The timeline holds because the feasibility audit precedes the build; it stretches when data problems are discovered mid-build instead of in week one.

What is the difference between an AI POC and an MVP?

A POC answers a feasibility question; an MVP tests the market. The POC is deliberately rough, time-boxed to 3–4 weeks, and judged on whether one falsifiable question was answered. The MVP is built on production architecture, put in front of real users, and instrumented so the pilot produces decision-grade data — activation, retention, and willingness to pay.

Why do most AI projects fail?

For organisational reasons, not algorithmic ones. RAND's root-cause analysis — built on interviews with 65 experienced data scientists — points to misaligned objectives, underestimated data engineering, and the gap between a working demo and a reliable production system. Gartner adds poor data quality, weak risk controls, escalating costs, and unclear business value. Every one is addressable with staged evidence gates; almost none with a better model.

Do I need data before starting AI product development?

You need to know what data exists — which is different from having it production-ready. The feasibility audit establishes whether the data the idea assumes exists, in what form, and who must commit access before the architecture is fixed. Discovering a data gap in week one costs a report; discovering it in month six costs the build. Some POCs run on proxy or synthetic data, but the verdict must say so explicitly.

Can an AI POC become the production product?

No — and it should not try. The shortcuts that make a POC fast, cheap, and falsifiable — hardcoded flows, single tenancy, no evaluation harness, no cost controls — make it misleading at scale. The POC's deliverable is a verdict, not a codebase. The MVP is rebuilt on production architecture with the feasibility risk retired, which is faster than rehabilitating prototype code.

How do you know whether an AI MVP worked?

Three numbers, all agreed before the pilot starts: activation — the share of invited users who complete the core workflow unaided; eval score against the golden set on live data; and cost per active user at pilot volume, projected to scale. A pilot that produces enthusiasm but none of those three numbers has not settled anything, and should not release the next tranche of budget.

Put your AI idea through its first evidence gate

A 30-minute consultation returns a scoped feasibility audit and a fixed-price POC plan — with kill criteria in writing before you commit build budget.

Last updated: