What is a Conversational & Generative AI System?

Which use case should you start with?

Start where question volume is high and the answer already exists in a written, current document — that pairing is the only one that produces a defensible number inside a quarter. The other three combinations are real work, but they belong after the first assistant has earned its budget.

Inside the winning quadrant, four patterns cover almost every first build we ship. They share the same stack and differ in what they are judged on:

• Support copilot — Grounded in the help centre and resolved tickets. Judged on containment with citation and on escalation quality — never on deflection alone, because a deflected customer who came back angry counts twice.
• Internal knowledge assistant — Policies, SOPs, and the knowledge that currently lives in three senior people. Judged on onboarding time and on the fall in repeat questions routed to those three.
• In-product copilot — Embedded where the user already works, answering from product documentation and tenant data under existing permissions. Judged on time to first successful action.
• Sales enablement assistant — Proposal and RFP drafting grounded in your case studies and pricing rules. Judged on cycle time and on how much of the draft survives review — the same grounded stack applied outbound, as in our guide to LLM-based outreach systems.

What changes by industry is the cost of a wrong answer, not the architecture. In e-commerce support, a wrong answer costs a return; the design pressure is on order-aware retrieval and speed. In regulated sectors, the same wrong answer is a compliance event, so answer-level audit trails, transcript retention rules, and a documented refusal policy become launch gates. In enterprise SaaS, tenant isolation is the constraint that shapes the index: retrieval must be scoped to one customer's data before ranking, not filtered afterwards.

How do you build a production conversational AI system?

A production conversational AI system is built by scoping one knowledge domain, indexing it with permissions intact, and wrapping the assistant in guardrails and evaluation before any user sees it. Scoped that way, a grounded assistant reaches pilot in four to eight weeks and a first production release inside 90 days — expansion by team and channel follows the pilot's measured numbers.

1. Scope one knowledge domain (weeks 1–2)

   Pick a single domain — the support knowledge base, HR policy, product docs — and record the baseline the system will be judged against: first-response time, handoff rate, cost per resolved conversation. This step forces an honest audit; knowledge owners must say which documents are current and which are quietly out of date.

2. Index with permissions intact (weeks 2–5)

   Connect the sources, extract structure, choose chunking and ranking deliberately, and carry access rules into the index. A retrieval layer that ignores permissions is a data leak with a chat interface.

3. Engineer the assistant (weeks 3–6)

   Prompt design, retrieval tuning, and mandatory citations. Multi-model routing balances quality, latency, and cost per query — the hardest questions go to the strongest model, the routine ones to the cheapest that passes evaluation.

4. Build guardrails and the evaluation suite (weeks 5–7)

   Red-team the assistant before users do. Subject experts review evaluation sets — not vibes — and the suite encodes the rule that matters most in production: refuse and escalate rather than guess.

5. Shadow-run, then tune escalation (weeks 7–9)

   Run alongside the human workflow. When confidence drops, the handoff must arrive with the transcript and retrieved context attached, and the receiving team must be staffed for the new volume mix — escalation quality is a launch gate, not a nice-to-have.

6. Launch, then mine the gaps (week 9 onward)

   Conversation analytics take over: containment, escalation quality, and the gap report feed a weekly loop of indexing and documentation. The assistant improves because the knowledge base improves.

How do you measure whether the system works?

A conversational AI system is measured on four production metrics — containment with citation, first-response time, escalation quality, and evaluation pass rate — against the baseline recorded before the pilot, never against demo impressions.

• Containment with citation — The share of conversations resolved by the assistant with a source shown and no handoff. On a well-indexed support domain we engineer toward roughly 45% resolved without handoff — from a starting point of zero.
• First-response time — The queue disappears for contained conversations: a first response that took 25 minutes in a human-only workflow arrives in about a minute.
• Escalation quality — The share of handoffs that arrive with correct context attached. A high containment number with broken escalations is a worse customer experience than no assistant at all.
• Evaluation pass rate — The suite's pass rate on every prompt or model change. Model updates shift behaviour under unchanged prompts, so this metric is permanent, not a launch-week artefact.

The arithmetic is worth doing before the build rather than after. A support team handling 4,000 conversations a month at twelve minutes of agent time each spends 800 hours a month answering questions. Contain 45% of them with a citation and 360 of those hours come back — and the queue in front of the remaining 2,200 conversations shortens, which is where the customer-visible improvement actually shows up.

That study measures humans assisted by a model rather than an autonomous assistant, and its shape holds in delivery work: gains land on newer staff and repetitive questions, not on your best agent's hardest call. It is also the argument for measuring by segment — a single containment average hides how the assistant handles the top twenty intents and how badly it handles the tail.

Two disciplines keep the numbers honest after launch. Transcript review — sampled, structured, and owned by a named person — catches the failures metrics compress away, even though reading your own users' conversations feels uncomfortable at first. And the gap report must feed a content backlog with an owner, because an assistant grounded in a decaying knowledge base decays with it. Programmes that scale past the pilot are the ones that treat these as operations, not as a research phase that ended at launch.

What happens after the pilot?

After the pilot the work shifts from building an assistant to running one: a content backlog with an owner, an evaluation suite that gates model upgrades, and a retrieval layer monitored like any other production dependency. Most assistants that quietly stop being used were fine at launch and unmaintained six months later.

Five patterns account for nearly every assistant that stalls after a promising launch. All five are operational, and cheaper to design out than to repair:

• A decaying index — Answers that were right at launch are wrong at month six because nothing re-indexed the policy that changed. Freshness is a pipeline property, not a launch task — the same discipline covered in our data ingestion primer.
• Permissions bolted on afterwards — Retrieval that ignores access rules is a data leak with a chat interface. Carrying permissions into the index on day one costs days; retrofitting them means rebuilding it.
• Escalation without context — A handoff that arrives as a bare transcript makes the customer repeat themselves, which is worse than never offering the assistant. The receiving agent needs the retrieved passages and the confidence signal too.
• Model upgrades shipped untested — Behaviour shifts under unchanged prompts whenever the underlying model changes. Without a regression suite you find out from a customer, and the fix arrives after the screenshot has been shared.
• No owner for the gap report — The list of questions the assistant could not cite is the most valuable output of the whole system. Unowned, it is a log file; owned, it is a documentation roadmap ranked by real demand.