Conversational analytics architecture: follow-ups, state and clarifying questions

Two fields share this name

Search for conversational analytics and half the results concern contact centers: transcribing calls and chats, then mining them for sentiment, coaching and compliance. The other half concern asking questions of structured business data. Both are legitimate, and they share almost nothing technically. If you came here about call transcripts, this post will not help. From here on, the term means conversing with your warehouse, CRM and other business data, which is sometimes called conversational BI.

Most pages on this version stop at the benefits, or describe a single question being turned into SQL. We have a separate post on text to SQL in production. This one covers what the second question adds. That’s where these systems either become useful or start producing numbers nobody can reconstruct.

Where a follow-up goes wrong

Take a synthetic exchange. A sales lead asks for net revenue by region last quarter, gets a table, then says "just EMEA", then "why is that down", then "what about the year before". The fourth message has no metric, no region, and a relative date whose anchor sits two turns back. The system must recover all of them, and each recovery is a chance to drift without anyone noticing.

The common design passes the transcript to the model and asks it to rewrite the question or produce fresh SQL. It works in most demos. It also has known limits: follow-ups are rephrased using earlier context, the model holds no state between requests so the full history is reprocessed each turn, and it cannot use the results of earlier SQL queries unless they are sent back. Cost also grows with each round, and long or shifting conversations may need a reset. Those are fair engineering trade-offs, and any transcript-based design inherits a version of them.

Regeneration is the failure I’d watch for. Let each turn write a new query from a transcript and turn four may use a different revenue table from turn one, at which point the numbers no longer reconcile. Nothing errors. The person sees a figure that is inconsistent with one on their screen and loses trust in all of them.

Keep the analytic frame outside the chat

The fix is to hold what the last answer meant as data. Call it the frame: the resolved definition, the filters as applied, the period with its absolute dates, the grain, and the sources. The model proposes a change to the frame, and code applies it and rebuilds or edits the query. History remains available for language, while the frame holds the facts.

Three benefits follow. A reader can be shown exactly what changed between turns, in words. Compaction of a long chat cannot lose the state. And a reference such as "the year before" resolves against stored absolute dates, so it gets the same answer however long the conversation has run.

{
  "metric": { "id": "net_revenue", "authority": "owner_approved" },
  "filters": [{ "field": "region", "op": "=", "value": "EMEA" }],
  "period": { "start": "2026-07-01", "end": "2026-09-30", "read_as": "last quarter" },
  "grain": ["month"],
  "sources": ["warehouse.fct_invoices"]
}

// Turn 4: "what about the year before"
{ "op": "add_comparison",
  "period": { "start": "2025-07-01", "end": "2025-09-30" } }

Where state lives, and what follows from it

Platforms offer both models. In a stateful design the platform stores and manages history in threads. In a stateless one your application stores history and sends it with each request. The choice affects more than convenience.

• Platform-held stateQuicker to build, with deletion and retention following the platform’s controls. Your frame, if you keep one, becomes a second source of truth to reconcile with it.
• Application-held stateYou own retention, the frame and portability across sources, including answering across two systems. You also own encryption, access checks on stored conversations and the bill for resending context.
• In either caseA stored conversation contains query results. Treat it as data of the highest sensitivity it has touched, and decide who can read it, including support staff.

Permissions apply per turn

A conversation can outlive a permission. Say someone loses access to a region while a chat stays open. The next follow-up must run under their current access, so it has to re-execute. Serving turn five by filtering turn one’s cached result is tempting for latency and wrong for security, and any cache key that omits identity leaks to the next person.

Sharing raises the same issue. When a person shares a conversation, the recipient should either see a static record that was permitted for the sharer to send, or a re-run under their own identity, and the interface must say which. Re-running can legitimately return different numbers. I think that’s correct behavior, and the design task is to explain it.

This also applies when analytics is reached over MCP from a general assistant. The MCP specification keeps the full conversation with the host and away from the server, so the server sees only tool arguments. That favors passing the frame explicitly, as arguments or as a server-minted handle. The specification says a handle is not proof of permission, and that the server should check the caller against it on every call.

When to ask, and when to answer

Conversation gives you something a dashboard lacks: the ability to ask. Use it sparingly, because each clarifying question is a cost, and people abandon systems that interrogate them. Research suggests the problem is real. The AMBROSIA benchmark, built around scope ambiguity, attachment ambiguity and vagueness in database questions, reports that even advanced models struggle to identify and interpret ambiguity. So detection should not rest entirely on the model noticing. A catalog that knows a measure has two approved meanings can trigger the question deterministically.

My rule: ask only if the plausible readings would give materially different answers and context can’t settle it, and name the options when you do. "Which revenue: invoiced or recognized?" takes a second. "Can you clarify?" sends the work back. With one approved meaning, answer and state it. For vagueness such as "top customers", pick a reasonable reading, say so, and make it easy to change.

In an MCP setting, the July 28, 2026 revision gives this a defined shape. A server can return an input_required result carrying an elicitation request, and the client retries the original call with the person’s response. Servers must not send an elicitation to a client that has not declared support for it, so keep a fallback that returns the choices as an ordinary result the assistant can relay. The specification also tells servers to treat any returned requestState as attacker-controlled, and to protect its integrity if it affects access or logic.

A why question is a different request

Follow-ups fall into two groups, and your architecture should treat them differently. Refinements such as a new filter or period edit the frame, and the result is a number you can check. Explanations such as "why is that down" have no single correct query. The system can only break the figure down by available dimensions and report which ones moved. Stating that general business insights are out of scope is a sensible boundary for a product to draw.

If you do offer explanation, run it as a separate, labeled analysis. Show the breakdowns tried and the rows behind each claim, and word the finding as an association the data shows. A model will happily write a causal story from two rows, and in a conversation that story then becomes context for the next turn.

Latency: give each phase its own signal

A follow-up feels like it should be instant, and a governed one may not be: look up context, propose, check, execute, verify, write. Streaming the words at the end does not help when the wait is in the warehouse. Show progress by phase, and show the frame change early, since it arrives well before the rows. That has a practical payoff. A person who sees "Compared with Jul to Sep 2025" and meant the prior quarter can correct it before paying for the scan. Then make cancellation real, so that abandoning a follow-up stops the query.

Test conversations, because single questions will pass

Question-level accuracy can look fine while dialogues fall apart. Write scripted conversations with an expected frame after every turn, and grade the frame as well as the result. These cases find most of the defects:

• Carry-overAdd a filter, ask a follow-up, and confirm the filter is still there. Then remove it in words and confirm it is gone.
• ReconciliationAsk for a total, then a breakdown. The parts must sum to the total, or the answer must explain why they do not.
• Topic changeSwitch to an unrelated question and confirm the old filters do not leak into it. This is the most common error in transcript-based designs, and nothing flags it.
• Relative datesSay "the quarter before that" after two previous shifts, and check the absolute dates.
• Revoked accessRemove a permission between turns in a test environment, and confirm the next turn reflects it.
• Second identityRun the same script as a person with narrower access, and read the entire response for leakage.

How we handle the turn

In Nexatron, "Read as" shows how the words were read, and you can edit it and Re-run. A measure with several meanings that give different answers gets one question naming them, and a single approved meaning gets an answer with no question. The query runs under the asker’s own identity, and the answer shows Answered as and Why you can see this. Shared, scheduled and embedded answers stand only on approved, customer or standard definitions and catalog relationships, and otherwise decline with a reason, while interactive answers may interpret and say so. Those scripted conversations make a fair evaluation of us, and of anyone else.