The short answer. NLSQL returns the same number to every user asking the same question because the language model never calculates anything. It translates the question into SQL and a sentence template, and the number is computed by the customer's own database. Metric definitions are stored as fixed SQL expressions, so "revenue" resolves to the same expression for everyone, and a validator discards any answer containing a digit that was not in the question.
Most AI data analytics tools work the opposite way: a model authors fresh analysis code on each request. Nothing pins the definition, so the same question can produce a different calculation - and a different number - on the next run.
Two analysts, one question, two numbers
A sales director asks the company's AI data analytics assistant what revenue was last quarter. A finance analyst asks the same thing an hour later. The answers differ by a few percent. Nobody gets an error. Both numbers look completely plausible.
This is worse than an outright failure. A broken query announces itself; two credible numbers do not. The meeting proceeds, both people trust their own screen, and the discrepancy surfaces weeks later in a board pack - at which point nobody can reconstruct which query produced which figure.
For reporting, consistency is not a nice-to-have on top of accuracy. It is accuracy. A metric that changes depending on who asked, or when, cannot be governed, audited, or put in front of a regulator.
Why LLM-based AI data analytics tools diverge
The current generation of AI data analytics tools - Hex's Magic, ChatGPT's Advanced Data Analysis, Gemini in BigQuery, Claude's analysis tool, and the many agentic notebooks built on the same idea - share one architectural decision: the model writes the analysis code, and usually runs it too.
That design is genuinely powerful for exploration. It is also the direct cause of the inconsistency, for reasons that have nothing to do with model quality:
- The calculation is re-invented per request. "Revenue last quarter" has to be turned into concrete choices every single time: which table is authoritative, whether to join to returns, whether refunds and cancellations are netted off, whether the quarter is calendar or fiscal, which timestamp column defines it, whether to convert currency at transaction or period-end rate. Each run makes those calls again.
- Nothing pins the definition. Without a stored metric expression, there is no artefact that says what revenue is at this company. The model's choice substitutes for a definition, and that choice is a function of phrasing, conversation history, and sampling.
- Every variant is valid. All of those queries parse, execute, and return a number. There is no error state to catch, so the disagreement is silent.
- Prose summarisation adds a second step that can drift. When a model reads a result table and writes "revenue grew strongly to about $4.1m", the figure has passed through a text-generation step. Rounding, mislabelling, or picking the wrong row of the result set all become possible.
- Free-form code means free-form scope. An agent that can write arbitrary SQL or Python can also touch tables the asker should not see, which turns a consistency problem into an access-control one.
None of this makes those products bad. It makes them the wrong shape for a number that appears in a report every month.
How NLSQL is built differently
NLSQL treats a business question as something to be compiled, not improvised. Five mechanisms enforce that, and they compose.
1. The model never produces a number
NLSQL's model output is a SQL SELECT plus a sentence template
containing a *{result}* placeholder - for example
"Total revenue for Germany last quarter was *{result}*." The figure is inserted
afterwards from the real query result.
This is enforced, not merely requested. A validator checks every digit sequence in the generated template against the user's original question; if the model emitted a number that was not in the question, the template is thrown away and replaced with a mechanical one. A fabricated figure has no path to the user.
2. NLSQL does not execute your analytical query
The backend returns the SQL and the template. Your own client, bot, or application runs the query inside your environment and fills the placeholder. The number is therefore produced by your database, from your data, under your permissions - and the underlying rows never have to leave your infrastructure to answer the question.
3. Metric definitions are pinned
This is the mechanism that actually delivers the same number to different people. A metric is configured once as an alias mapped to an exact SQL expression. When a question matches that alias, the engine is required to use that expression verbatim rather than composing its own. "Revenue", "active customer", "churn", "billable hours" stop being matters of interpretation and become definitions your finance team owns.
4. Identical questions can bypass generation entirely
Frequently asked questions can be pinned. NLSQL normalises the question - lowercases it, collapses whitespace, strips trailing punctuation - and hashes it with SHA-256. On an exact match it returns the pinned answer without calling a model at all. For those questions determinism is not probabilistic; there is no generation step to vary.
5. Every generated query is validated, then logged
Anything the model does produce passes three deterministic gates before it is returned:
the SQL is parsed and must be exactly one SELECT; every table,
column and join is checked against an allow-list built from your configured schema; and
string filter values are checked against the real values in your columns, so a query
cannot silently filter on a category that does not exist. Generation runs at temperature 0
- greedy decoding rather than sampling. Failures feed corrective detail back for a retry,
and if the query still cannot be validated, the request falls back to NLSQL's rule-based
engine, which contains no model at all.
Each query writes an audit row capturing the question, the generated SQL, the guardrail verdict and any retries. A number produced last quarter can be traced to the exact query that produced it.
Side-by-side comparison
| LLM notebook / agent tools Hex Magic, ChatGPT ADA, Gemini in BigQuery, Claude analysis |
NLSQL | |
|---|---|---|
| Who computes the number | Code the model wrote, executed by the tool; often summarised in prose afterwards | Your database, executing SQL that was validated before it was handed over |
| Metric definition | Re-derived per request from the prompt and context | Stored once as a fixed SQL expression and reused verbatim |
| Same question, different users | May compile to a different query, so the number can differ | Same metric expression for everyone; pinned questions skip generation entirely |
| Can a figure be invented | Possible wherever a model writes or restates numbers | Blocked - a template digit absent from the question causes the template to be discarded |
| Schema access | As broad as the connection and generated code allow | Restricted to an allow-list of tables, columns and permitted joins |
| Statement types allowed | Whatever the generated code contains | Exactly one SELECT; anything else is rejected |
| Filter values | Model may guess or "correct" a value | Checked against the real values present in your columns |
| Auditability | Varies; often a notebook cell or chat transcript | Audit row per query: question, SQL, guardrail verdict, retries |
| Best suited to | Open-ended exploration, ad-hoc analysis, visualisation, data science workflows | Repeatable business answers for non-technical staff, in Teams or Slack |
Test it yourself
Do not take a consistency claim on trust - including this one. The test takes ten minutes and works on any AI data analytics tool:
- Pick a metric that appears in a real report, ideally one with an edge case - refunds, fiscal periods, multi-currency.
- Ask the identical question in five separate, fresh sessions. Do not refine or follow up; start clean each time.
- Repeat from a second user account with different permissions.
- Record the number and the generated query or code for each run.
- Diff them. If the query text changes between runs, the number can change too, and any consistency you observed was luck rather than architecture.
With NLSQL there are two extra things to inspect: the stored metric expression, which tells you exactly what the definition is, and the audit log, which lets you confirm after the fact that all five runs used it.
Where NLSQL can still vary - and what bounds it
An honest account of the limits, because a claim without limits is marketing rather than engineering.
Two differently phrased questions that mean the same thing will not always compile to byte-identical SQL. If a question maps to a pinned metric, the metric expression is fixed and the number is the same. If it does not - a genuinely novel, ad-hoc question - the generated SQL is bounded by the guardrails but its exact shape can differ, and two differently worded questions may resolve a date range or a grouping differently.
What that means in practice: the guarantee is strongest exactly where it matters most, on the recurring, governed metrics that go into reports. The controls that keep the rest in bounds are the metric layer, pinned answers for known questions, the allow-list and literal validation, temperature 0, and the rule-based fallback. And regardless of path, the hallucinated-number guarantee never depends on the model behaving: the number always comes from your database, and the digit validator always runs.
Which tool should you use?
These are complementary, not interchangeable, and the honest recommendation is often both.
Use an agentic notebook - Hex, ChatGPT's data analysis, Gemini, Claude - when a skilled analyst is exploring, the questions are one-off, the output is a chart or a hypothesis, and a human reviews the code. Freedom is the feature there, and these tools are very good at it.
Use NLSQL when a non-technical colleague needs a governed number in Teams or Slack, the same question will be asked every month, the answer will land in a report, and someone may later have to prove where it came from.
Frequently asked questions
Why do AI data analytics tools give different answers to the same question?
Because most of them ask a language model to author fresh analysis code on every request. Without a fixed definition of each metric, the same question can compile into a different join, filter, date boundary or de-duplication rule each time. Each version is valid code, so nothing errors - the numbers simply disagree.
Does NLSQL give the same number to different users asking the same question?
Yes. Metrics are stored as fixed SQL expressions, so a question that matches a metric alias always compiles to that exact expression regardless of who asks. Pinned answers match on a SHA-256 hash of the normalised question and return the same answer without calling a model at all, and generation runs at temperature 0.
Can NLSQL hallucinate a number?
No. The model never produces figures. It writes SQL plus a sentence template containing a
*{result}* placeholder, and a validator rejects any template
containing a digit that is not literally present in the user's question. The number is
computed by the customer's own database and inserted into the placeholder afterwards.
Does NLSQL execute the SQL it generates?
No. The NLSQL backend never executes the analytical query. It returns the SQL and an answer template, and your own client or bot runs the query inside your environment and fills in the result. The data never has to leave your infrastructure for the number to be produced.
How is NLSQL different from Hex, ChatGPT Advanced Data Analysis or Gemini in BigQuery?
Those tools are agentic analysis environments: a model writes and runs code, then describes the output in prose. They are strong for open-ended exploration. NLSQL is built for repeatable business answers instead - it constrains generation to an allow-listed schema with pinned metric definitions and validates every query before it is returned, trading exploratory freedom for reproducibility.
How can I verify that an AI data analytics tool is consistent?
Ask the identical question in five separate sessions, ideally from different user accounts, and compare both the number and the generated query. If the query text changes between runs, the number can change too. Consistency you cannot reproduce on demand is not consistency.
Is every NLSQL query auditable?
Yes. Each query handled by the LLM engine writes an audit row recording the question, the generated SQL, the guardrail verdict and any retries, so a number produced months ago can be traced back to the exact query that produced it.
NLSQL Product Family
Deterministic AI data analytics is the foundation the rest of the NLSQL platform builds on:
- NLSQL: Core natural-language-to-SQL platform for querying enterprise databases conversationally.
- NLSQL AI Agent: Adds document intelligence for querying PDFs, SharePoint, and unstructured knowledge.
- NLSQL AI Employee: Workforce automation assistant for HR, IT, and operations.
- NLSQL AI Anomaly Detection: Proactive email alerts for business-data anomalies.
- NLSQL Web App Builder AI: Enterprise vibe-coding - ship production apps inside Azure.
Get the same number every time - 30 days free
Deploy NLSQL into your own Azure environment and point it at your database. No credit card required for the trial, and your data stays inside your tenant.