This article covers six lessons from a year of building and shipping analytics products that use large language models for enterprises.

The lessons are simple:

  1. Start with the user’s workflow.
  2. Get the data model right.
  3. Define how answers will be tested.
  4. Build security into every layer.
  5. Make every answer easy to inspect.
  6. Plan for model and vendor changes from the start.

1. Start From Workflows, Not From the Model

Do not start with the model. Start with the person who will use the product and the moment when they will use it.

A finance lead may use it during month-end close. A risk analyst may use it during portfolio reviews. An operations team may use it during daily monitoring.

Write down the questions people ask again and again. Do not start by trying to answer every question they could ask.

Build a narrow first version around one user and one workflow. Use only the data needed for that workflow, then make the full path work end to end.

A useful test is this:

Can users ignore the system for a week without worrying that something important will break?

If not, the product is not ready to expand.

2. The Data Model Is the Real Prompt

Large language models are good with language. They cannot fix messy data by themselves.

The product needs a clear data model. It also needs clear rules for how data is prepared and used.

Keep raw data and curated data separate. Raw data is the source as received. Curated data is the version the product can use after the team has cleaned it.

Create data contracts that people can read. Each contract should define the metric and say what level of detail it uses. It should also state when the data refreshes. If the metric has tolerance rules, put those rules in the contract instead of leaving them in code or Slack threads.

Domain experts need to be involved early. They help design the data model. They help shape the prompts. They validate the expected answers. They also test edge cases that developers may miss.

When developers own prompts and business definitions by themselves, the result can look impressive and still be wrong.

3. Measure It Like Any Other Critical System

A product that uses a large language model still needs normal production measurement.

Four measures catch many problems:

  • Answer quality: Use a fixed set of questions and define what a good answer should contain.
  • Coverage: Track how many real user questions the system can answer without a handoff.
  • Layer-level checks: Test each layer separately. Start with the user interface and the ingestion path. Then test whether the generated SQL returns the expected data before it reaches the chart.
  • Cost per question: Track the model calls made for each question. Add token usage and cloud cost to the same log record.

The team should log every API call and every contract clearly. Use structured logs so people can search and compare them later.

Keep detailed traces. Store the prompt next to the response. Link both to the query that ran and to any validation result.

Use large language models as judges only when the task is about surface quality. Do not use them as the only judge for correctness.

4. Security and Safety Need Layers

Security cannot depend on one prompt or one filter. It needs checks at each layer.

Request Filtering

Check whether the request is too long or oddly shaped. Reject language the product should not accept. Block prompt injection attempts and any request that tries to send raw SQL or expose schema details. Add rate limits for repeated abuse.

Data Access and Model Context

Enforce role-based access control at the storage or query layer. Use read-only identities where possible. Limit how much data the system can pull for one question.

Build model context only from data and schema that the user is allowed to see.

Response Checking

Check that the response stays within the intended scope. It should not expose restricted data. It should also avoid schema details and internal instructions.

Add a deterministic gatekeeper step when the risk is high. This step should use fixed rules rather than another open-ended model response.

The system should fail safely. A safe failure is better than a confident answer that exposes data or invents facts.

5. Make It Inspectable So Users Can Trust It

Users should be able to check an answer in less than a minute.

For any chart, show how the answer was calculated. If the answer came from a query, let the user inspect that query and the filters applied to it. Let users move from a written answer to the source rows or documents.

Let them change the slice themselves instead of treating the assistant as the only way to explore the data.

Make it easy to report a bad answer. The report should capture the original question and the answer the user saw. It should also link to the source data and trace details.

Users should see how the answer was produced. Show the work. Cite the source. Expect the user to question the result.

6. Treat Vendors, Model Versions, and Failures as Part of the Design

The model or vendor will change over time. Plan for that from the start.

One demo failure led us to make three changes.

First, we added graceful fallback paths. When the model was slow or unavailable, the system could show a cached answer. For repeatable reports, it could use a template instead of waiting on the model.

Second, we added a light provider abstraction. This gave us enough separation to switch models without rewriting the product.

Third, we treated model upgrades like production changes. Each upgrade needed a rollout plan. Before rollout, we ran regression checks and kept a revert path ready.

One simple test helps. Replace the current model with the smaller version from the same provider.

If the product still works reasonably well, the design is more likely to hold up in an enterprise setting. If it breaks badly, the product is too dependent on one model’s behavior.

Closing Thought

The basics do not change much across tools or vendors.

Start with the workflow. Get the data model right before you tune prompts. Bring domain experts in while the definitions are still being set. Treat the product like production software, with tests and logs that show where it failed.

When those things are done well, the fact that the product uses a large language model stops being the main point. It becomes one line in the architecture.

For serious enterprise systems, that is where it belongs.