Let's get something out of the way first.
Having data is good. Having a database full of reviews, commits, and org activity sitting there quietly, untouched, unread, never once glanced at by a human being with a coffee and an opinion? That's not "having data." That's a very expensive data graveyard.
At LiveReview, we build what we call a Blast-Radius Aware AI Code Review for Business-Critical Systems.
Which is a fancy way of saying: we review your code, we figure out how bad it would be if a change goes wrong, and we don't shut up about it until someone fixes it.
Along the way we accumulate a review data: who reviewed, how much, how fast, how often, which repos are on fire.
And for a while, that pile just sat there.
Engineering leaders would ask "is adoption increasing?" and get back a vibe, not an answer.
So we built Livi, a chat bot that answers real questions about that data with real charts, not paragraphs of hedging.
This post technically about how Livi draws those charts.
Specifically: why we never let the LLM touch a pixel, how the same chart definition ends up as both a live interactive graph in your browser and a flat PNG in a Slack thread, and why teaching a language model to pick the right chart shape is a surprisingly deep rabbit hole.
The core decision: don't ask the LLM to draw, ask it to describe
The tempting, wrong idea is: "let's have the LLM generate an image." Please don't.
Image-generating models are a different beast entirely, and even if you got one to draw a bar chart, you'd have no way to verify the numbers on it are real.
You'd be trusting a model that hallucinates plausible-sounding review counts to also render them faithfully into pixels.
That's not a chart, that's chart-shaped fan fiction.
The actually good idea, and the one every serious LLM-charting integration eventually converges on, is: the LLM writes Vega-Lite, a JSON grammar for describing charts declaratively.
You don't say "draw a blue bar going up." You say:
{
"mark": "bar",
"encoding": {
"x": { "field": "month", "type": "temporal" },
"y": { "field": "review_count", "type": "quantitative" }
}
}
That's it.
That's the entire chart.
No pixels, no drawing, just a description of what the data means and how it should be mapped to a picture.
Vega-Lite does the actual drawing.
The LLM's job shrinks down to something it's genuinely good at: filling in a well-defined schema.
Models are much better at "pick mark: bar or mark: line" than "hallucinate 600 pixels of a correct y-axis."
And critically: the LLM never sees the actual numbers before they're rendered.
It writes the SQL, we run it, and the real result set gets stitched into data.values by our own Go code.
The model can be as creative as it wants about presentation.
It gets zero creative license over the numbers.
What actually happens between "how many reviews last month" and a chart on your screen
Here's the pipeline, roughly:
A few things worth dwelling on here, because each one exists because something went wrong first.
Why two SQL-writing steps instead of one? Because the model needs to know how many rows the answer will have before it can decide whether a chart makes sense or whether it should hand you a CSV instead.
Nobody wants a bar chart with 4,000 bars.
So step one is basically "how big is this going to be," and step two is "okay, now actually get me the data and tell me how to draw it."
Why is there a SQL guard at all? Because an LLM writing raw SQL against a multi-tenant database is one confidently-worded prompt away from org_id = 1 OR 1 = 1.
We run every generated query through a guard that rejects anything that isn't read-only, checks every table against a denylist, and specifically looks for the shape of a tenant-isolation bypass (constant-vs-constant comparisons, bare OR TRUE, all the classics).
It's not glamorous work, but it's the difference between "cool AI feature" and "why is Org A looking at Org B's review data" showing up in an incident channel.
(meme placeholder: "Well Yes, But Actually No" / bike fall guy. Top text: "the query has an
org_idfilter." Bottom text (mid-fall): "WHERE org_id = 1 OR 1=1")
Why does the LLM only ever see a narrowed slice of the schema, not the whole database?
Because our actual schema has north of fifty tables, and dumping all of them into every prompt is both expensive and a great way to get the model confused about which created_at belongs to which table. More on this in a second, it deserves its own section.
Teaching the model which tables even exist: dbctx
Here's a problem that doesn't show up until your schema stops being a toy demo.
We're at 58 tables and counting: reviews, pull requests, AI comments, review feedback, billing, licensing, job queues, the works.
If we pasted the full schema into every single prompt, we'd be burning thousands of tokens per question just describing license_seat_assignments to a model that was asked "how many reviews happened last month" and does not, will never, care.
So instead of "here is the entire database, good luck," we use dbctx, a Go library built specifically for this problem: given a natural-language question and a live Postgres connection, hand back only the tables that actually matter, formatted as compact, LLM-friendly text instead of a raw information_schema dump.
dbctx
Compile a PostgreSQL database into compact, queryable context.
dbctx is a Go library and CLI tool that compiles a PostgreSQL database into a portable, queryable context index (.dtx file). It extracts schema, relationships, field semantics, representative values, JSONB structure, and builds a full-text search index — all from deterministic introspection, statistics, and heuristics, with no generative LLM and no external services required for the core index. It also supports an optional local semantic embedding signal and an optional, user-controlled terminology dictionary — both additive, both off-by-default-cost, described below.
Use it to give text-to-SQL systems, AI agents, and database-aware applications a compact, relevant slice of your database schema at query time, instead of dumping the entire information_schema into every prompt.
Longer writeup with real numbers and diagrams: Introducing dbctx.
Key features:
- Natural-language query — find relevant tables, columns, and relationships from a text question
- Semantic retrieval (optional, on…
It does this with a genuinely layered retrieval pipeline, not just a keyword grep.
Lexical and fuzzy matching against table and column names, full-text search, matching against actual sampled values in the data, an optional semantic embedding pass, and a curated terminology layer we feed it ourselves (so "LOC" resolves to billable_loc, and "MR" and "PR" both resolve to the same underlying pull-request concept, because our users say both depending on which Git host they came from).
Whatever scores above zero gets pulled in, plus anything reachable through a foreign key from something that scored, because a join target with zero lexical overlap with the question is still often exactly what the query needs.
The payoff is not subtle.
On our real schema, a typical question narrows 58 tables down to somewhere around 25 to 30, which is roughly half the context gone before the model has written a single character of SQL.
That's not a rounding-error optimization, that's the difference between a prompt the model can actually reason clearly about and one where the important tables are buried in a wall of billing and license-seat noise.
The part where pixels finally show up: vl-convert
So now we have a Vega-Lite spec. Great.
If the user is on the web dashboard, we're basically done, more on that in a second.
But what about Slack? What about Discord? Those platforms don't run a JavaScript charting library inside a chat message.
A Slack message is not a browser tab.
You cannot politely ask Slack to interpret a Vega-Lite spec and render SVG for you.
So for anywhere that isn't our own frontend, we need an actual image file.
This is where vl-convert comes in: a Rust binary (with Python and Node bindings, but we shell out to the CLI) that takes a Vega-Lite spec and rasterizes it straight to PNG, no headless browser required.
That last part matters more than it sounds.
The old-and-busted way to render a chart server-side is to spin up a headless Chrome instance, load a page with a charting library, screenshot it, and pray your Docker image doesn't balloon to two gigabytes.
vl-convert skips all of that.
It's a single binary, it takes JSON in, it gives PNG bytes out, and it's fast enough that nobody notices it happening.
(meme placeholder: Kombucha Girl, disgusted-then-intrigued two-panel. Disgusted panel: "spin up headless Chrome to screenshot a chart." Intrigued panel: "one Rust binary, JSON in, PNG out.")
Same spec, two very different destinies
Here's the part I actually think is neat.
We generate one Vega-Lite spec per chart.
What happens to it next depends entirely on where it's going.
(meme placeholder: Trade Offer / Minecraft villager trading. Give: "one Vega-Lite spec." Take: "a live interactive chart in the browser, or a flat PNG in a Slack thread, depending on where it lands.")
On the web, the frontend just hands the raw spec to react-vega and lets the browser do the work.
You get hover tooltips, you get resizing, you get an actually interactive chart, and our backend does zero image rendering for that path.
It just ships JSON.
For Slack and Discord, the exact same spec gets routed through vl-convert instead, turned into a flat PNG, and attached as a file to the message.
The bot doesn't know or care that it's the "same" chart taking a different road.
From the pipeline's point of view, a Vega-Lite spec is just a Vega-Lite spec.
Where it ends up decides whether it becomes living, breathing SVG or a JPEG-adjacent screenshot sitting quietly in a chat thread forever.
This split is also why we can add new chart destinations cheaply.
Want an emailed weekly digest with embedded charts? Same spec, same vl-convert path, new delivery mechanism.
The hard problem (getting a correct, sensible chart spec out of an LLM) is solved exactly once.
Where this leaves us
The whole point of Livi was never "add a chatbot," it was "close the loop between the data we're already collecting and the person who actually needs to act on it."
A CTO asking "are engineers actually using this thing" should get an answer that looks like a calendar heatmap of usage rhythm, not a spreadsheet and a shrug.
The recipe, if you're building something similar, is genuinely not complicated:
- Never let the model touch pixels. Let it write a declarative spec.
- Never let the model see real numbers before you've run its query yourself.
- Guard the query like you mean it, not like a vibes-based regex.
- Pick one rendering pipeline (
vl-convert, in our case) and let destination decide static-vs-interactive, not the model. - Budget real prompt space for chart taste. This is the part that actually takes iteration.
LiveReview reviews your code, tells you how bad a change could go, and won't shut up until it's fixed — and Livi turns all that accumulated review data into charts a human can actually act on.
If you liked the post, drop a ⭐ on the repo and try LiveReview now.
Top comments (5)
One thing that would take a lot of pressure off that SQL guard: run the generated query as a Postgres role with row level security on, scoped to the org. Then
OR 1=1still comes back with only that tenant's rows, and the guard gets to be a second layer instead of the only thing standing between orgs.This is a really clean separation of concerns — the LLM handles schema/shape decisions (what it's actually good at), while Vega-Lite handles rendering (what it's actually good at). The SQL guard for tenant isolation is the quiet hero here; it's the kind of thing that only gets noticed when it's missing.
I've been running into a similar design pattern with memory retrieval in agent systems: the model shouldn't "draw" the memory graph from scratch either — it should describe what it needs, and a separate layer does the actual lookup and stitching. Keeping the model in "describe, don't compute" mode seems to generalize well beyond charting.
The two-SQL-pass approach (size estimation before full query) is also a nice touch — essentially a cost/shape pre-check before committing to a full render. Did you find the overhead from the extra round-trip worth it in practice, or does it occasionally mispredict and still return an awkwardly large result?
The chart-spec boundary is the right kind of boring. It turns the model into a translator instead of a renderer, and it gives reviewers an object they can diff. I would be most nervous about silent query broadening, so the tenant guard is doing more product work than the chart layer.
the 'LLM writes
Vega-Lite, real data gets stitched in by Go code' is the only architecture that's safe here. letting the model see the numbers AND decide how to present them is where you get charts that look right and have plausible sounding values that don't match the data.we've been doing something similar: the model writes the query logic and schema bindings, a deterministic layer runs the query and injects results. the model has no creative license over what gets plotted, only over how.
the chart shape selection problem is legitimately hard though. do you handle the case where the right answer to 'show me adoption over time' is actually two charts, not one?
the line i keep coming back to is that the model can be as creative as it wants about
presentation and gets zero creative license over the numbers. thats the right cut and i want to
push on where you drew it, because i think the boundary is hard on one side and soft on the other.
charts have basically never lied by getting the numbers wrong. they lie through encoding. a bar
mark with a truncated y baseline, a temporal type on a field thats actually irregular so the
spacing implies a cadence that isnt there, the wrong aggregation quietly turning a sum into a
mean. every one of those produces a picture that misrepresents data that is completely real. and
encoding is exactly the surface you handed the model full creative license over.
your point 5 sort of names this as chart taste and budgets prompt space for it, which reads like a
quality problem. i think its a correctness problem wearing quality clothes. real numbers do not
make a chart true.
the fix is a move you already made one layer down. you dont trust the model with the numbers
because go stitches data.values in after the fact, outside the models reach. thats structural
exclusion, not validation. you could do the same to the spec. once the result set exists you know
the domain, the cardinality and the field types, so a deterministic pass can reject or rewrite a
spec that contradicts them. bar mark with a quantitative y whose scale domain excludes zero,
temporal encoding on a field with non uniform intervals, aggregate that doesnt match what the sql
actually computed. same shape as your sql guard, pointed at the encoding instead.
on the sql guard itself, one thing worth naming. you describe it as looking for the shape of a
tenant bypass, constant vs constant, bare OR TRUE, all the classics. thats an enumeration of known
bad, and enumeration is the weak form even when the list is good, because it only ever covers what
somebody already thought of.
the structural version is to not let the model write the tenant predicate at all. either parse and
inject org_id yourself on every table reference after generation, or run the query under a
postgres role with row level security so isolation is enforced by the database regardless of what
the sql says. then a bypass isnt caught, its unrepresentable.
and that is the same argument as the first half. you have structural exclusion for the numbers and
validation for the tenant boundary, in the same system, one layer apart. the numbers side is the
stronger pattern and its already yours.
small one, and probably fine at your stakes. the two step sql means the size estimate and the
actual fetch are separate reads, so rows can move between them and the chart versus csv decision
gets made on a count that is already stale. one repeatable read transaction closes it if it ever
starts mattering.