Where AI actually fits in the data pipeline

Every data vendor now has “AI” in the headline. Almost none of them tell you which part of your pipeline it will actually change. After a year of shipping LLM features into real dbt projects and real warehouses, here is my scorecard.

How much AI actually helps at each stage

Ingest · today

Connector generation from API docs works. Schema drift handling is still brittle and you still babysit auth.

Scores are my judgement from shipping this stuff, not a survey. Tap a stage to read why.

Ingest

Connector generation from docs works. Schema drift and auth still need a human, so this stage scores low and stays low.

Transform

The honest win. SQL from a description, refactors across a DAG, explanations of legacy models. Code has a compiler to check the answer.

Quality

Anomaly detection is good. Root-causing a bad number across lineage is still mostly you.

Semantics

Defining metrics with a model in the loop helps. Governing them is politics, and models do not help with politics.

Serve

Text-to-SQL demos well and fails on real schemas. Without a semantic layer the model guesses which revenue column is real.

Consume

Noticing what changed and telling the right person in a sentence. Not a SQL problem, and models are already good at it.

Eighteen months out

The layers that gain most are the ones with a contract to check against: transform, quality, and consumption. Serve only catches up once a semantic layer exists.

Your turn

Tap any stage and flip between today and eighteen months. Scores are my judgement from shipping this, not a survey.

The pattern

Look at the shape. AI helps most where two things are true at once:

The input is already structured text. SQL, YAML, dbt models, metric definitions. Models are good at these because they are code, and code has a compiler to check the answer.
The output can be reviewed as a diff. A proposed model is a pull request. A proposed metric is a YAML change. A human can approve it in thirty seconds.

It helps least where the input is a flaky third-party API or a human’s unstated intent. Ingest and text-to-SQL both fail for the same reason: there is no contract to ground the model against.

Transform is the honest win

If you take one thing from this post: the transformation layer is where the leverage is right now, and it is not close. Refactoring a 400-model dbt project used to be a quarter of work. It is now a week of review. Explaining what a legacy model does, writing tests for it, and proposing the staging split all work reliably because the model can run the SQL and check.

-- what I actually ask for now
-- "split stg_orders into stg_orders and int_orders_enriched,
--  keep the grain, add not_null tests on the keys"
-- and then I read the PR.

Serve is the honest disappointment

Text-to-SQL demos are gorgeous and production text-to-SQL is a support ticket generator. Without a semantic layer underneath, the model guesses which of your four revenue columns is the real one. It will guess confidently. Your CFO will notice.

The fix is not a better model. It is a metric layer the model is forced to use. Once “revenue” has one definition and one owner, question answering becomes safe. That is why the semantic layer score rises fastest in the 18-month view.

Consume is where it gets interesting

The last stage is the one most teams ignore, and it is where I think the biggest behaviour change is coming. The question is not “can a model write the query” but “can a model notice what changed and tell the right person in a sentence.”

That is a summarisation and routing problem, not a SQL problem, and models are already good at it. The warehouse knows the answer, the model can phrase it, and the person just needs it to show up where they already are.

What to actually do

Put AI in your transform workflow first. It pays back immediately.
Build or buy a semantic layer before you buy a chat interface.
Measure consumption by whether decisions changed, not by dashboard views.