A DAG at 50, 500 and 5,000 models

A dbt project does not fail all at once. It fails in stages, and each stage has its own shape. I generated three projects at 50, 500 and 5,000 models with the same layer structure and let the pathologies grow with size. None of it is a real project. The shapes are the ones I keep meeting.

What the graphs are

Everything above comes from a seeded generator, not a customer warehouse. Sources feed staging one to one, staging feeds intermediates, intermediates feed marts, and fan-in grows with the log of project size. At 500 and 5,000 the generator also adds the things I have watched happen in real projects: one intermediate that everything joins to, a long chain of sequential revenue models, marts nobody reads, and marts that are copies of other marts with _v2 or _final on the end. Build times per model are random too, so the run estimates are a model of a model.

Fifty: everything is visible

At fifty models the DAG is a picture you can hold in your head. One person owns it. The whole run takes minutes, so nobody thinks about run time, and when a number is wrong you read four files and find it. The only real risk at this size is that the habits you form here do not survive growth. Nobody writes tests at fifty models because they can see everything. That is exactly when tests are cheapest to add.

Five hundred: ownership breaks first

Somewhere between 200 and 500 models, one intermediate becomes the thing everyone joins to. In the generated project it is int_customers_enriched, and 212 of 500 models sit downstream of it. That is not a design decision anyone made. It is the path of least resistance: the model already has the columns you need, so you ref it.

The problem is not the fan-out. It is that the god model now has a dozen implicit owners and no explicit one. Every change to it is a change to half the warehouse, so nobody changes it, so it accumulates columns instead. Run time is still fine at this scale. Ownership is what breaks.

Semantics start to slip here too. Two teams add revenue columns to two marts with two definitions, and both are downstream of the same god model, so both look authoritative.

Five thousand: run time and semantics

At 5,000 models the picture stops being a picture. In the generated project there are 11,558 edges and a critical path of 29 models. That path takes about 48 minutes to build in sequence, and no thread count removes it. With eight threads the estimate for the full run is closer to nine hours, which is why teams at this size stop doing full refreshes and start arguing about what “the nightly run” even means.

The orphans and copies are the other cost. 195 generated models that nothing reads and 80 that are copies of another mart. They are not free: they run, they get tested, they show up in lineage, and every one of them is a place a new hire can ref the wrong thing. The _final suffix is the tell. Nobody deletes the original because they are not sure who reads it.

What actually helps

I have three refactors that I think earn their cost, in this order.

Layer contracts. Make the intermediate layer the only thing marts can ref, and make staging the only thing intermediates can ref from sources. Enforce it with a test on the manifest, not a wiki page. This does nothing for run time. It does everything for the god model, because the god model can only exist when marts reach past it into whatever they want.

Selectors that match how you deploy. A 5,000 model project should never run as one graph. Define selectors per domain and per cadence, run the critical path on its own schedule, and treat dbt build with no selector as a mistake. The critical path in the demo is the revenue chain. That chain wants its own job, its own owner, and its own alert.

Exposures for anything that is read. If a mart has no exposure, it has no reader on record, and after a quarter it is a deletion candidate. This is how you find the orphans without a meeting. It also gives the copies a reason to die: when fct_orders_final has an exposure and fct_orders does not, you know which one to keep.

What to do this quarter

• Count direct dependents per model in your manifest. Anything over ten is a god model in waiting. Give it an owner and a column contract.
• Find your critical path by summed run time, not depth. Put it on its own job with its own alert.
• Add exposures for every dashboard and export you know about. Then list the marts with none and start deleting.
• Grep for _v2, _final, _new and _old in model names. Each one is a decision someone deferred. Make it.
• If you are under 100 models, add the layer test now. It costs an hour and it is the one thing that keeps the 500 model version from happening.