ADE-bench
dbt Labs' 75-task analytics-engineering benchmark, scored on whether the dbt project's tests actually pass.
- Analytics Engineering Benchmark
- Published by dbt Labs
- 75 tasks
- 2 published runs
What ADE-bench measures
ADE-bench hands an agent a real dbt project running on DuckDB and asks it to fix a broken model, build a new one, or extend the semantic layer. Each of the 75 tasks is graded by running the project's own tests afterwards, so the benchmark measures completed engineering work rather than plausible-looking SQL. Tasks span eight domains, and there is no partial credit: a task counts only when the resulting project genuinely builds and passes.
60 / 75 tasks resolved
Nine tasks the baseline failed became passes, and four moved the other way, for five more tasks solved with the same model. The gains show up in the benchmark’s checks of the finished work: identifiers preserved, a real model version shipped, a feature retired completely, and tests that catch the boundary.
Measured against the same Claude Sonnet 5.5 with no plugin, on identical prompts for all 75 tasks. The earlier 76% result on this suite used Claude Sonnet 5, so the difference between the two runs is not Dex’s alone.
Run history
Every published run, newest first. Each is a permanent record.
| Run | Best result | Accuracy | Resolved |
|---|---|---|---|
| 5 November 2026 | Dex + Claude Sonnet 5.5 | 80.0% | 60 / 75 |
| 21 July 2026 | Dex + Claude Sonnet 5 | 76.0% | 57 / 75 |
About ADE-bench
Q01What is ADE-bench?
ADE-bench is dbt Labs' Analytics Engineering Benchmark: a 75-task suite that hands an agent a real dbt project running on DuckDB and asks it to fix a broken model, build a new one, or extend the semantic layer. The agent's work is then scored on whether the project's tests actually pass, so it measures completed engineering work rather than plausible-looking SQL.
Q02Who publishes ADE-bench?
dbt Labs publishes and maintains ADE-bench as an open benchmark at github.com/dbt-labs/ade-bench. It is a third-party suite, not one Exmergo authored, which is what makes results on it comparable across tools.
Q03How is ADE-bench scored?
Each of the 75 tasks is graded pass or fail by running the dbt project's own tests after the agent finishes. Accuracy is the share of tasks resolved across the full suite. There is no partial credit, so a task counts only when the resulting project genuinely builds and passes.
Q04How does Dex perform on ADE-bench?
In Exmergo's 5 November 2026 run, Dex with Claude Sonnet 5.5 solved 60 of 75 tasks (80%), against 55 of 75 (73.3%) for the same model without Dex: 6.7 percentage points higher and 25% fewer failed tasks, with one attempt per task in each configuration. The earlier 21 July 2026 run measured 76% with Claude Sonnet 5. For context, dbt's own published agent skills reported 58% on this benchmark with Opus 4.6.
Q05Does adding Dex actually change the result, or is it the model?
Both runs measured it against the same model with no plugin. On 5 November 2026, Claude Sonnet 5.5 solved 60 of 75 tasks with Dex against 55 without: nine tasks moved from fail to pass and four the other way, on identical prompts. On 21 July 2026, Claude Sonnet 5 resolved 57 with Dex against 53 without, and with Dex accuracy held in a 72-76% band across three models while cost ranged from $36 to $92. The comparison is of the complete Dex configuration, including its workflow prompt, and single runs do not establish statistical significance on their own.