← All ADE-bench runs
dbt Labs’ ADE-bench · Run
Dex makes agents better analytics engineers.
Accuracy on ADE-benchDex + Claude Sonnet 5
76.0%
57 / 75 tasks resolved
Results
Each run is one attempt per task across the full 75-task suite.
| Run | Resolved | Accuracy | Cost |
|---|---|---|---|
| Dex + Claude Sonnet 5 | 57 / 75 | 76.0% | $35.95 |
| Dex + Claude Fable 5 | 56 / 75 | 74.7% | $91.98 |
| Dex + Claude Opus 4.8 | 54 / 75 | 72.0% | $43.38 |
| Claude Sonnet 5 (no Dex) | 53 / 75 | 70.7% | $30.76 |
Dex + Sonnet 5 leads at 76%, for about 2.5x less than Fable 5 and ~17% less than Opus 4.8. With Dex, accuracy holds in a 72–76% band across all three models while cost ranges from $36 to $92, so the practical call is to run an inexpensive model.
For context, dbt's published agent skills reported 58% on this benchmark with Opus 4.6.
How we ran it
ADE-bench hands an agent a dbt project on DuckDB and asks it to fix a broken model, build a new one, or extend the semantic layer, then scores whether the project's tests pass.
Run record
- Tasks
- 75 tasks across 8 domains (airbnb, f1, asana, intercom, quickbooks, helixops_saas, analytics_engineering)
- Agent
- Claude Code, one attempt per task, up to 50 episodes
- Dex
- Supplied as the exmergo/dex skill plugin
- Baseline
- Identical setup with no plugin
Reading the numbers
- These are single-run results (one attempt per task), so treat small gaps between runs as noise.
- The raw results.json for every run is committed under experiments/, alongside the harness configuration.