Benchmarks
How Dex performs on public analytics-engineering benchmarks. Every run is published under its own dated page, including the baseline it was compared against, so the numbers can be checked rather than taken on trust.
80.0%
With the same Claude Sonnet 5.5 model, Dex solves 60 of 75 ADE-bench tasks (80.0%) against 55 without it (73.3%): 6.7 percentage points higher, and 25% fewer failed tasks.
Measured against the same Claude Sonnet 5.5 with no plugin, on identical prompts for all 75 tasks. The earlier 76% result on this suite used Claude Sonnet 5, so the difference between the two runs is not Dex’s alone.
Benchmarks we run against
Public suites published by third parties, not internal test sets.
ADE-bench →
dbt Labs' 75-task analytics-engineering benchmark, scored on whether the dbt project's tests actually pass.
75 tasks · 2 published runs
data-eng-bench →
Snowflake's 103-task data-engineering benchmark, scored pass/fail by whether the dbt project's own pytest suite runs clean.
103 tasks · 1 published run
All runs
Newest first. Each run is a permanent record: a re-run gets its own page rather than editing an old one.
60 / 75 resolved
With the same Claude Sonnet 5.5 model, Dex solves 60 of 75 ADE-bench tasks (80.0%) against 55 without it (73.3%): 6.7 percentage points higher, and 25% fewer failed tasks.
Read the run →59 / 103 resolved
Dex + Claude Sonnet 5 resolves 59 of 103 tasks (57.3%) on Snowflake's data-eng-bench at k=1, the highest published figure for any Sonnet 5 configuration, and passes 94.7% of the verifiers' 2,850 individual assertions.
Read the run →57 / 75 resolved
On dbt Labs' 75-task analytics-engineering benchmark, Dex + Claude Sonnet 5 resolves more tasks than any other run we measured, and beats dbt's own published agent skills.
Read the run →How we benchmark
Q01Why publish benchmark results at all?
Claims about agent quality are cheap; measured results on a public suite are not. Dex is run against third-party analytics-engineering benchmarks and every run is published, including the baseline it is compared against, so the numbers can be checked rather than taken on trust.
Q02Why is every run published under its own date?
A benchmark result is only meaningful alongside the date it was measured, the models it used, and the harness configuration. Each run gets a permanent dated page instead of overwriting the previous one, so older results stay citable and improvements can be traced over time.
Q03Are these results reproducible?
Yes. Each run page lists the agent, the models, the number of attempts per task, and the exact plugin configuration used, along with any prompt the run added, reproduced verbatim. The suites are public, and every figure on a run page is recomputed from the run's own result files.
Q04What does k=1 mean, and why does it matter?
One attempt per task, with no re-runs and no best-of selection. Snowflake's published data-eng-bench figures are k=3, so theirs is the same measurement on three times the sample and ours is the noisier estimate. A k=1 run also cannot produce a Pass^3 figure, so none is published.