Results

Rank	Team	LLM used									Report

Pass³ is the primary score: a task counts only when all three independent trials succeed. Latency and token telemetry are self-reported and could not be independently verified; cost is estimated from reported tokens using standardized pricing and a 75% prompt-cache assumption.

Award Winners

Recognized teams

The Rank Award recognizes the strongest Track 1 Pass³ result; the Innovation and Cerebras Awards recognize distinctive advances in reliable and compute-aware agent design.

AI-generated report summary

AI-generated; verify details against the submitted report.

Submitted technical report

Open PDF · Download

Pass³ Primary score

A task counts only when all three trials succeed.

How to read the scores

Pass³ requires all three trials for a task to succeed. Pass@3 requires at least one successful trial.

Pass@1 is the mean single-trial success rate across the benchmark. Successful trials is the raw total across 90 runs.

Base, Hallucination, and Disambiguation show the corresponding task-category subscore beneath each pass metric.

Success consistency measures how often an initial task success is repeated across the remaining trials. Latency, tokens, and cost measure efficiency; lower is better.

Ranks and color shading compare submissions within the same track. Gray cards indicate that a reliable comparison was unavailable.

Telemetry note: Latency and token figures were self-reported by submissions and could not be independently verified. Cost is estimated from those reported tokens using standardized pricing and a 75% prompt-cache assumption.

Track 1: Open Track

Recognized teams

Submitted technical report