Exact prompt sent to every model
Write runner.py with async def run(tasks: dict[str, Task], *, concurrency: int = 4) -> Report.
Task is a dataclass with deps: list[str] and fn: Callable[[], Awaitable[Any]]. Run tasks in
dependency order with at most concurrency running at once. If a task raises, mark it failed, mark
every transitive dependent skipped, and keep running everything else. Detect cycles before starting
and raise CycleError naming the cycle. Detect unknown dependency names and raise KeyError.
Report is a dataclass with succeeded, failed (name to exception), skipped (name to the failed
ancestor name), and order (start order). Use asyncio only. Type hints throughout.
Return only the file in one code block.
Responses
Latest published run: gemini-3.8-flash. For each model the repeat closest to its published median is shown; every repeat is in the repository.
No published attempts for this task yet
When the run is published, every model's raw response, per-dimension scores, judge rationales, human review notes and rendered artifacts appear here side by side.