Benchmark report

Task verified

10m 6s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
A branch-status poll loop stalled ~2 min because the API returned ACTIVE_HEALTHY, not a documented branch status, forcing a TaskStop and rerun.

Where the time went

10:06 agent time
  • Research0:27
  • Attempt7:28
  • Verify2:11
10m 6sTotal elapsed
5:19Time to first success
0:00Waited on the owner

Judge reason

Saved curl/psql scripts and full responses show the real flow: POST /v1/projects created tpcbench-b916825 (ref eqrsmsorrprhlzturuwc, HTTP 201), psql over aws-0-us-east-1.pooler.supabase.com:5432 sslmode=require created items with 2 rows (05-parent-seed.txt). The branch came from the documented Branching API, POST /v1/projects/{ref}/branches -> HTTP 201 {"id":"ef40d10f...","project_ref":"odbpyjtrnbmxfcxlxvuy","parent_project_ref":"eqrsmsorrprhlzturuwc","with_data":false}, and the branch is listed in GET /v1/projects/{ref}/branches beside main - not a plain copy. The fresh branch showed items with 0 rows; the report cites the fetched doc (outputs/11-branching-doc.md line 18, 'Data-less by default: ... new branches do not start with any data') to justify seeding the two parent rows. Branch-only ALTER TABLE ... ADD note + one insert; 10-verify-branch-vs-parent.txt, run after the migration over TLS on both, shows BRANCH item_rows=3, has_note_column=1, columns id/name/note vs PARENT item_rows=2, has_note_column=0, columns id/name (ssl=f behind Supavisor is explicitly allowed). Cleanup: DELETE /v1/branches/{id} -> 200 {"message":"ok"} with follow-ups GET branch -> 404, branch absent from the branch list, GET branch project -> 404; DELETE /v1/projects/{ref} -> 200 with GET -> 404 'Resource has been removed' and final project list containing only the pre-existing sandbox. No password, password-bearing connection string or token appears in any saved file (only the API's own [YOUR-PASSWORD] placeholders); secrets stayed in /tmp/bench-private. Nothing mocked or hard-coded.

Attempt record

  1. 1:09Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.