Benchmark report

Task verified

7m 3s elapsed

The saved attempt met the documented success criterion.

Experience score

85/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs A, Current AAInstallAAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The pooler connection_string returned a literal [YOUR-PASSWORD] placeholder, causing a failed auth connect and a diagnose-plus-rewrite of run_03.sh before the SQL run succeeded.

Where the time went

7:03 agent time
  • Research0:23
  • Attempt4:50
  • Verify1:50
7m 3sTotal elapsed
3:18Time to first success
0:00Waited on the owner

Judge reason

Verified end to end from saved code and outputs. work/01_create_project.py created project tpcbench-0ec2534f (POST /v1/projects -> 201, ref vdpusoimjpfrkhdfkopl, outputs/01c) and polled ACTIVE_HEALTHY. work/02 fetched the Supavisor pooler config (outputs/02a: aws-0-us-east-1.pooler.supabase.com:6543, user postgres.vdpusoimjpfrkhdfkopl). run_03.sh built DATABASE_URL with sslmode=require into an env var without echoing it; 03_sql_workload.py asserts sslmode=require in the DSN and connected with psycopg 3.3.8 to a live PostgreSQL 17.11 server. outputs/03a_sql_transcript.json and 03b stdout show CREATE TABLE items(id integer PRIMARY KEY, name text), three INSERTs via %s bound parameters with bound_parameters [1,alpha],[2,beta],[3,gamma] each 'INSERT 0 1', and SELECT id,name FROM items WHERE id >= %s ORDER BY id with bound parameter 2 -> SELECT 2 printing exactly (2, beta) then (3, gamma); no string concatenation anywhere. Cleanup: DELETE /v1/projects/vdpusoimjpfrkhdfkopl -> HTTP 200 with body, follow-up GET -> 404 'Resource has been removed' and GET /v1/projects showing the ref absent (outputs/04a, 04b, 04c). My own recursive grep of /workspace/home found no raw token and only [REDACTED-DB-PASSWORD] inside the pooler URIs; the agent's own leak scan returned NONE. Transcript tool results match every saved artifact byte for byte, so nothing is fabricated.

Attempt record

  1. 1:05Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.