Benchmark report

Task verified

5m 21s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Python packaging friction: PEP 668 pip refusal plus psycopg installed to user site-packages, costing two failed script runs before the SQL work started.

Where the time went

5:21 agent time
  • Research0:24
  • Attempt3:31
  • Verify1:26
5m 21sTotal elapsed
1:51Time to first success
0:00Waited on the owner

Judge reason

Live end-to-end run verified. POST /v1/projects -> HTTP 201 created tpcbench-6694ce2e (ref qsjphbcjgintaudrxirv) (evidence/02-create-project.json, transcript idx 28); GET /v1/projects/{ref}/config/database/pooler returned the Supavisor host aws-0-us-east-1.pooler.supabase.com:6543 (evidence/04-pooler-config.json). work/db_task.py connects with psycopg3, sslmode=require, prepare_threshold=None, and uses bound parameters only ('INSERT INTO items (id, name) VALUES (%s, %s)' with (1,alpha),(2,beta),(3,gamma); 'SELECT id, name FROM items WHERE id >= %s ORDER BY id' with (2,)) - no concatenation. Runtime output (evidence/05-sql-session.log, transcript idx 61) shows CREATE TABLE, three 'INSERT 0 1' statusmessages from the real server, PostgreSQL 17.12 version string, and rows read back exactly (2, 'beta') then (3, 'gamma'), matching evidence/05-sql-session.json. TLS confirmed by psql \conninfo against the pooler: 'SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384)' (evidence/06-tls-psql.txt); ssl_is_used() absent behind the pooler is explicitly optional. Cleanup: DELETE /v1/projects/{ref} -> HTTP/2 200 with body {ref, name}; follow-up GET of the project -> HTTP 403, GET /v1/projects lists only 'Rick sandbox' (deleted project still listed: False), and a post-delete psql connect fails FATAL (ENOTFOUND) tenant/user not found (evidence/07-delete-and-verify.txt). No mocked or hard-coded data. Secret scan of /workspace/home found no password or password-bearing connection string: the only credential-shaped URL is Supabase's own '[YOUR-PASSWORD]' placeholder in the pooler response; db_pass is '***REDACTED***' in the saved request, and the password file was shredded.

Attempt record

  1. 0:42Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.