Benchmark report

Task verified

8m 52s elapsed

The saved attempt met the documented success criterion.

Experience score

91/ 100

Agent-Ready
  • SInstall
  • AAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute S, Docs S, Current ASInstallAAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Undocumented org_id requirement on GET /projects (400) plus the post-delete endpoint teardown lag that needed a timed recheck.

Where the time went

8:52 agent time
  • Research0:26
  • Attempt6:25
  • Verify2:00
8m 52sTotal elapsed
3:20Time to first success
0:00Waited on the owner

Judge reason

Saved code and real API/psql output support every criterion. POST /api/v2/projects -> 201 created tpcbench-0e55790a8d451002 (id lively-firefly-11605975, evidence/01-*). Admin connected over sslmode=verify-full&sslrootcert=system (02-admin-setup.txt: TLSv1.3) and created items with 3 rows. Scoped credential made by the documented SQL path (CREATE ROLE tpcbench_readonly, GRANT CONNECT/USAGE/SELECT, CREATE USER tpcbench_reader, GRANT role), citing neon.com/docs/manage/database-access, whose saved text (evidence/database-access.txt lines 18, 25, 191) confirms SQL is Neon's documented least-privilege mechanism. 04-scoped-credential-select-and-denied-insert.txt shows a verify-full connect as tpcbench_reader, current_user=tpcbench_reader, the 3-row SELECT, and 'ERROR: permission denied for table items' on INSERT. 05-revoke-and-connect-fails.txt shows REVOKE/DROP ROLE, 0 remaining roles, and 'password authentication failed for user tpcbench_reader' on reconnect. 06-cleanup.txt shows DELETE -> 200 with the project body, follow-up GET -> 404 'project not found', and absence from the org project list; 09 records the endpoint ceasing to serve connections ~60s later. A secret scan (transcript tool result after leakcheck) reported 0 files containing the API key, admin password, or reader password; all saved URIs show [REDACTED]. Nothing mocked or hard-coded.

Attempt record

  1. 1:16Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.