Benchmark report

Task verified

13m 29s elapsed

The saved attempt met the documented success criterion.

Experience score

91/ 100

Agent-Ready
  • SInstall
  • AAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute S, Docs S, Current ASInstallAAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Async compute-endpoint teardown: ~6 minutes of polling (08_poll_endpoint.sh, worsened by an rc-capture bug) before a clean re-check confirmed the endpoint was gone.

Where the time went

13:30 agent time
  • Research0:23
  • Attempt11:11
  • Verify1:54
13m 29sTotal elapsed
2:13Time to first success
0:00Waited on the owner

Judge reason

Saved artifacts prove the full flow against live Neon. POST /api/v2/projects returned HTTP/2 201 (01_create_project.headers) creating project divine-recipe-64789759 'tpcbench-52f71eac'. 03_admin_setup.out shows an admin psql connect with sslmode=verify-full&sslrootcert=system (\conninfo: TLSv1.3, user neondb_owner), CREATE TABLE items, 3 rows inserted, then the documented SQL path (CREATE ROLE items_reader LOGIN + GRANT CONNECT/USAGE/SELECT, cited to the saved neon.com/docs/manage/roles#manage-roles-with-sql and database-access docs), with pg_has_role(...,'neon_superuser')=f. 04_reader_test.out shows a verify-full connect as items_reader, SELECT returning the 3 rows, and 'ERROR: permission denied for table items' on INSERT (psql rc=1). 05_revoke.out shows REVOKE+DROP ROLE, pg_roles count 0, and the revoked credential failing with 'password authentication failed for user items_reader'. 06_cleanup.out shows DELETE HTTP 200 and follow-up GET HTTP 404 'project not found'; 07_verify_gone.out shows the project absent from the org list and branches 404; 09_final_check.out shows the endpoint refusing with 'The requested endpoint could not be found'. No real password or password-bearing URI appears in any saved file (only Neon doc placeholders); responses were redacted by 02_extract_and_redact.py and a secret scan ran. Nothing mocked or hard-coded.

Attempt record

  1. 0:42Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.