Benchmark report

Task verified

9m 11s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Branch-create API shape was not on the branch docs page, so the agent had to fetch llms.txt and the raw OpenAPI spec to find mode:inherit/parentID.

Where the time went

9:11 agent time
  • Research0:26
  • Attempt6:07
  • Verify2:38
9m 11sTotal elapsed
4:15Time to first success
0:00Waited on the owner

Judge reason

Live Xata run verified from saved artifacts and transcript. Project tpcbench-20261110012531-39fa6d39 created (out/06, 201); parent branch 'main' mnigsgo7q52295p8jpp8l0u6hg created (out/08, 201) and seeded with items(id,name) = alpha,beta (out/11). Branch made via the documented endpoint POST /organizations/{org}/projects/{project}/branches with body {"name":...,"mode":"inherit","parentID":"mnigsgo7..."} (out/13 request, out/14 response 201 carrying parentID) — the BranchFromParent schema in the saved OpenAPI/doc says the branch 'will inherit the parent branch configuration and data', so it is Xata copy-on-write branching, not a copy, and no non-inheritance citation was needed (branch showed id,name + alpha,beta pre-migration, out/16 step 1). ALTER TABLE items ADD COLUMN note + one insert ran on the child only; out/17 shows parent = 2 rows, has_note_column f, child = 3 rows (gamma 'added on the branch only'), has_note_column t. TLS: credentials endpoint string plus sslmode=require, psql \conninfo reports 'SSL connection (protocol: TLSv1.3, cipher: TLS_AES_128_GCM_SHA256)' and sslmode=disable is refused (out/12); pg_stat_ssl ssl=f is the documented edge-proxy case and optional. Cleanup: child branch DELETE 204 -> GET 404 and list shows only main; parent DELETE 204 -> GET 404 and list {"branches":[]}; project DELETE 204 -> GET 404 and no tpcbench- project remains (out/18). Code in run/code is real and runnable (curl/psql, redact.py filter); no mocks. Saved files contain no password or password-bearing connection string (scan found only the redact.py docstring placeholder); one raw connection string from the branch-create API response did appear once in terminal output before the redaction filter was installed, and the agent redacted the on-disk copy immediately.

Attempt record

  1. 1:42Answered from saved key: XATA_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.