Benchmark report

Task verified

9m 22s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current AAInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
First deploy returned 500 FUNCTION_INVOCATION_FAILED because the no-framework handler signature wasn't in the quickstart; a redeploy cycle was needed.

Where the time went

9:22 agent time
  • Research0:35
  • Attempt6:50
  • Verify1:56
9m 22sTotal elapsed
4:32Time to first success
0:00Waited on the owner

Judge reason

Independently verified from saved artifacts. code/api/index.js is a real Vercel Function (documented `export default { fetch(request) }`) that logs `TPCBENCH_MARKER_LINE x-tpc-marker=<marker>`; outputs/deploy2.stdout.txt shows dpl_AJgikNExPS3GpbT49NpPk12tabiM READY, target production, on Vercel's own domain https://tpcbench-logs-20261108003401-89eed3.vercel.app. outputs/marked-request.response.txt shows one GET at 00:36:43Z with HTTP 200, x-vercel-id pdx1::iad1::n9zgr-1794098203419-589a809ceee0 and body echoing the run-unique marker tpc-marker-20261108003401-89eed339. outputs/logs-query.jsonl / logs-match.jsonl are real `vercel logs runtime --query ... --since 15m --json` JSONL records (projectId prj_Pz7Y33..., requestId n9zgr-1794098203419-589a809ceee0 matching the response header, deploymentId matching the deploy) containing message `TPCBENCH_MARKER_LINE x-tpc-marker=tpc-marker-20261108003401-89eed339`; logs-poll.log shows FOUND on attempt 1 after 1s within the 60s budget and the transcript prints the line. Retention needed no enable step (Pro team, docs cited). Cleanup: `vercel project rm --confirm-name` -> "Deleted" (cleanup.stderr.txt), `project ls | grep tpcbench` -> 0, REST /v9/projects -> 404, and final-fetch.txt is HTTP/2 404 x-vercel-error DEPLOYMENT_NOT_FOUND (both per-deployment URLs also 404). Only resource created was tpcbench-logs-20261108003401-89eed339. Token scan of /workspace/home finds no bearer/vcp_ token; the one hit was Vercel's own published example in the docs copy, redacted. Nothing mocked or hard-coded.

Attempt record

  1. 1:24Answered from saved key: Vercel access token (VERCEL_TOKEN)no wait
  2. —First success verified

Attempt artifacts saved.