Benchmark report

Task verified

4m 41s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Docs' snake_case fields vs the SDK's camelCase inboxId/displayName broke the first run and left an orphan inbox to clean up before the successful rerun.

Where the time went

4:42 agent time
  • Research0:14
  • Attempt3:02
  • Verify1:26
4m 41sTotal elapsed
2:05Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (agentmail-task/run.js) uses the real agentmail SDK v0.5.52 against https://api.agentmail.to with a tee of every HTTP call. evidence/http-transcript.jsonl (byte-identical to http-transcript-success.jsonl) shows live, unmocked calls: POST /v0/inboxes -> 200 with inbox_id/email bench-1793922886859@agentmail.to; GET that inbox -> 200; DELETE /v0/inboxes/{id} -> 202 Accepted; poll GET -> 200 at +0s then 404 not_found at +2.19s, well inside the 60s bound. run-console.log shows the identifier and deliverable address printed, and 06-final-state.log re-confirms 404 for that inbox plus the orphan from an earlier aborted run, which was also deleted. Setup was clean: npm init -y + npm install agentmail, bearer am_ key read from a private file and never printed. Reported end state matches the documented deleted state rather than the intermediate 202.

Attempt record

  1. 0:50Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.