Benchmark report

Task verified

3m 56s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
PEP 668 blocking the plain pip install, forcing a venv; plus a brief SDK signature introspection pass before writing the script.

Where the time went

3:56 agent time
  • Research0:16
  • Attempt2:35
  • Verify1:05
3m 56sTotal elapsed
1:49Time to first success
0:00Waited on the owner

Judge reason

Saved code (agentmail-domain/verify_domain.py) plus the raw httpx transcript show five real calls to https://api.agentmail.to/v0: POST /v0/domains -> 200 creating tpc-bench-54a91f6b.example.com with domain_id=tpc-bench-54a91f6b.example.com and status NOT_STARTED, returning 5 DNS records (DKIM TXT agentmail._domainkey.tpc-bench-54a91f6b, MX tpc-bench-54a91f6b -> inbound-smtp.us-east-1.amazonaws.com pri 10, MX mail.* -> feedback-smtp.us-east-1.amazonses.com pri 10, SPF TXT mail.*, DMARC TXT _dmarc.*), all printed in run_output.txt and REPORT.md with type/name/value; POST /v0/domains/{id}/verify -> 204; GET /v0/domains/{id} -> 200 with status PENDING, reason dns_records_missing and every record MISSING (report explicitly does not claim verified); DELETE /v0/domains/{id} -> 204; confirming GET -> 404 not_found. No dashboard, SMTP, mock or hard-coded response; the key came from benchmark-request (AGENTMAIL_API_KEY) and was never printed. All success criteria met.

Attempt record

  1. 0:51Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.