Benchmark report

Task verified

4m 9s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Locating the documented blackhole test recipient: it was not in the fetched API docs and two guessed support-article URLs returned 404, forcing a web search.

Where the time went

4:10 agent time
  • Research0:19
  • Attempt2:44
  • Verify1:06
4m 9sTotal elapsed
1:43Time to first success
0:00Waited on the owner

Judge reason

Saved code postmark-send/send_email.py POSTs to the documented https://api.postmarkapp.com/email with the X-Postmark-Server-Token header (token requested via benchmark-request, transcript idx 13-14, validated by GET /server -> HTTP 200, server 19755949). Recipient is Postmark's documented blackhole test address test@blackhole.postmarkapp.com; the sender rick@promptingco.com was discovered through the API (GET /messages/outbound -> 200, 18 prior Sent messages from that address) after /senders correctly returned 401 'please use a valid Account token'. request.json matches the PAYLOAD literal in the code (From/To/Subject/TextBody/MessageStream). Live response in response.json and transcript idx 60: HTTP 200 with {"ErrorCode":0,"Message":"OK","MessageID":"c6f09376-e0ab-44bf-b468-74e0b2258f0e","SubmittedAt":"2026-11-02T07:18:57.5475831Z","To":"test@blackhole.postmarkapp.com"} - the documented Postmark send shape; printed status and MessageID in send-output.txt match it exactly. An independent read-back (verify_message.py, GET /messages/outbound/<id>/details -> HTTP 200) returns Status 'Sent', the same Subject, identical TextBody, and a Delivered event 'smtp;250 2.0.0 OK (sandbox blackhole)', which no mock or hard-coded value could produce. Exactly one email, sent over HTTPS REST, not SMTP or dashboard.

Attempt record

  1. 0:43Answered from saved key: Postmark Server API tokenno wait
  2. —First success verified

Attempt artifacts saved.