Benchmark report

Task verified

4m 46s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Undocumented Postmark rule rejecting example.com as a public domain (422/503) killed the first run and forced a 6-candidate base-domain probe before rerunning.

Where the time went

4:47 agent time
  • Research0:18
  • Attempt2:42
  • Verify1:46
4m 46sTotal elapsed
1:45Time to first success
0:00Waited on the owner

Judge reason

Saved stdlib Python (work/postmark_domain_task.py) makes real urllib calls to https://api.postmarkapp.com with X-Postmark-Account-Token; work/run-output.log and responses/01-07 show the full documented lifecycle: POST /domains -> HTTP 200 ID 8274984; GET /domains/8274984 -> DKIM TXT 20261109204640pm._domainkey..., Return-Path CNAME pm-bounces...->pm.mtasv.net, SPF TXT v=spf1 a mx include:spf.mtasv.net ~all, all three printed with type/name/value; PUT /verifyDkim and PUT /verifyReturnPath both HTTP 200; status read back DKIMVerified=false, DKIMUpdateStatus=Pending, ReturnPathDomainVerified=false, reported as pending with no verified claim; DELETE -> HTTP 200 {ErrorCode:0,'Domain tpc-bench-978e509b.example removed.'} and GET-after-delete -> 510 'This domain was not found.'. No mocks or hard-coded responses; no dashboard or SMTP. One product-enforced deviation, independently evidenced in responses/00-base-domain-probe.json and the first run's log: POST with tpc-bench-978e509b.example.com (and .example.org/.example.net) returns HTTP 422 ErrorCode 503 'You can't use public domains.', so the agent kept the required tpc-bench-978e509b label on the RFC 6761 reserved TLD .example, which still cannot have DNS published, and disclosed the change in REPORT.md and result.json. Probe-created domains were deleted immediately. All success criteria are met by the saved code and response evidence.

Attempt record

  1. 0:52Answered from saved key: Postmark Account API Token (X-Postmark-Account-Token)no wait
  2. —First success verified

Attempt artifacts saved.