Benchmark report

Task verified

7m 6s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Hunting the test-recipient and LMX docs across 404/307 pages and a 942KB llms-full.txt, plus Loops requiring a 4-call create/update/publish chain before any send.

Where the time went

7:07 agent time
  • Research0:19
  • Attempt4:49
  • Verify1:58
7m 6sTotal elapsed
3:25Time to first success
0:00Waited on the owner

Judge reason

Saved code src/send-transactional.mjs performs an authenticated POST https://app.loops.so/api/v1/transactional with Bearer key from env, and evidence/30-send-transactional-email.json records the live response: HTTP 200, body {"success": true}, with real Cloudflare headers (cf-ray a57f6cc3a9b6e6e0-PDX, x-ratelimit-remaining 9, date Mon, 09 Nov 2026 20:04:09 GMT). The saved request body {transactionalId: cmw9ow1s102l50j0jt1t242ou, email: loops-benchmark@example.com, dataVariables: {customerName, orderNumber}} matches the code constants exactly; the template id came from the live create (201, evidence/20) + update draft (200, evidence/21 carrying subject 'Loops API benchmark: transactional send', emailFormat 'plain', LMX body) + publish (200, evidence/22 returns dataVariables [customerName, orderNumber]). Recipient is Loops' documented reserved test domain: saved docs/transactional.md:266-268 states sends to @example.com/@test.com return normal success responses and are never delivered. Sender was discovered through the API (GET /v1/transactional-emails then GET /v1/email-messages/cmw9o30ar01b00jwaebp5y050 -> fromName 'Prompting Co', fromEmail 'rick', reply-to rick@promptingco.com) and applied to the draft. The documented 200 schema TransactionalSuccessResponse in the saved OpenAPI excerpt is exactly {success: boolean}, so stdout's 'Message identifier: NONE' is correct and matches the saved response. No inbox was created, so no deletion response is required. All calls were REST, not SMTP or dashboard; no mocks.

Attempt record

  1. 1:47Answered from saved key: Loops API keyno wait
  2. —First success verified

Attempt artifacts saved.