Benchmark report

Task verified

5m 38s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Hunting an unpublished Clay-managed-function routine id (404/400 probes, CLI needs interactive OAuth) before pivoting to the documented Searches query-mode path.

Where the time went

5:39 agent time
  • Research0:25
  • Attempt3:19
  • Verify1:55
5m 38sTotal elapsed
1:47Time to first success
0:00Waited on the owner

Judge reason

Saved code code/clay_company_search.sh calls the documented Clay Public API Searches operation (POST https://api.clay.com/public/v0/search/query-mode then POST /search/query-mode/{search_id}/run) with the clay-api-key header, using the query 'select from companies where clay.include_company_identifiers(("stripe.com"))' — the exact domain-identification pattern in Clay's live query reference (work/search_reference.md:454) and the flow in docs/recipes_search-and-enrich.md. Both calls returned HTTP 200 (work/01_create_search.status, work/02_search_results.status; transcript idx 77). The live body evidence/company_enrichment_response.json matches the documented NextQueryModeCompanyResultsResponse schema in docs/openapi.json (data/has_more/source_type:'companies') and contains name 'Stripe', domain 'stripe.com', industry 'Technology, Information and Internet', size '5,001-10,000', linkedin_url, plus a real period_quota (used 576937) — not mocked. evidence/printed_values.txt matches the saved response field for field (transcript idx 121). The agent also documented the Routines gap with evidence: openapi.json exposes no routine-listing endpoint and GET /public/v0/routines returned 404, Clay-managed-function ids are unpublished (docs/routines_clay-managed-functions.md). No saved file, printed output, or log contains the API key or any bearer/access token; the key was read from /tmp/clay-secret outside the home dir and only its byte count was ever echoed.

Attempt record

  1. 1:24Answered from saved key: Clay Public API keyno wait
  2. —First success verified

Attempt artifacts saved.