Octocrawl / Dashboard

Jobs

Hosted batch and crawl jobs on this account: their progress, what they cost and where their results are.

Batch and crawl jobs

 
StartedJobStatusProgressCreditsFinishedResults

A job reads its pages a slice at a time, each priced as a scrape: 1 credit over HTTP, 5 in a browser. A running job's progress refreshes here every few seconds. Results are kept 7 days after a job finishes.

Read a job's results

Docs →

Export on a finished job downloads every page it read as one gzipped JSON Lines file, one result with its Evidence Record a line. The link it opens works for 15 minutes.

MCP, with the job's id

get_batch_items { "id": "<job id>" }
get_crawl_pages { "id": "<job id>" }

REST, with a key from API keys

curl https://api.octocrawl.dev/v1/batches/<job id>/items \
  -H 'authorization: Bearer oc_YOUR_KEY'
curl https://api.octocrawl.dev/v1/crawl/<job id>/pages \
  -H 'authorization: Bearer oc_YOUR_KEY'
curl https://api.octocrawl.dev/v1/jobs/<job id>/export \
  -H 'authorization: Bearer oc_YOUR_KEY'