The tools

HOW THEY WORK

Paste, read, take.

  1. Paste a public URL

    Any page anyone can open without signing in. No account, no key, no install.

  2. Octocrawl checks, then reads

    It reads the site’s robots.txt first, then the page over HTTP, and keeps what the tool is for.

  3. Copy or download

    Take the result as text or a file. If the page can’t be read, the reason comes instead of an empty file.

FROM YOUR CODE OR AGENT

The same reads, one call away.

For more than five a day, a whole site, or inside your agent.

  • REST API

    Hosted, no key for 20 pages a day; a free account’s key for 1,000 credits a month.

    curl https://api.octocrawl.dev/v1/scrape -H 'content-type: application/json' -d '{"url":"https://example.com","formats":["markdown"]}'API reference
  • MCP

    Claude Code, Cursor, Codex and OpenCode read pages through Octocrawl’s tools.

    claude mcp add --transport http octocrawl https://mcp.octocrawl.dev/mcpConnect MCP
  • CLI

    On your own computer, with no daily limit, plus crawl and batch.

    npx octocrawl scrape https://example.com --markdownAdvanced reference
  • Playground

    Signed in, run any page on your credits, read in a browser when it needs one.

    octocrawl.dev/dashboard/scrape/Open the Playground

FAQ

Free tools,
answered plainly.

Are these tools free?

Yes. Every tool here runs in your browser without an account: five pages a day per visitor (at most 150 a day for the whole site), reset at 00:00 UTC.

Do I need an account?

No. Sign in when you want more: a free account gets 1,000 credits a month for the hosted API, MCP and the Playground, which also reads pages that need a browser. No card.

Can I use them from code or an agent?

Yes. The same reads are one call to the hosted API (api.octocrawl.dev) or the MCP server (mcp.octocrawl.dev), or run npx octocrawl serve on your own computer with no limit.

What happens when a page can’t be read?

You get the reason instead of an empty file: the site’s robots.txt disallows it, it blocks the request, it needs a login, or it draws its content with JavaScript, which these free tools don’t run.

More than five a day? Start free.

A free account gets 1,000 credits a month for the hosted API, MCP and the Playground. No card. Or run Octocrawl on your own computer, with no limit.