Blog
Web ExtractionHow to Find All Pages on a Website, With Their Sources
List every page of a website from its sitemaps and links in seconds, see where each URL was found, and learn why some pages are left out of the list.
Octocrawl team
Scrape a Website With Login Without Sharing Your Password
Read pages that need your login in the Chrome you already use. Your password and cookies stay in your browser, and you pass any check yourself, not a bot.
Extract Structured Data From a Web Page, Without AI
Pull fields like price and availability from a web page into JSON, each with the place on the page it was read from, and a reason when a field is missing.
How to Scrape a List of URLs and Keep Every Failure
Scrape a list of up to 1,000 URLs into Markdown files and a CSV, with one row per URL, the reason for every failure and the evidence behind each page.