Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Receba novas skills de Pesquisa e Web toda segunda
1 email curto, só com as skills novas de Pesquisa e Web. 4 minutos de leitura, sem spam, cancela com 1 clique.
Você confirma o email no primeiro envio. Sem spam. Cancela com 1 clique.
Ler descrição completaRecolher descrição
firecrawl crawl
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
When to use
You need content from many pages on a site (e.g., all /docs/)
You want to extract an entire site section
Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → interact
Quick start
# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
# Full crawl with dep
[Description truncada. Veja o README completo no GitHub.]
