Top web-crawler tools
Tools tagged “web-crawler” gaining the most momentum right now, any age.
- 93· newfirecrawl🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.ai-agentsmcp-servers+3098.6
- 294· newScrapegraph-aiPython scraper based on AIrag-evalai-crawler+1995.1
- 493· newwigoloThe go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.ai-agentsai-dev-tools+2291.2
- 1762· newspiderGet web data for AI agents and LLMsai-agentsai-agent+957.7
- 2056· newbrowserlessThe headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.automationbrowser-automation+647.3
The weekly drop
Get the weekly rising list
The top risers across agents, MCP servers and dev tools, once a week in your inbox. No spam, unsubscribe anytime.