MCP server

TheCrawler

io.github.manchittlab/thecrawler

Checked 26 Oct 2026, 08:31 UTC · Still published on its registry; checked against OSV.dev; never run by us

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Runs on your computer

Source

At a glance

Type: MCP server
Publisher: Not verified
Status: Runs on your computer
Sets up in: Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, VS Code
Last checked: 26 Oct 2026, 08:31 UTC

Connect it to your agent

Add to Cursor Add to VS Code VS Code Insiders

Opens the app, which shows the server and asks you to confirm. It then runs the package on your computer with your permissions. For other apps, copy the setup below.

Claude Code

claude mcp add thecrawler -- npx -y thecrawler@0.5.8

Claude Desktop

{"mcpServers":{"thecrawler":{"command":"npx","args":["-y","thecrawler@0.5.8"]}}}

Codex (config.toml)

[mcp_servers.thecrawler]
command = "npx"
args = ["-y","thecrawler@0.5.8"]

Gemini CLI

{"mcpServers":{"thecrawler":{"command":"npx","args":["-y","thecrawler@0.5.8"]}}}

Cursor

{"mcpServers":{"thecrawler":{"command":"npx","args":["-y","thecrawler@0.5.8"]}}}

VS Code

{"servers":{"thecrawler":{"type":"stdio","command":"npx","args":["-y","thecrawler@0.5.8"]}}}

Runs on your computer with your permissions: check the package and its source before installing.

Browse

Related MCP servers

Civilquants
Parametric estimating engine: CESMM4/NRM2/SMM7/MMHW Bills of Quantities for civils assemblies.
Fuze Data Tools
No-code web scraping & data-extraction Apify actors: scrape and enrich web data at scale.
inktomd MCP Server
Convert files, URLs, and documents to clean, AI-ready Markdown via MCP.
doc.page PDF Extraction
Extract PDFs to Markdown, RAG chunks and cited tables; publish tracked Doc Links with read stats.
Oxylabs
Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
ScorpTech x402 API
112 x402 pay-per-call tools: web data, security/OSINT, SEC and public data, Solana/EVM lookups
HasData
All HasData scraping tools in one MCP server: Google, TikTok, Instagram, maps, e-commerce and more.
Agentdata Nl
x402-paid tools: EU company & sanctions data, web search, LLM chat, crypto intel, x402 monitoring.

Badge for your README

[![AptLynx](https://aptlynx.net/badge/io.github.manchittlab/thecrawler.svg)](https://aptlynx.net/servers/io.github.manchittlab/thecrawler)

Listing text is written by the maker; AptLynx lists tools and does not vouch for them. Find another tool · Browse by topic · For agents