MCP server

Website to Markdown

io.github.saulius876-lgtm/website-content-crawler

Checked 26 Oct 2026, 08:29 UTC · It answered and asked for a sign-in

Web pages and whole sites to clean Markdown for RAG and AI agents. Crawls links and sitemaps.

Website

At a glance

Type

MCP server

Publisher

Not verified

Status

Signs in with your account

Sets up in

Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, VS Code, Antigravity, ChatGPT, OpenAI API, Grok

Last checked

26 Oct 2026, 08:29 UTC

Connect it to your agent

Add to CursorAdd to VS CodeVS Code Insiders

Opens the app, which shows the server and asks you to confirm. For other apps, copy the setup below.

Claude Code

claude mcp add --transport http website-content-crawler https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api

Claude Desktop

Settings → Connectors → Add custom connector, and paste the URL.

Codex (config.toml)

[mcp_servers.website-content-crawler]
url = "https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"

Gemini CLI

{"mcpServers":{"website-content-crawler":{"httpUrl":"https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"}}}

Cursor

{"mcpServers":{"website-content-crawler":{"url":"https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"}}}

VS Code

{"servers":{"website-content-crawler":{"type":"http","url":"https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"}}}

Antigravity

agy mcp add website-content-crawler https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api

ChatGPT

Settings → Apps & Connectors → Advanced settings: turn on Developer mode, then Create a connector and paste the URL. Which plans can do this varies.

OpenAI API (Responses tools)

{"type":"mcp","server_label":"website-content-crawler","server_url":"https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api","require_approval":"always"}

Grok (xAI API tools)

{"type":"mcp","server_label":"website-content-crawler","server_url":"https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"}

Signs in with your account: your agent app opens a sign-in page the first time you use it.

Browse

• GitHub tools
• Web scraping
• Converting to Markdown
• GitHub and code hosting
• Searching your documents (RAG)

Related MCP servers

• Apify
  Apify MCP — run web-scraping Actors and fetch their dataset results.
• US Company Intelligence for AI Agents (SEC EDGAR, x402)
  SEC EDGAR company briefs for agents: cited synthesis, XBRL metrics. x402/USDC, no account.
• mcp-my-site
  Search and fetch tools for one website, served from an index you build with the mcpmysite crawler.
• AIPayGen Site Intel
  Website and company intel for agents: DNS, WHOIS, SSL, tech stack, email security, profiles
• JustAutomate website
  JustAutomate, AI agents and process automation agency: site search, pages, workshop price, contact.
• Scrapingdog
  Scrapingdog MCP — wraps Scrapingdog (scrapingdog.com), a proxy-based web
• Hydrafetch
  Turn any URL into clean Markdown and structured data. Scrape, crawl, search and extract.
• Site SEO & AI Crawler Audit
  Check public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion.

Badge for your README

[![AptLynx](https://aptlynx.net/badge/io.github.saulius876-lgtm/website-content-crawler.svg)](https://aptlynx.net/servers/io.github.saulius876-lgtm/website-content-crawler)

Listing text is written by the maker; AptLynx lists tools and does not vouch for them. Find another tool · Browse by topic · For agents