The web

Web scraping: tools for AI agents

Fetch web pages and pull out their text, links and data, including pages built with JavaScript. The best matches among the MCP servers, command-line tools, web APIs and Claude Code plugins AptLynx checks every night, working ones first.

MCP servers

• Ujeebu: ujeebu_scrape_extract Verified publisher · Working at last check
  Extract structured data from web pages using custom rules. Define CSS selectors or XPath queries to extract specific data. Always returns JSON with extracted…
• Proweblook: scrape_url Working at last check
  Scrape any web page: get clean Markdown, rendered/raw HTML, all links, SEO metadata, and/or a full-page screenshot in a single call. Costs 1 scraping credit…
• Hasdata: hasdata_web_scraping_web_scraping_scrapeWebPage Verified publisher · Working at last check
  Scrape Web Page Universal web scraper that fetches any public URL through managed proxies (datacenter or residential, geo-targeted) with optional JS…
• OfficialSpicrawl: spicrawl_scrape Working at last check
  Retrieve a web page and return its content as markdown (default), text, HTML, a JSON envelope, or a printed PDF. Handles the fetch vs. browser decision,…
• Hydrafetch Web: scrape Verified publisher · Working at last check
  Fetch one web page and return its main content as clean markdown, with optional HTML, links or structured data. Use it to read a page whose URL you already…
• Blackwood State: blackwood_webpage_markdown Working at last check
  Use when you need web page to markdown, url to markdown, to scrape a web page or extract content from a url for an LLM. Do not use for pages that need…
• Agenttools Hub: dev_scrape_selector Working at last check
  Fetch a web page server-side (SSRF-guarded) and return every element matching a CSS selector as structured JSON: text content, attributes and HTML.…
• Scrapiq: scrapiq_extract Verified publisher · Working at last check
  Extract a web page into clean, structured content for LLM/RAG pipelines via the Scrapiq API. Strips boilerplate, navigation, ads, and scripts. Returns title,…
• Mifactory Scraping Api: scrape Working at last check
  Extract clean text from any URL
• com.mcparmory/scrapingant Verified publisher · Runs on your computer
  Scrape webpages, handle JavaScript and CAPTCHA, extract structured data
• Agentdevx: webscrape.scrape Working at last check
  Scrape a webpage and extract its structure: title, headings, links, images, and text content. Uses static HTML fetch (does not render JavaScript).
• Web Search: fetch_page_content Verified publisher · Working at last check
  Fetch and extract content from a web page. Returns the page content in markdown format.
• Pipeworx Apify: apify_scrape_url Verified publisher · Working at last check
  Scrape a single web page (or crawl from it) and get clean, RAG-friendly markdown content. Convenience wrapper around the "apify~website-content-crawler"…
• Render Api: extract_page Verified publisher · Working at last check
  Fetch a public web page and extract its content as either clean, readable markdown (the default) or as structured JSON fields matched against a…
• com.mcparmory/firecrawl Verified publisher · Runs on your computer
  Scrape, crawl, and extract structured data from websites at scale

Command-line tools

• defuddle
  Extract article content and metadata from web pages
• trafilatura
  Discovery, extraction and processing for Web text
• htmlq
  Uses CSS selectors to extract bits content from HTML files
• you-get
  Dumb downloader that scrapes the web
• site_icons
  Website icon scraper that fetches sizes (with WASM support)
• wget
  Internet file retriever cross-platform downloader open-source

Web APIs

• Dataflow Kit Web Scraper
  Render Javascript driven pages, while we internally manage Headless Chrome and proxies for you. - Build a custom web scraper with our Visual point-and-click…
• Indexing API
  Notifies Google when your web pages change. analytics media
• Urlbox API
  A plugin that allows the user to capture screenshots of a web page from a URL or HTML using ChatGPT. developer_tools
• Amazon Detective
  Detective uses machine learning and purpose-built visualizations to help you to analyze and investigate security issues across your Amazon Web Services…
• Medium API
  Medium API helps you to quickly extract data from Medium's Website (https://medium.com). You can gather data related to users, publications, articles…
• Scrape Website Email API
  ScrapeWebsiteEmail is a service that exposes an api to fetch e-mails from a website. email tools

Claude Code plugins

• firecrawl
  Web scraping and crawling powered by Firecrawl. Turn any website into clean, LLM-ready markdown or structured data. Scrape single pages, crawl entire sites,…
• brightdata-plugin
  Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown…
• zyte-web-data
  Web scraping skills for Claude Code powered by the Zyte API — scrape sites, generate and run Scrapy spiders, define extraction schemas, and ship to Scrapy…
• pptx
  Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or…

Search for something more specific

Related topics

• Browser automation
• Web search
• SEO
• Website monitoring
• Domains and DNS
• Website screenshots

Listing text is written by each tool's maker; AptLynx lists tools and does not vouch for them.