Skip to content

Rubric v1 · checked 2026-10-27

Web scraping MCP servers, compared on a published rubric

There is no single best web scraping MCP server. Pick the category first: browser control, crawl platform, self-hosted crawler or single-page content API. Then check the shortlist on a linked, dated rubric.

Read the MCP docsRequest access

Teneo Labs runs this site and makes one product in these tables. Disclosed on every page.

rubric-v1.yaml (one server)
server: live-web-data-api  # the operator's own product
category: single-page content API
checked: 2026-10-27
rows:  # the ten rubric rows, in guide order
  transport_auth: "Streamable HTTP, bearer key"
  tool_surface: "2 tools"
  output: "markdown | text | html; max_characters"
  failure_behavior: "one error sentence; not charged"
  freshness: "always live; no cache"
  scope: "one public page; MCP tool: no JS, crawl, search"
  hosting_license: "hosted only"
  pricing_shape: "credits per successful page"
  compliance: "AUP; robots.txt not promised (Terms 7.3)"
  maintenance: "not documented"
score: none  # we publish values, not points

The problem

Why most scraping MCP server lists don't help you choose

Search for a scraping MCP server and you mostly get rankings written by companies that sell one. Four things make them hard to use.

  • The author's product wins

    Many roundups come from scraping vendors who rank their own server first, with no disclosure box and no written criteria for the order.

  • Four kinds of tool, one list

    Browser drivers, crawl platforms, self-hosted crawlers and page-reading APIs solve different jobs. Ranking them on one line hides the only choice that matters.

  • Tool surface costs tokens

    In clients that load tools upfront, every tool a server registers is described to the model in each session. A server with 25 tools spends context before it reads a single page.

  • No check date, no source

    These servers ship often. A claim with no link to the project's docs and no date it was checked can be wrong by the time you read it.

How it works

How the rubric works: from job to shortlist

Four steps, in this order. Skipping the first one is how teams end up running a full browser to read a docs page.

  1. Step 1: Name the job

    Read one public page, click through a form flow, map a whole docs site, or keep every request on your own machines. Write it down in one sentence.

  2. Step 2: Pick the category

    The job decides it: a content API for reading pages, a browser server for interaction, a crawl platform for many pages, a self-hosted crawler. The pick-by-job guide maps each.

  3. Step 3: Run the rubric

    Check the ten rows: transport and auth, tool surface, output, failure behavior, freshness, scope, hosting and license, pricing shape, compliance, maintenance.

  4. Step 4: Test on example.com

    Connect one candidate, read a known page and a known 404, and look at the output size and the error text before your agent depends on it.

Pick by job

What each kind of MCP scraper is good at

Six jobs that come up again and again, each with the guide that works through it on the rubric.

  • Read docs in a coding agent

    Claude Code or Cursor needs a library's docs page as text. A small read-only server fits better than a browser server with about 25 tools and a snapshot after each step.

    Read the guide

  • Replace Firecrawl MCP

    Switching for cost shape, self-hosting, a smaller tool surface or browser interaction? Each reason leads to a different shortlist.

    Read the guide

  • Keep scraping on your machines

    Crawl4AI, Scrapling and Playwright MCP run locally under open-source licenses. You trade a bill for operating work, updates and a request-rate policy.

    Read the guide

  • Keep the context window small

    Compare output formats, max-characters controls and how many tools each server registers, so pages fit the budget your agent actually has.

    Read the guide

  • Budget before you standardize

    Credits per page, pay per event, browser minutes or your own servers: learn each pricing shape and what a failed call costs under it.

    Read the guide

  • Get a server approved

    A review checklist for security and compliance: what the server can reach, how it authenticates, what it logs, and its stance on robots.txt.

    Read the guide

Try one

Connect a content-API MCP server and read one page

The rubric's test step with the operator's own product: one remote server, two tools, a bearer key. Swap in any candidate and compare the output.

mcp.json
{
  "mcpServers": {
    "live-web-data": {
      "url": "https://mcpscraper.net/mcp",
      "headers": {
        "Authorization": "Bearer ${env:LIVE_WEB_DATA_API_KEY}"
      }
    }
  }
}
Tool call
{
  "name": "get_page_contents",
  "arguments": {
    "url": "https://example.com/docs/getting-started",
    "format": "markdown",
    "max_characters": 8000
  }
}
REST (curl)
curl -X POST https://livewebdata.com/api/v1/contents \
  -H "Authorization: Bearer $LIVE_WEB_DATA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/docs/getting-started"]}'

Keys work once your plan is active. Step-by-step setup for each MCP client is in the shared MCP docs. This site compares servers; it does not teach setup.

Comparison

Scraping MCP servers compared by category

A web scraping MCP comparison by category, not by brand. Self-hosted crawlers sit in the third column. Named servers are in the rubric guide.

Scraping MCP servers compared by category
CriterionSingle-page content APIBrowser-control serverCrawl platform or crawler
Unit of workOne public URL per callOne browser session, step by stepA site, a sitemap or a URL list
Typical tool surface2 (Live Web Data API) to 12 (Jina), some filterable16 (Scrapling) to about 25 (Playwright)7 (Crawl4AI) to 25+ (Firecrawl), often configurable
Clicks, forms, loginsNoYes, that is the pointSome offer an interact or actions tool
JavaScript-heavy pagesVaries; Live Web Data API does not render themYes, a real browser runsUsually, via a headless browser
Output into contextMain content as Markdown or text, cappedAccessibility snapshots after each stepMarkdown, HTML, or JSON per page
Where it runsHosted, remote MCPYour machine or a hosted browserHosted, self-hosted or both
Pricing shapeCredits per pageFree to run locally; you pay computeCredits, pay per event or your own infra
Skip it whenYou need clicks, crawling or searchYou only need to read pagesYou only need a few known URLs

Teneo Labs runs MCPScraper and makes Live Web Data API, a single-page content API. Its MCP server reads one public page per call and does not render JavaScript, crawl, search or extract structured fields. Rows checked against each project's docs on 2026-10-27.

Guides

The rubric and the guides built on it

Start with the rubric, then read the guide for your job. Every guide lists its sources and the date each row was checked.

All guides

FAQ

Questions about scraping MCP servers

What is the best MCP server for web scraping?

There is no single best one. Playwright MCP fits jobs that need clicks and forms, Firecrawl or Apify fit crawling many pages, Crawl4AI or Scrapling fit self-hosting, and a single-page content API fits reading known public pages. Decide the job first, then compare the shortlist on the rubric.

What is an MCP scraper?

An MCP server that gives an AI client such as Claude Code, Cursor or VS Code tools to fetch or read web pages. The client discovers the tools over the Model Context Protocol and the model calls them when it needs page content. Some drive a browser, some crawl sites, some return one page as text.

Do I need Playwright MCP to scrape websites?

Only if the job needs interaction: clicking, typing, logging in to your own test app or stepping through a flow. To read public pages, a browser server adds many tools and large page snapshots to the context. A content API or crawler is usually the lighter fit.

Why does Playwright MCP use so many tokens?

It returns an accessibility snapshot of the page after browser actions, and it registers about 25 core tools whose descriptions sit in the context. Both are useful for interaction and expensive for plain reading. See the output and token cost guide for ways to keep pages small.

Why compare by category instead of ranking servers?

Because a browser server, a crawl platform, a self-hosted crawler and a page-reading API answer different jobs. A single ranked list puts Playwright MCP and a two-tool reader on the same line, which hides the only choice that matters. The category decides the shortlist; the rubric rows decide inside it.

How is a scraping MCP server different from a scraping API?

A scraping API is called by your code over HTTP. A scraping MCP server wraps tools like that so an AI client can discover them and let the model call them. Many vendors ship both. For an agent, the MCP side adds tool descriptions to the context, so tool count and output size matter more than with a plain API.

Who runs MCPScraper, and does Live Web Data API render JavaScript?

Teneo Labs runs this site and makes Live Web Data API, which appears in the tables and is checked on the same rubric rows, including the rows where it falls short. Its MCP server reads one public page per call, live, as Markdown, text or HTML. The MCP tool does not render JavaScript (the REST API can, with its render option), and the product does not crawl sites, run searches or extract structured fields.

Blog

Rubric updates and head-to-heads

Release-driven rubric changes, head-to-heads and category explainers. Each post is dated and lists its sources.

All posts

Need one public page as clean text? Read the MCP docs.

A two-tool content API keeps the context small for known public pages. Sign-up is for businesses only; only Business, Enterprise and Custom sign-ups are reviewed before checkout. Plans are on the pricing page.

Read the MCP docsSee pricing