Integration

WebScraping.AI integration

WebScraping.AI fetches public web pages through rotating datacenter, residential or stealth proxies, with headless Chrome JavaScript rendering by default, and returns the full HTML, the HTML of one CSS-selected area, or the main content as clean Markdown with the page title and description. It can also answer a free-text question about a page or extract named fields with an LLM, return parsed Google search results (with a free lookup of supported search locations), return structured data for public pages on supported sites such as YouTube, TikTok, X, LinkedIn, Instagram and Reddit, and report the account's remaining credits. Not exposed: the multi-selector route, whose answer is an unnamed array of arrays (call get_selected_html once per selector instead), the plain-text and XML variants of the page routes (every page tool answers JSON), and the proxy-server mode. Authentication is an API key sent as the api_key query parameter; calls are billed in credits per successful request, and failed requests are free.

Connect WebScraping.AI How integrations work

Ask it in chat

Say what should happen.
The workflow builds itself.

When this happens, do that in WebScraping.AI.Built from the 9 WebScraping.AI tools below, on the connection you add once.

What you can do with WebScraping.AI

Each of these is one operation on one endpoint, available both as a workflow step and as a tool an AI agent can call on its own. Nothing bundles several operations into one action, so what a run did stays readable afterwards.

Most endpoints run on the free plan. A few, mainly the ones that publish on your behalf, need a paid plan.

Page, 3 operations

ask_page_questionAsk page questionGETFetches a web page (through a proxy, with headless Chrome JavaScript rendering unless js is false) and has an LLM answer a free-text question or instruction about its content, returned as plain text in result.
get_page_htmlGet page htmlGETFetches the full HTML of a web page through a rotating proxy, rendering JavaScript in headless Chrome unless js is false, and returns it as a string in result.
get_selected_htmlGet selected htmlGETFetches a web page and returns, in result, the inner HTML of the first element matching a CSS selector, so only that area of the page has to be parsed. If nothing matches, the call fails with a 400 Element not found.

Account, 1 operation

get_accountGet accountGETReturns the account's remaining credits (monthly plan, pay-as-you-go and total), when the next billing cycle starts, and how many concurrent requests remain.

Data, 1 operation

get_site_dataGet site dataGETReturns structured JSON for a public page on a supported site (for example a YouTube video, a TikTok profile, an X post, a LinkedIn company, an Instagram reel or a Reddit thread), detected from the page's normal URL.

Field, 1 operation

extract_page_fieldsExtract page fieldsGETFetches a web page (through a proxy, with headless Chrome JavaScript rendering unless js is false) and has an LLM extract the fields you name, returned under result keyed by your field names; a field the page does not contain comes back null.

Location, 1 operation

search_serp_locationsSearch serp locationsGETSearches the locations search_google accepts as its location parameter: Google's geo targets (cities, districts, regions, states and countries), most populous first. Pass a result's canonical_name or id to search_google.

Serp, 1 operation

search_googleSearch googleGETRuns a Google search and returns the parsed results: organic results (position, title, link, domain, displayed link, snippet, date), related searches, spelling-correction details and pagination.

Text, 1 operation

get_page_textGet page textGETFetches a web page and converts its main content to clean Markdown (boilerplate stripped; headings, lists and links kept; table cells come through as plain paragraphs), returned with the page title and description.

Use WebScraping.AI from Claude Code, Cursor or Codex

Connect WebScraping.AI in LiveContext, create an API key in Settings, then add the LiveContext MCP server to your client. Its WebScraping.AI operations become tools the client can call, next to every other integration you connected.

Claude Code

claude mcp add --transport http livecontext https://livecontext.ai/mcp --header "X-API-Key: YOUR_API_KEY"

Codex (~/.codex/config.toml)

[mcp_servers.livecontext]
url = "https://livecontext.ai/mcp"

[mcp_servers.livecontext.http_headers]
"X-API-Key" = "YOUR_API_KEY"

Cursor, Claude Desktop and other JSON configs

{
  "mcpServers": {
    "livecontext": {
      "type": "http",
      "url": "https://livecontext.ai/mcp",
      "headers": {
        "X-API-Key": "YOUR_API_KEY"
      }
    }
  }
}

Set up the MCP server step by step

How to connect WebScraping.AI

Create your workspace
Sign up free. Nothing to install.
Connect WebScraping.AI once
Add your WebScraping.AI connection in Settings. Every tool from this integration reuses it, and secrets are encrypted at rest and visible only to you.
Describe the job in chat
The workflow builds itself in front of you and picks the endpoints it needs. Drag the steps yourself on the canvas whenever you would rather be explicit.