Let Your AI Browse the Web and Scrape Pages with MCP
Two MCP servers cover web access: Fetch (for simple URL fetching and content extraction) and Puppeteer (for full browser control, including JavaScript-heavy pages, screenshots, and form interaction). Together they make your AI a capable web agent.
Servers You'll Need
Fetch MCP Server
OfficialFetch any web page or API endpoint and pass the content to your AI
The official Fetch MCP server lets your AI assistant retrieve content from any URL. It fetches web pages, converts HTML …
Puppeteer MCP Server
OfficialLet your AI control a real browser — screenshot, click, and scrape
The official Puppeteer MCP server gives your AI assistant full control of a Chrome browser via Puppeteer. Navigate to UR…
Step-by-Step Setup
- 1
Choose your server based on need
Use Fetch for simple pages and documentation. Use Puppeteer when the page requires JavaScript to render, needs a login, or you want screenshots.
- 2
Install Fetch
`npm install -g @modelcontextprotocol/server-fetch`
- 3
Install Puppeteer server
`npm install -g @modelcontextprotocol/server-puppeteer`. Puppeteer downloads Chromium automatically on first run.
- 4
Add both to your config
Add both servers to your AI client config. They work independently — use whichever is more appropriate for a given task.
Example Prompts to Try
- “Fetch the content of this documentation page and summarize the API”
- “Take a screenshot of example.com and describe what you see”
- “Scrape the pricing table from this competitor's website”
- “Check if this URL returns a 200 status code”
- “Fill in and submit this form on the test site”
Tips
- Fetch is faster and lighter — prefer it for static content.
- For Puppeteer tasks, be specific about what you want captured (text, screenshot, specific element).
- Web scraping may violate a site's terms of service — check before automating.
FAQ
Can the AI log into a site using Puppeteer?
Yes, Puppeteer can fill forms and click buttons. You'd tell the AI the credentials (or have them in a config). Be careful with this — only use it on sites you own or have permission to automate.
Does Fetch work with paywalled content?
No. Fetch makes an unauthenticated HTTP request. You'd need Puppeteer with a logged-in session for paywalled content.