
Product
Secure Your AI-Generated Code with Socket MCP
Socket MCP brings real-time security checks to AI-generated code, helping developers catch risky dependencies before they enter the codebase.
@aigne/example-mcp-server-puppeteer
Advanced tools
A demonstration of using AIGNE Framework and Puppeteer MCP Server to extract content from websites using Puppeteer
This is a demonstration of using AIGNE Framework and Puppeteer MCP Server to extract content from websites using Puppeteer.
flowchart LR
in(In)
out(Out)
agent(AI Agent)
puppeteer(Puppeteer MCP Agent)
navigate(Navigate to URL)
evaluate(Evaluate JS)
in --> agent <--> puppeteer
subgraph MCP Agent
puppeteer <--> navigate
puppeteer <--> evaluate
end
agent --> out
classDef inputOutput fill:#f9f0ed,stroke:#debbae,stroke-width:2px,color:#b35b39,font-weight:bolder;
classDef processing fill:#F0F4EB,stroke:#C2D7A7,stroke-width:2px,color:#6B8F3C,font-weight:bolder;
class in inputOutput
class out inputOutput
class agent processing
class puppeteer processing
class navigate processing
class evaluate processing
Following is a sequence diagram of the workflow to summarize content from a website:
sequenceDiagram
participant User
participant AI as AI Agent
participant P as Puppeteer MCP Agent
participant N as Navigate to URL
participant E as Evaluate JS
User ->> AI: summarize content from https://www.arcblock.io
AI ->> P: extract content from https://www.arcblock.io
P ->> N: navigate to https://www.arcblock.io
N ->> P: navigation completed
P ->> E: evaluate document.body.innerText
E ->> P: content extracted
E ->> AI: extracted content as context
AI ->> User: The content is as follows: ...
export OPENAI_API_KEY=YOUR_OPENAI_API_KEY # setup your OpenAI API key
npx -y @aigne/example-mcp-server-puppeteer # run the example
git clone https://github.com/AIGNE-io/aigne-framework
cd aigne-framework/examples/mcp-server-puppeteer
pnpm install
Setup your OpenAI API key in the .env.local
file:
OPENAI_API_KEY="" # setup your OpenAI API key here
pnpm start
The following example demonstrates how to extract content from a website:
import assert from "node:assert";
import { AIAgent, OpenAIChatModel, ExecutionEngine, MCPAgent } from "@aigne/core";
const { OPENAI_API_KEY } = process.env;
assert(OPENAI_API_KEY, "Please set the OPENAI_API_KEY environment variable");
const model = new OpenAIChatModel({
apiKey: OPENAI_API_KEY,
});
const puppeteerMCPAgent = await MCPAgent.from({
command: "npx",
args: ["-y", "@modelcontextprotocol/server-puppeteer"],
});
const engine = new ExecutionEngine({
model,
tools: [puppeteerMCPAgent],
});
const agent = AIAgent.from({
instructions: `\
## Steps to extract content from a website
1. navigate to the url
2. evaluate document.body.innerText to get the content
`,
});
const result = await engine.call(agent, "extract content from https://www.arcblock.io");
console.log(result);
// output:
// {
// $message: "The content extracted from the website [ArcBlock](https://www.arcblock.io) is as follows:\n\n---\n\n**Redefining Software Architect and Ecosystems**\n\nA total solution for building decentralized applications ...",
// }
await engine.shutdown();
This project is licensed under the MIT License.
FAQs
A demonstration of using AIGNE Framework and Puppeteer MCP Server to extract content from websites using Puppeteer
The npm package @aigne/example-mcp-server-puppeteer receives a total of 3 weekly downloads. As such, @aigne/example-mcp-server-puppeteer popularity was classified as not popular.
We found that @aigne/example-mcp-server-puppeteer demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket MCP brings real-time security checks to AI-generated code, helping developers catch risky dependencies before they enter the codebase.
Security News
As vulnerability data bottlenecks grow, the federal government is formally investigating NIST’s handling of the National Vulnerability Database.
Research
Security News
Socket’s Threat Research Team has uncovered 60 npm packages using post-install scripts to silently exfiltrate hostnames, IP addresses, DNS servers, and user directories to a Discord-controlled endpoint.