next-aeo

🧠 next-aeo

This package generates a llms.txt file that summarizes your Next.js app's content for use by language models. It scans your project for informative pages (like marketing, legal, and pricing pages) and outputs a structured Markdown file containing high-level summaries.

🚀 Usage

Before running next-aeo, you must build your Next.js app first. In the root of your project, run your build script e.g.

npm run build

Then, run the following command whenever there are significant changes to your site:

npx -y next-aeo@latest

The generated llms.txt file is saved to public/llms.txt, making it accessible at /llms.txt on your website.

If you want to make generating an llms.txt file easier, you can also add it as a script to your package.json file and specify that the build script should be run before the llms script.

"scripts": {
  "prellms": "npm run build",
  "llms": "npx -y next-aeo@latest"
}

👇 Keep reading to learn more about the llms.txt standard and how it works.

🤔 Why llms.txt?

The llms.txt file is an emerging standard that helps AI systems better understand your website's content and purpose. By providing a structured summary of your site, you can:

• Improve AI interactions: Help chatbots and AI assistants provide more accurate information about your product or service
• Enhance content discovery: Make it easier for AI-powered search engines to understand and index your content
• Enable better integrations: Allow AI tools to provide contextually relevant suggestions and answers
• Future-proof your site: Prepare for the growing ecosystem of AI-powered web tools and services

Learn more about the llms.txt standard:

• llms.txt specification - The proposed specification and guidelines
• llms.txt overview - An overview of the llms.txt standard by LangGraph
• MCP example - A real-world example of an llms.txt file by Anthropic

✨ Features

• ✅ Smart content extraction: Automatically identifies and extracts meaningful content from your pages
• 🧹 Noise filtering: Skips non-content pages (404, 500) and removes navigation, ads, and other clutter
• 🚀 Performance optimized: Processes files in batches for efficient handling of large sites
• 🧠 AI-powered summarization: Uses advanced language models to create concise, relevant summaries
• 📄 Standard compliant: Outputs to public/llms.txt following the llms.txt specification
• 🔧 Zero configuration: Works out of the box with any Next.js project

🏗️ How It Works

• Scans your Next.js build: Analyzes HTML files in your .next/server directory
• Filters content: Removes non-essential elements like scripts, styles, navigation, and ads
• Converts to Markdown: Transforms HTML content into clean, readable Markdown
• Generates summaries: Uses AI to create concise summaries of each page's content
• Creates llms.txt: Outputs a structured file that AI systems can easily understand

📄 What Gets Included

Included pages:

• Marketing pages (home, about, features)
• Product information pages
• Pricing and plans
• Legal pages (privacy, terms)
• Documentation and help content
• Blog posts and articles

Automatically excluded:

• Error pages (404, 500, _not-found)
• Navigation menus and headers
• Footers and sidebars
• Advertisements and promotional banners
• Scripts and stylesheets
• SVG icons and decorative elements

🐛 Debug Mode

To see detailed information about the processing:

Custom Models

You can use a custom OpenAI model by passing the --openai-key and --openai-model options.

npx -y next-aeo@latest --openai-key <your-api-key> --openai-model <your-model>

npx -y next-aeo@latest --debug

🔗 Related Resources

• Next.js documentation
• Next.js deployment guide

Made with ❤️ and 🤖 by Profound