
Product
Announcing Precomputed Reachability Analysis in Socket
Socket’s precomputed reachability slashes false positives by flagging up to 80% of vulnerabilities as irrelevant, with no setup and instant results.
disallow-ai
Advanced tools
robots.txt
snippet.Opt your web properties out from known AI scraper tools.
yarn add disallow-ai
or npm i disallow-ai
See below for API.
This is an opinionated, maintained list of known user agents of scraper bots that use web content to train AI models.
This package is intended to help webmasters automatically opt out of training AI/machine learning models with the content of a property. Its intention is to remain visible for search engines and productivity tools. It's optimized for Node.js servers (e.x., Express/Next.js) but you can also copy-paste contents from src/robots.txt
directly into your robots.txt
file on any web server.
printRobotsTXT(options)
prints text string for robots.txt
(same content as the above snippet).
options.path
if you want to set a disallow path to something other than /
, you'll need to pass a value to this key.userAgents
is a direct reference to an array of objects with all the user agent info.const express = require("express");
const server = express();
const { printRobotsTXT } = require("disallow-ai");
server.get("/robots.txt", (req, res, next) => {
res.type("text/plain");
res.send(printRobotsTXT());
});
server.listen();
You can run an example server with node ./example/server.js
.
Contributions welcome.
FAQs
Opt your web properties out from known AI scraper tools.
The npm package disallow-ai receives a total of 0 weekly downloads. As such, disallow-ai popularity was classified as not popular.
We found that disallow-ai demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket’s precomputed reachability slashes false positives by flagging up to 80% of vulnerabilities as irrelevant, with no setup and instant results.
Product
Socket is launching experimental protection for Chrome extensions, scanning for malware and risky permissions to prevent silent supply chain attacks.
Product
Add secure dependency scanning to Claude Desktop with Socket MCP, a one-click extension that keeps your coding conversations safe from malicious packages.