express-robots.txt
Advanced tools
Comparing version
{ | ||
"name": "express-robots.txt", | ||
"version": "1.0.0", | ||
"version": "1.0.1", | ||
"description": "Express middleware to handle different robots.txt responses on different domains", | ||
@@ -5,0 +5,0 @@ "main": "index.js", |
@@ -1,2 +0,2 @@ | ||
## express-robots.txt | ||
### express-robots.txt | ||
@@ -7,35 +7,45 @@ Express middleware to handle different robots.txt responses on different domains. | ||
### Install | ||
#### Install | ||
Install with npm: `npm install express-robots.txt --save` | ||
Install with npm: | ||
``` | ||
npm install express-robots.txt --save | ||
``` | ||
### Setup | ||
`robotsTxt(allowedDomains[, allowedText] [, disallowedText]);` | ||
#### Setup | ||
A minimum of an array of domain names must be supplied. | ||
Optional the response text for allowed domains and disallowed domains can be passed. | ||
### Defaults | ||
``` | ||
robotsTxt(allowedDomains[, allowedText] [, disallowedText]); | ||
``` | ||
allowedText defaults to: | ||
Simplest usage: | ||
``` | ||
User-agent: * | ||
Disallow: | ||
var robotsTxt = require('express-robots.txt'); | ||
var robotsTxtHandler = robotsTxt(['domain.com', 'www.domain.com']); | ||
app.use(robotsTxtHandler); | ||
``` | ||
disallowedText defaults to: | ||
#### Defaults | ||
*allowedText* defaults to: | ||
``` | ||
User-agent: * | ||
Disallow: / | ||
Disallow: | ||
``` | ||
### Usage: | ||
*disallowedText* defaults to: | ||
``` | ||
var robotsTxt = require('enxpress-robots.txt'); | ||
var robotsTxtHandler = robotsTxt(['domain.com', 'www.domain.com']); | ||
app.use(robotsTxtHandler); | ||
User-agent: * | ||
Disallow: / | ||
``` |
3401
5.2%51
24.39%