Security News
OpenJS: “XZ Utils Cyberattack Likely Not an Isolated Incident”
OpenJS is warning of social engineering takeovers targeting open source projects after receiving a credible attempt on the foundation.
es6-crawler-detect
Advanced tools
Weekly downloads
Readme
This Library is an ES6 version of the original PHP class @CrawlerDetect, it helps you detect bots/crawlers and spiders only by scanning the user-agent string or from the global request.headers
.
npm install es6-crawler-detect
'use strict';
const express = require('express')
const { Crawler, middleware } = require('es6-crawler-detect')
const app = express()
app.get('your/route', function async (request, response) {
// create a new Crawler instance
var CrawlerDetector = new Crawler(request)
// check the current visitor's useragent
if ( CrawlerDetector.isCrawler() )
{
// true if crawler user agent detected
}
// or check a user agent string
if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )
{
// true if crawler user agent detected
}
// Output the name of the bot that matched (if any)
response.send(CrawlerDetector.getMatches())
})
/**
* Or by using the middleware
*/
app.use(middleware((request, reponse) => {
// do something here
// e.g. console.log(request.body)
// e.g. return response.status(403).send('Forbidden')
}))
app.get('/crawler', function async (request, response) {
// or check a user agent string
request.Crawler.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')
// Output the name of the bot that matched (if any)
response.send(request.Crawler.getMatches())
})
<head>
<!-- Metas -->
<meta name="title" content="ES6-CrawlerDetect">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="keywords" content="">
<title>CrawlerDetect - the web crawler detection library</title>
<!-- Scripts -->
<script type="text/javascript" src="./your/path/to/main.bundle.js"></script>
</head>
// create a new Crawler instance
var CrawlerDetector = new Crawler.Crawler();
var userAgentString = navigator.userAgent;
// check the current visitor's useragent
if ( CrawlerDetector.isCrawler(userAgentString) )
{
// true if crawler user agent detected
}
// Output the name of the bot that matched (if any)
console.debug(CrawlerDetector.getMatches());
If you find a bot/spider/crawler user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the data
array in ./crawler/crawlers.js
.
FAQs
This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.
The npm package es6-crawler-detect receives a total of 14,041 weekly downloads. As such, es6-crawler-detect popularity was classified as popular.
We found that es6-crawler-detect demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
OpenJS is warning of social engineering takeovers targeting open source projects after receiving a credible attempt on the foundation.
Company News
Come meet the Socket team at BSidesSF and RSA! We're sponsoring several fun networking events and we would love to see you there.
Security News
OSI is starting a conversation aimed at removing the excuse of the SaaS loophole for companies navigating licensing and the complexities of doing business with open source.