You're Invited:Meet the Socket Team at BlackHat and DEF CON in Las Vegas, Aug 7-8.RSVP
Socket
Socket
Sign inDemoInstall

@nodebb/spider-detector

Package Overview
Dependencies
0
Maintainers
4
Versions
2
Alerts
File Explorer

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

@nodebb/spider-detector

A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS


Version published
Maintainers
4
Created

Readme

Source

spider-detector

Build Status npm version

A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS

It might be useful when you have a single page app but want to deliver static pages for spiders.

Install

npm install spider-detector // or `yarn install spider-detector`

Direct Example

const detector = require('spider-detector')
detector.isSpider('baiduspider') // return true

ExpressJS example

const detector = require('spider-detector')
const express = require('express')
const app = express()

app.use(detector.middleware())

app.get('/*', function(req, res) {
  if (req.isSpider()) {
    // do something else, i.E. send a static page
  } else {
    // send single page app
  }
})

Why? There are already modules out there

Well, I wanted one which does not use readFileSync and comes with optional middleware. Furthermore some hackers do not classify Googlebot as a spider anymore which poses a problem sometimes, see next question.

What about Googlebot?

Yep, Googlebot is able to deal with single page apps but this feature is pretty unstable. Especially under AngularJS when hash fragments are disabled with $locationProvider.html5Mode(true). That's why - against all odds - I have classified googlebot as a spider in this module.

Keywords

FAQs

Package last updated on 10 Jun 2024

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc