pentesterland-writeups-scraper

Installation

npm install -S pentesterland-writeups-scraper

Usage

ES6

import load from 'pentesterland-writeups-scraper'

const options = { year: '2020' } // see details below
const getWriteups = async () => await load(options)

const writeups = getWriteups()

// do something with writeups
console.log(writeups)

CommonJS

const load = require('pentesterland-writeups-scraper')

const options = { group: true } // see details below

load(options).then(function (writeups) {
  // do something with writeups
  console.log(writeups)
})

Options

• group - true or false - returns writeups grouped by year ( object instead of array )

{
  '2020': [...],
  '2019': [...],
  '2018': [...],
  ...
  '2012': [...],
  unknown: [...]
}

• year - YYYY - returns list of writeups for specified year. Can be unknown, to get all writeups without date.
• month - MM - returns list of writeups for specified month (year option is required)
• order - asc or desc - returns list of writeups in asc or desc order

Schema

{
  url: 'https://blog.assetnote.io/bug-bounty/2019/03/19/rce-on-mozilla-zero-day-webpagetest/',
  domain: 'blog.assetnote.io',
  title: 'Discovering a zero day and getting code execution on Mozilla’s AWS Network',
  hackers: [
    { name: 'Shubham Shah', twitter_handle: 'infosec_au' },
    { name: 'Mathias Karlsson', twitter_handle: 'avlidienbrunn' }
  ],
  programs: [ 'Mozilla' ],
  weaknesses: [ 'RCE' ],
  bounty: 500,
  published: { year: '2019', month: '03', day: '19' },
  published_at: '2019-03-19'
}

In some cases there's a few programs, hackers and weaknesses attached to a single writeup, so i decided to go with array type for field, to have consistent format across all writeups.

Each writeup may also have additional properties, such as:

• status - informative, not_applicable, duplicate or out_of_scope
• swag - true or false
• vdp - true or false
• hall_of_fame - true or false

hacker object may also have website property.