@marbec/web-auto-extractor

Web Auto Extractor 2.0

This project is a fork of indix/web-auto-extractor.

Parse semantically structured information from any HTML webpage.

Supported formats:

• Encodings that support Schema.org vocabularies:
  • Microdata
  • RDFa-lite
  • JSON-LD
• Meta tags

Popularly, many websites mark up their webpages with Schema.org vocabularies for better SEO. This library helps you parse that information to JSON.

Installation

npm i --save @marbec/web-auto-extractor

Usage

import WebAutoExtractor from '@marbec/web-auto-extractor';

const parsed = new WebAutoExtractor({
  // Add location information to the root elements in the parsed data.
  // Location is stored as start,end offset values in the @location property.
  addLocation: false,

  // Embed the source HTML in the root elements in the parsed data using the @source property.
  // This property is either a boolean to embed sources for all data types or an array of data types to embed sources for.
  embedSource: false,
}).parse(sampleHTML);

// Output format
/* {
    "metatags": {},
    "microdata": {},
    "rdfa": {},
    "jsonld": {}
} */

Browser

You can run the parser directly in the browser on any website using the following commands:

const { default: WebAutoExtractor } = await import(
  'https://unpkg.com/@marbec/web-auto-extractor@latest/dist/index.js'
);
new WebAutoExtractor().parse(document.documentElement.outerHTML);

Examples

See test cases for sample in- and outputs.