Product
Introducing Enhanced Alert Actions and Triage Functionality
Socket now supports four distinct alert actions instead of the previous two, and alert triaging allows users to override the actions taken for all individual alerts.
web-auto-extractor
Advanced tools
Readme
Automatically extracts semantically structured information from any HTML webpage.
Supported formats:-
Demo it on tonicdev
npm install web-auto-extractor
Sample code:-
import parseWeb from 'web-auto-extractor'
import request from 'request'
request(pageUrl, function (error, response, body) {
const data = parseWeb(body)
console.log(data.micro) // Microdata Result
//console.log(data.rdfa) // RDFa-Lite Result
//console.log(data.jsonld) // JSON-LD Result
//console.log(data.meta) // Meta tags Result
})
CommonJS import style:-
var parse = require('web-auto-extractor').default
Output:-
The output will be a JSON in JSON-LD format
[
{
"@context": "http://schema.org/",
"@type": "Product",
"brand": "ACME",
"name": "Executive Anvil",
"image": "anvil_executive.jpg",
"description": "Sleeker than ACME's Classic Anvil, the\n Executive Anvil is perfect for the business traveler\n looking for something to drop from a height.",
"mpn": "925872",
"aggregateRating": {
"@context": "http://schema.org/",
"@type": "AggregateRating",
"ratingValue": "4.4",
"reviewCount": "89"
},
"offers": {
"@context": "http://schema.org/",
"@type": "Offer",
"priceCurrency": "USD",
"price": "119.99",
"priceValidUntil": "5 November!",
"seller": {
"@context": "http://schema.org/",
"@type": "Organization",
"name": "Executive Objects"
},
"itemCondition": "http://schema.org/UsedCondition",
"availability": "http://schema.org/InStock"
}
}
]
You can also pass in a configuration object to the function
const config = {
normalize: false,
withSelector: true
}
const data = parseWeb(html, config)
default: false
Set to true if you'd want the result to include the selector
object.
The selector
object provides you with two fields:-
select
: The css-selector of the HTMLElementextract
: The HTMLElement property from which the value
object was extracted from.[
{
"@context": "http://schema.org/",
"@type": "Product",
"image": {
"value": "anvil_executive.jpg",
"selector": {
"select": "[itemtype=\"http://schema.org/Product\"]:eq(0) [itemprop=\"image\"]:eq(1)",
"extract": {
"attr": "src"
}
}
},
"name": {
"value": "Executive Anvil",
"selector": {
"select": "[itemtype=\"http://schema.org/Product\"]:eq(0) [itemprop=\"name\"]:eq(1)",
"extract": {
"attr": "@text"
}
}
},
...
...
]
default: true
Set to false if you'd want to work with the intermediate non-normalized result.
See relevant output in test case.
//The keys are the md5 hash of the respective HTML element
{
"ed08397308d9c31da5e50485f2dfe184": {
"context": "http://schema.org/",
"type": "Product",
"name": "Product",
"value": null,
"properties": {
"brand": [
"e3abccbeed2389fd64e4fe57439c4ab6"
],
"name": [
"8da2f7f6cb7e420b13442c075d4a1a17"
],
...
...
},
"parentTypeId": null
},
"e3abccbeed2389fd64e4fe57439c4ab6": {
"name": "brand",
"value": "ACME",
"properties": {},
"parentTypeId": "ed08397308d9c31da5e50485f2dfe184"
},
"8da2f7f6cb7e420b13442c075d4a1a17": {
"name": "name",
"value": "Executive Anvil",
"properties": {},
"parentTypeId": "ed08397308d9c31da5e50485f2dfe184"
},
...
...
}
}
...
}
See test cases for more examples.
FAQs
Automatically extracts structured information from webpages
The npm package web-auto-extractor receives a total of 6,418 weekly downloads. As such, web-auto-extractor popularity was classified as popular.
We found that web-auto-extractor demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket now supports four distinct alert actions instead of the previous two, and alert triaging allows users to override the actions taken for all individual alerts.
Security News
Polyfill.io has been serving malware for months via its CDN, after the project's open source maintainer sold the service to a company based in China.
Security News
OpenSSF is warning open source maintainers to stay vigilant against reputation farming on GitHub, where users artificially inflate their status by manipulating interactions on closed issues and PRs.