crawler-url-parser
Advanced tools
Readme
An URL parser for crawling purpose.
npm install crawler-url-parser
const cup = require('crawler-url-parser');
let url = cup.parse("../ddd","http://question.stackoverflow.com/aaa/bbb/ccc/");
console.log(url.normalized);
console.log(url.host);
console.log(url.domain);
console.log(url.subdomain);
console.log(url.protocol);
console.log(url.path);
const cup = require('crawler-url-parser');
let htmlStr=
'html> \
<body> \
<a href="http://www.stackoverflow.com/internal-1">test-link-4</a><br /> \
<a href="http://www.stackoverflow.com/internal-2">test-link-5</a><br /> \
<a href="http://www.stackoverflow.com/internal-2">test-link-6</a><br /> \
<a href="http://faq.stackoverflow.com/subdomain-1">test-link-7</a><br /> \
<a href="http://faq.stackoverflow.com/subdomain-2">test-link-8</a><br /> \
<a href="http://faq.stackoverflow.com/subdomain-2">test-link-9</a><br /> \
<a href="http://www.google.com/external-1">test-link-10</a><br /> \
<a href="http://www.google.com/external-2">test-link-11</a><br /> \
<a href="http://www.google.com/external-2">test-link-12</a><br /> \
</body> \
</html>';
let currentUrl= "http://www.stackoverflow.com/aaa/bbb/ccc";
let result = cup.extract(htmlStr,currentUrl);
console.log(result.length);
mocha
or
npm testFAQs
An `URL` parser for crawling purpose.
The npm package crawler-url-parser receives a total of 298 weekly downloads. As such, crawler-url-parser popularity was classified as not popular.
We found that crawler-url-parser demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket now supports four distinct alert actions instead of the previous two, and alert triaging allows users to override the actions taken for all individual alerts.
Security News
Polyfill.io has been serving malware for months via its CDN, after the project's open source maintainer sold the service to a company based in China.
Security News
OpenSSF is warning open source maintainers to stay vigilant against reputation farming on GitHub, where users artificially inflate their status by manipulating interactions on closed issues and PRs.