Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
@zachleat/spider-pig
Advanced tools
Readme
Get a list of local URL links from a root URL. Works with JavaScript generated content. Can also act as a live-DOM CSS search across multiple files (find all the templates that are using the CSS selector I want to change).
<a href="test.html">Test</a><!-- match -->
<a href="test2.html">Test</a><!-- match -->
<a href="test2.html">Duplicate Test</a><!-- do not match -->
<a href="root.html">URL to self</a><!-- match -->
<a href="mailto:test@example.com">Email link</a><!-- do not match -->
<a href="http://www.google.com/">External</a><!-- do not match -->
Normalizes all of the matching URLs to be full absolute URLs (including host and protocol and path, etc).
Available on npm.
npm install @zachleat/spider-pig
$ spiderpig http://zachleat.localhost/web/
http://zachleat.localhost/web/
http://zachleat.localhost/web/about/
http://zachleat.localhost/web/best-of/
…
$ spiderpig http://zachleat.localhost/web/ --filter="/about/"
http://zachleat.localhost/web/about/
Useful to see where CSS changes might regress on a project. This takes a bit of time. You can also mix in --filter
here too.
$ spiderpig http://zachleat.localhost/web/ --selector=".header"
Found 180 urls.
Looking for urls with CSS selector ".header":
http://zachleat.localhost/web/ ✅ Yes
http://zachleat.localhost/web/about/ ✅ Yes
http://zachleat.localhost/web/best-of/ ✅ Yes
http://zachleat.localhost/web/projects/ ✅ Yes
Use --selectorlimit
to set an upper bound on the number of URLs that get spidered.
$ spiderpig http://zachleat.localhost/web/ --selector=".header" --selectorlimit=3
Found 180 urls
Looking for urls with CSS selector ".header" (limit 3):
http://zachleat.localhost/web/ ✅ Yes
http://zachleat.localhost/web/about/ ✅ Yes
http://zachleat.localhost/web/best-of/ ✅ Yes
…
$ DEBUG=SpiderPig spiderpig http://zachleat.localhost/web/
const SpiderPig = require("@zachleat/spider-pig");
(async function() {
let sp = new SpiderPig();
await sp.start();
let urls = await sp.fetchLocalUrls("http://localhost/myproject/");
// Optional, filter (case sensitive)
urls = sp.filterUrls(urls, "views");
})();
const SpiderPig = require("@zachleat/spider-pig");
(async function() {
let sp = new SpiderPig();
let selector = ".test-css-selector:nth-child(2)";
await sp.start();
let urls = await sp.fetchLocalUrls("http://localhost/myproject/");
for(let url of urls) {
if( await sp.hasSelector(url, selector) ) {
// has it
} else {
// doesn’t have it
}
}
})();
FAQs
Get a list of local URL links from a root URL.
The npm package @zachleat/spider-pig receives a total of 499 weekly downloads. As such, @zachleat/spider-pig popularity was classified as not popular.
We found that @zachleat/spider-pig demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.