A node.js powered scrapper 🔥 that iterates trough all the internal links of the specified url.
It works on CSR pages (React, Angular) with dynamic urls.
Once it is done it generates a sitemap.xml
file with all the urls found, ready to be uploaded to Google Search Console.
Usage:
$ sitemap https://vvlog.dev
Params:
Parameter | type | default | description |
---|
--wait | integer | 1500 | Specify the time (milliseconds) to wait (So the fetches are completed) before starting to parse the page. |
--limit | integer | 999999 | Specify the limit of urls to parse before stopping the scrapper. |
Todo: