static-sitemap-cli - npm Package Compare versions

Comparing version 0.2.0 to 1.0.0

package.json

 {
   "name": "static-sitemap-cli",
   "description": "Simple CLI to pre-generate XML sitemaps for static sites locally.",
-  "version": "0.2.0",
+  "version": "1.0.0",
   "author": "Jason Lee <[e-mail]>",
@@ -15,8 +15,14 @@ "bin": {
     "@oclif/plugin-help": "^2.2.0",
+    "fast-glob": "^3.0.4",
     "get-stdin": "^7.0.0",
-    "globby": "^10.0.1",
-    "minimatch": "^3.0.4",
-    "sitemap": "^3.2.2"
+    "js2xmlparser": "^4.0.0",
+    "micromatch": "^4.0.2"
   },
-  "devDependencies": {},
+  "devDependencies": {
+    "@oclif/test": "^1.2.5",
+    "chai": "^4.2.0",
+    "eslint": "^6.1.0",
+    "mocha": "^6.2.0",
+    "nyc": "^14.1.1"
+  },
   "engines": {
@@ -29,3 +35,3 @@ "node": ">=8.0.0"
   ],
-  "homepage": "https://github.com/zerodevx/static-sitemap-cli",
+  "homepage": "https://npmjs.com/package/static-sitemap-cli",
   "keywords": [
@@ -44,4 +50,5 @@ "XML",
   "scripts": {
-    "test": "echo WIP! && exit 1"
+    "posttest": "eslint .",
+    "test": "nyc mocha --forbid-only \"test/**/*.test.js\""
   }
 }

README.md

 ![npm](https://img.shields.io/npm/v/static-sitemap-cli)
-![npm](https://img.shields.io/npm/dw/static-sitemap-cli)
+![npm](https://img.shields.io/npm/dm/static-sitemap-cli)
 
@@ -20,3 +20,3 @@ # static-sitemap-cli
 
-Syntax: `static-sitemap-cli <BASEURL> <options>`
+Syntax: `static-sitemap-cli <BASEURL> [options]`
 
@@ -53,9 +53,11 @@ At its simplest, just go to your `dist` folder and run:
 | -r     | --root       | [default: current dir] root directory to start from             |
-| -m     | --match      | [default: **/*.html] list of globs to match                     |
-| -i     | --ignore     | [default: 404.html] list of globs to ignore                     |
-| -p     | --priority   | comma-separated glob/priority pair; eg: foo/*.html,0.1          |
-| -f     | --changefreq | comma-separated glob/changefreq pair; eg: foo/*.html,daily      |
+| -m     | --match      | [default: **/*.html,!404.html] list of globs to match           |
+| -p     | --priority   | glob-priority pair (eg: foo/*.html=0.1)                         |
+| -f     | --changefreq | glob-changefreq pair (eg: foo/*.html=daily)                     |
 | -n     | --no-clean   | disable clean URLs                                              |
 | -s     | --slash      | add trailing slash to all URLs                                  |
+| -t     | --text       | output as .TXT instead                                          |
+| -v     | --verbose    | be more verbose                                                 |
 
+
 #### Clean URLs
@@ -65,11 +67,14 @@
 
-`https://example.com/foo/index.html` becomes `https://example.com/foo`.
+`rootDir/foo/index.html` becomes `https://example.com/foo`.
 
+`rootDir/foo/bar/foobar.html` becomes `https://example.com/foo/bar/foobar`.
+
 Pass `-n` option to disable this behavior.
 
+
 #### Trailing Slashes
 
-Control whether or not URLs should include trailing slashes. For example:
+Controls whether or not URLs should include trailing slashes. For example:
 
-`https://example.com/bar/index.html` becomes `https://example.com/bar/`.
+`rootDir/bar/index.html` becomes `https://example.com/bar/`.
 
@@ -79,58 +84,98 @@ For obvious reasons, this cannot be used together with `-n`.
 
+#### Ignore Some Files
+
+The `-m` flag allows multiple entries to be input. By default it's set to the following globs: `**/*.html` and `!404.html`.
+You can change the glob pattern matches to suit your use-case, like:
+
+`sscli https://example.com -m '**/*.html' -m '!404.html' -m '!**/ignore/**' -m '!this/other/specific/file.html'`
+
+
+#### Glob-* Pairs
+
+The `-p` and `-c` flags allow multiple entries and accept `glob-*` pairs as input. A `glob-*` pair is basically
+`<glob-pattern>=<value>`, where `=` separates the two. For example, a glob-frequency pair should be input as
+`events/**/*.html=daily`.
+
+Entries input later will override the earlier ones. So for example in this,
+
+`sscli https://example.com -f '**/*=weekly' -f 'events/**=daily'`
+
+all page entries will contain `<changefreq>weekly</changefreq>` while pages that match `event/**` will contain
+`<changefreq>daily</changefreq>`.
+
+
+#### Output as Text
+
+Sitemaps can be generated in a simple [text file](https://support.google.com/webmasters/answer/183668?hl=en) format as well,
+where each line contains exactly one URL. Pass the option `-t` to do so. In this case, `--priority` and `--changefreq`
+are redundant.
+
+
 ## Examples
 
-#### Create sitemap for `dist/` folder
+#### Create sitemap for `dist` folder
 
-```
-static-sitemap-cli https://example.com -r dist/ > dist/sitemap.xml
-```
+`static-sitemap-cli https://example.com -r dist > dist/sitemap.xml`
 
 OR
 
-```
-sscli https://example.com -r dist/ > dist/sitemap.xml
-```
+`sscli https://example.com -r dist > dist/sitemap.xml`
 
-Note: Just put `dist/` for that location, not `dist/.` or `./dist/**`.
 
 #### Ignore a bunch of files
 
-```
-sscli https://example.com -i=404.html foo/**/* > sm.xml
-```
+`sscli https://example.com -m '**/*.html' '!404.html' '!**/ignore/**' '!this/other/specific/file.html' > sm.xml`
 
+
 #### Set priority of certain pages
 
 By default, the optional `<priority>` label ([protocol reference](https://www.sitemaps.org/protocol.html)) is excluded,
-so every pages' default is 0.5. To change the *relative* priority (to 0.1) of certain pages:
+so every pages' default is 0.5. To change the *relative* priority of certain pages:
 
-```
-sscli https://example.com -p=**/{foo,bar}/**,0.1 **/important/**,0.9 > sm.xml
-```
+`sscli https://example.com -p '**/{foo,bar}/**=0.1' '**/important/**=0.9' > sm.xml`
 
+
+#### Set changefreq of all pages to weekly, and some to daily
+
+`sscli https://example.com -f '**/*=weekly' -f 'events/**=daily' > sm.xml`
+
+
 #### Pipe in the base URL
 
-```
-echo https://example.com | sscli > sm.xml
-```
+`echo https://example.com | sscli > sm.xml`
 
 
+
 ## To-do
 
-Add tests! :sweat_smile:
+~~Add tests! :sweat_smile:~~
 
 
+## Tests
+
+Run `npm run test`.
+
+
 ## Changelog
 
-**v0.2.0 - 2019-07-31:**
+**v1.0.0** - 2019-08-15:
+* **BREAKING:** `--ignore` is deprecated. Use `--matches` instead.
+* **BREAKING:** Glob-* pairs are no longer comma-seperated. Use `=` instead.
+* **BREAKING:** Logic for multiple glob-* pairs changed. Later pairs override the earlier ones now.
+* Major refactor of original codebase; discontinued usage of [globby](https://www.npmjs.com/package/globby) and [sitemap](https://www.npmjs.com/package/sitemap) in favour of [fast-glob](https://www.npmjs.com/package/fast-glob), [micromatch](https://www.npmjs.com/package/micromatch), and [js2xmlparser](https://www.npmjs.com/package/js2xmlparser).
+* Resulting code should be much easier to reason with and maintain now.
+* Add feature to output as text (one URL per line).
+* Add verbose mode to see some console feedback.
+* And finally, add tests with ~95% coverage.
+
+**v0.2.0** - 2019-07-31:
 * Allow BASEURL to be piped in also.
 * Refactor some dependencies.
 
-**v0.1.1 - 2019-07-27:**
+**v0.1.1** - 2019-07-27:
 * Bugfix: properly check rootDir before replacing.
 * Add new alias `sscli` because the original is quite a mouthful.
 
-**v0.1.0 - 2019-07-26:**
+**v0.1.0** - 2019-07-26:
 * Initial release.
 * Built in 10 minutes. :stuck_out_tongue_winking_eye:
-

src/index.js

 const {Command, flags} = require('@oclif/command');
 const getStdin = require('get-stdin');
-const globby = require('globby');
-const minimatch = require('minimatch');
-const sitemap = require('sitemap');
+const fg = require('fast-glob');
+const mm = require('micromatch');
+const parser = require('js2xmlparser');
 
-
 class StaticSitemapCliCommand extends Command {
@@ -16,26 +15,59 @@
       if (!argv.length) {
-        this.error('you must include a BASEURL - type "sscli --help" for help.', { code: 'BASEURL_NOT_FOUND', exit: 1 });
+        this.error('you must include a BASEURL - type "sscli --help" for help.', {
+          code: 'BASEURL_NOT_FOUND',
+          exit: 1
+        });
       }
       baseUrl = argv[0];
     }
-    baseUrl = baseUrl.slice(-1) === '/' ? baseUrl.slice(0, -1) : baseUrl;
-    let rootDir = flags.root || '';
-    if (rootDir.length) {
-      rootDir = rootDir.slice(-1) === '/' ? flags.root : flags.root + '/';
+
+    const addSlash = path => path.slice(-1) === '/' ? path : `${path}/`;
+    const getUrl = path => {
+      let url = baseUrl + path;
+      if (!flags['no-clean']) {
+        if (url.slice(-11) === '/index.html') {
+          url = url.slice(0, -11);
+        } else if (url.slice(-5) === '.html') {
+          url = url.slice(0, -5);
+        }
+      }
+      if (flags.slash) {
+        url = url + '/';
+      }
+      return url;
+    };
+
+    baseUrl = addSlash(baseUrl);
+
+    const files = await fg(flags.match, {
+      cwd: flags.root,
+      stats: true
+    });
+    if (flags.verbose) {
+      console.warn(`[static-sitemap-cli] found ${files.length} files!`);
     }
-    const globs = [...flags.match.map(g => `${rootDir}${g}`), ...flags.ignore.map(g => `!${rootDir}${g}`)];
-    const files = await globby(globs);
-    console.warn(`Found ${files.length} files!`);
 
+    if (flags.text) {
+      let out = '';
+      for (let a = 0; a < files.length; a++) {
+        out += `${getUrl(files[a].path)}
+`;
+      }
+      this.log(out);
+      return;
+    }
+
     let urls = [];
     for (let a = 0; a < files.length; a++) {
+      if (flags.verbose) {
+        console.warn(files[a].path);
+      }
       let obj = {
-        lastmodrealtime: true,
-        lastmodfile: files[a]
+        loc: getUrl(files[a].path),
+        lastmod: files[a].stats.mtime.toISOString()
       };
       if (flags.priority) {
         for (let b = 0; b < flags.priority.length; b++) {
-          if (minimatch(files[a], flags.priority[b].split(',')[0])) {
-            obj.priority = parseFloat(flags.priority[b].split(',')[1]);
-            break;
+          if (mm.isMatch(files[a].path, flags.priority[b].split('=')[0])) {
+            obj.priority = parseFloat(flags.priority[b].split('=')[1]);
           }
@@ -46,33 +78,25 @@ }
         for (let b = 0; b < flags.changefreq.length; b++) {
-          if (minimatch(files[a], flags.changefreq[b].split(',')[0])) {
-            obj.changefreq = flags.changefreq[b].split(',')[1];
-            break;
+          if (mm.isMatch(files[a].path, flags.changefreq[b].split('=')[0])) {
+            obj.changefreq = flags.changefreq[b].split('=')[1];
           }
         }
       }
-      let url = files[a].replace(rootDir, '/');
-      if (!flags['no-clean']) {
-        if (url.slice(-5) === '.html') {
-          url = url.slice(0, -5);
-          if (url.slice(-5) === 'index') {
-            url = url.slice(0, -5);
-          }
-        }
-      }
-      if (flags.slash) {
-        url = url.slice(-1) === '/' ? url : url + '/';
-      } else {
-        url = url.slice(-1) === '/' ? url.slice(0, -1) : url;
-      }
-      obj.url = url;
       urls.push(obj);
     }
 
-    const sm = sitemap.createSitemap({
-      hostname: baseUrl,
-      urls: urls,
+    let sitemap = parser.parse('urlset', {
+      '@': {
+        xmlns: 'http://www.sitemaps.org/schemas/sitemap/0.9'
+      },
+      url: [urls]
+    }, {
+      declaration: {
+        encoding: 'UTF-8'
+      },
+      format: {
+        doubleQuotes: true
+      }
     });
+    this.log(sitemap);
 
-    this.log(sm.toString());
-
   }
@@ -85,8 +109,6 @@ }
 At its most basic, just run from root of distribution:
-> static-sitemap-cli https://example.com > sitemap.xml
-Or:
-> sscli https://example.com > sitemap.xml
-This creates the file 'sitemap.xml' into your root dir.
-CLI by default outputs to 'stdout', and accepts 'stdin' as BASEURL.`;
+$ sscli https://example.com > sitemap.xml
 
+CLI by default outputs to 'stdout'; BASEURL can be piped in via 'stdin'.`;
+
 StaticSitemapCliCommand.args = [{
@@ -99,7 +121,7 @@ name: 'baseUrl',
 StaticSitemapCliCommand.flags = {
-  version: flags.version({char: 'v'}),
+  version: flags.version({char: 'V'}),
   help: flags.help({char: 'h'}),
   root: flags.string({
     char: 'r',
-    description: 'root directory to start from',
+    description: '[default: current] root working directory',
     default: '',
@@ -110,20 +132,14 @@ }),
     multiple: true,
-    description: 'list of globs to match',
-    default: ['**/*.html'],
+    description: 'globs to match',
+    default: ['**/*.html', '!404.html'],
   }),
-  ignore: flags.string({
-    char: 'i',
-    multiple: true,
-    description: 'list of globs to ignore',
-    default: ['404.html'],
-  }),
   priority: flags.string({
     char: 'p',
     multiple: true,
-    description: 'comma-separated glob/priority pair; eg: foo/*.html,0.1',
+    description: 'glob-priority pair [eg: foo/**=0.1]',
   }),
   changefreq: flags.string({
-    char: 'f',
+    char: 'c',
     multiple: true,
-    description: 'comma-separated glob/changefreq pair; eg: bar/*.html,daily',
+    description: 'glob-changefreq pair (eg: bar/**=daily)',
   }),
@@ -141,4 +157,15 @@ 'no-clean': flags.boolean({
   }),
+  text: flags.boolean({
+    char: 't',
+    description: 'output as .TXT instead',
+    default: false,
+    exclusive: ['priority', 'changefreq'],
+  }),
+  verbose: flags.boolean({
+    char: 'v',
+    description: 'be more verbose',
+    default: false,
+  }),
 }
 
-module.exports = StaticSitemapCliCommand
+module.exports = StaticSitemapCliCommand;

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

No v1

Quality

Package is not semver >=1. This means it is not stable and does not support ^ ranges.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

12270

increased by28.25%

Lines of code

149

increased by19.2%

Number of low quality alerts

0

decreased by-100%

Number of lines in readme file

178

increased by33.83%

Worsened metrics

Dev dependency count

5

increased byInfinity%

Dependency changes

• + Addedfast-glob@^3.0.4

• + Addedjs2xmlparser@^4.0.0

• + Addedmicromatch@^4.0.2

• + Addedjs2xmlparser@4.0.2(transitive)

• + Addedxmlcreate@2.0.4(transitive)

• - Removedglobby@^10.0.1

• - Removedminimatch@^3.0.4

• - Removedsitemap@^3.2.2

• - Removed@types/glob@7.2.0(transitive)

• - Removed@types/minimatch@5.1.2(transitive)

• - Removed@types/node@22.9.1(transitive)

• - Removedbalanced-match@1.0.2(transitive)

• - Removedbrace-expansion@1.1.11(transitive)

• - Removedconcat-map@0.0.1(transitive)

• - Removedfs.realpath@1.0.0(transitive)

• - Removedglob@7.2.3(transitive)

• - Removedglobby@10.0.2(transitive)

• - Removedinflight@1.0.6(transitive)

• - Removedinherits@2.0.4(transitive)

• - Removedlodash.chunk@4.2.0(transitive)

• - Removedlodash.padstart@4.6.1(transitive)

• - Removedlodash.sortby@4.7.0(transitive)

• - Removedminimatch@3.1.2(transitive)

• - Removedonce@1.4.0(transitive)

• - Removedpath-is-absolute@1.0.1(transitive)

• - Removedpunycode@2.3.1(transitive)

• - Removedsitemap@3.2.2(transitive)

• - Removedtr46@1.0.1(transitive)

• - Removedundici-types@6.19.8(transitive)

• - Removedwebidl-conversions@4.0.2(transitive)

• - Removedwhatwg-url@7.1.0(transitive)

• - Removedwrappy@1.0.2(transitive)

• - Removedxmlbuilder@13.0.2(transitive)