Socket
Socket
Sign inDemoInstall

sitemap

Package Overview
Dependencies
Maintainers
1
Versions
73
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

sitemap - npm Package Compare versions

Comparing version 3.2.2 to 4.0.1

dist/cli.d.ts
dist/cli.js
dist/cli.js.map
dist/lib/xmllint.d.ts
dist/lib/xmllint.js
dist/lib/xmllint.js.map
schema/all.xsd
schema/sitemap-image.xsd
schema/sitemap-mobile.xsd
schema/sitemap-news.xsd
schema/sitemap-video.xsd
schema/sitemap.xsd
schema/xhtml-strict.xsd

43

CHANGELOG.md

@@ -0,1 +1,44 @@

# 4.0.1
Validation functions which depend on xmllint will now warn if you do not have xmllint installed.
# 4.0.0
This release is geared around overhauling the public api for this library. Many
options have been introduced over the years and this has lead to some inconsistencies
that make the library hard to use. Most have been cleaned up but a couple notable
items remain, including the confusing names of buildSitemapIndex and createSitemapIndex
- A new experimental CLI
- stream in a list of urls stream out xml
- validate your generated sitemap
- Sitemap video item now supports id element
- Several schema errors have been cleaned up.
- Docs have been updated and streamlined.
## breaking changes
- lastmod option parses all ISO8601 date-only strings as being in UTC rather than local time
- lastmodISO is deprecated as it is equivalent to lastmod
- lastmodfile now includes the file's time as well
- lastmodrealtime is no longer necessary
- The default export of sitemap lib is now just createSitemap
- Sitemap constructor now uses a object for its constructor
```
const { Sitemap } = require('sitemap');
const siteMap = new Sitemap({
urls = [],
hostname: 'https://example.com', // optional
cacheTime = 0,
xslUrl,
xmlNs,
level = 'warn'
})
```
- Sitemap no longer accepts a single string for its url
- Drop support for node 6
- Remove callback on toXML - This had no performance benefit
- Direct modification of urls property on Sitemap has been dropped. Use add/remove/contains
- When a Sitemap item is generated with invalid options it no longer throws by default
- instead it console warns.
- if you'd like to pre-verify your data the `validateSMIOptions` function is
now available
- To get the previous behavior pass level `createSitemap({...otheropts, level: 'throw' }) // ErrorLevel.THROW for TS users`
# 3.2.2

@@ -2,0 +45,0 @@ - revert https everywhere added in 3.2.0. xmlns is not url.

7

dist/index.d.ts

@@ -6,6 +6,9 @@ /*!

*/
import * as sm from './lib/sitemap';
import { createSitemap } from './lib/sitemap';
export * from './lib/sitemap';
export * from './lib/sitemap-item';
export * from './lib/sitemap-index';
export * from './lib/errors';
export * from './lib/types';
export default sm;
export { xmlLint } from './lib/xmllint';
export default createSitemap;

15

dist/index.js

@@ -5,9 +5,2 @@ "use strict";

}
var __importStar = (this && this.__importStar) || function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k in mod) if (Object.hasOwnProperty.call(mod, k)) result[k] = mod[k];
result["default"] = mod;
return result;
};
Object.defineProperty(exports, "__esModule", { value: true });

@@ -19,7 +12,11 @@ /*!

*/
const sm = __importStar(require("./lib/sitemap"));
const sitemap_1 = require("./lib/sitemap");
__export(require("./lib/sitemap"));
__export(require("./lib/sitemap-item"));
__export(require("./lib/sitemap-index"));
__export(require("./lib/errors"));
__export(require("./lib/types"));
exports.default = sm;
var xmllint_1 = require("./lib/xmllint");
exports.xmlLint = xmllint_1.xmlLint;
exports.default = sitemap_1.createSitemap;
//# sourceMappingURL=index.js.map

3

dist/lib/errors.d.ts

@@ -57,1 +57,4 @@ /*!

}
export declare class XMLLintUnavailable extends Error {
constructor(message?: string);
}

10

dist/lib/errors.js

@@ -96,2 +96,3 @@ "use strict";

class InvalidAttrValue extends Error {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
constructor(key, val, validator) {

@@ -134,2 +135,11 @@ super('"' + val + '" tested against: ' + validator + ' is not a valid value for attr: "' + key + '"');

exports.InvalidNewsAccessValue = InvalidNewsAccessValue;
class XMLLintUnavailable extends Error {
constructor(message) {
super(message || 'xmlLint is not installed. XMLLint is required to validate');
this.name = 'XMLLintUnavailable';
// @ts-ignore
Error.captureStackTrace(this, XMLLintUnavailable);
}
}
exports.XMLLintUnavailable = XMLLintUnavailable;
//# sourceMappingURL=errors.js.map

27

dist/lib/sitemap-index.d.ts

@@ -1,3 +0,2 @@

import { Sitemap } from './sitemap';
import { ICallback } from './types';
import { ICallback, ISitemapIndexItemOptions, SitemapItemOptions } from './types';
/**

@@ -34,10 +33,9 @@ * Shortcut for `new SitemapIndex (...)`.

* @param {String} conf.xmlNs
* @param {String} conf.lastmod When the referenced sitemap was last modified
* @return {String} XML String of SitemapIndex
*/
export declare function buildSitemapIndex(conf: {
urls: Sitemap["urls"];
urls: (ISitemapIndexItemOptions | string)[];
xslUrl?: string;
xmlNs?: string;
lastmodISO?: string;
lastmodrealtime?: boolean;
lastmod?: number | string;

@@ -49,14 +47,13 @@ }): string;

declare class SitemapIndex {
hostname?: string;
urls: (string | SitemapItemOptions)[];
targetFolder: string;
hostname?: string | undefined;
sitemapSize?: number | undefined;
xslUrl?: string | undefined;
callback?: ICallback<Error, boolean> | undefined;
sitemapName: string;
sitemapSize?: number;
xslUrl?: string;
sitemapId: number;
sitemaps: string[];
targetFolder: string;
urls: Sitemap["urls"];
chunks: Sitemap["urls"][];
callback?: ICallback<Error, boolean>;
chunks: (string | SitemapItemOptions)[][];
cacheTime?: number;
xmlNs?: string;
/**

@@ -68,3 +65,3 @@ * @param {String|Array} urls

* @param {String} sitemapName optional
* @param {Number} sitemapSize optional
* @param {Number} sitemapSize optional This limit is defined by Google. See: https://sitemaps.org/protocol.php#index
* @param {Number} xslUrl optional

@@ -74,4 +71,4 @@ * @param {Boolean} gzip optional

*/
constructor(urls: Sitemap["urls"], targetFolder: string, hostname?: string, cacheTime?: number, sitemapName?: string, sitemapSize?: number, xslUrl?: string, gzip?: boolean, callback?: ICallback<Error, boolean>);
constructor(urls?: (string | SitemapItemOptions)[], targetFolder?: string, hostname?: string | undefined, cacheTime?: number, sitemapName?: string, sitemapSize?: number | undefined, xslUrl?: string | undefined, gzip?: boolean, callback?: ICallback<Error, boolean> | undefined);
}
export {};

61

dist/lib/sitemap-index.js

@@ -7,4 +7,3 @@ "use strict";

const errors_1 = require("./errors");
/* eslint-disable @typescript-eslint/no-var-requires */
const chunk = require('lodash.chunk');
const utils_1 = require("./utils");
/**

@@ -36,2 +35,3 @@ * Shortcut for `new SitemapIndex (...)`.

* @param {String} conf.xmlNs
* @param {String} conf.lastmod When the referenced sitemap was last modified
* @return {String} XML String of SitemapIndex

@@ -53,9 +53,3 @@ */

}
if (conf.lastmodISO) {
lastmod = conf.lastmodISO;
}
else if (conf.lastmodrealtime) {
lastmod = new Date().toISOString();
}
else if (conf.lastmod) {
if (conf.lastmod) {
lastmod = new Date(conf.lastmod).toISOString();

@@ -67,7 +61,4 @@ }

if (url.lastmod) {
lm = url.lastmod;
lm = new Date(url.lastmod).toISOString();
}
else if (url.lastmodISO) {
lm = url.lastmodISO;
}
url = url.url;

@@ -94,3 +85,3 @@ }

* @param {String} sitemapName optional
* @param {Number} sitemapSize optional
* @param {Number} sitemapSize optional This limit is defined by Google. See: https://sitemaps.org/protocol.php#index
* @param {Number} xslUrl optional

@@ -100,5 +91,9 @@ * @param {Boolean} gzip optional

*/
constructor(urls, targetFolder, hostname, cacheTime, sitemapName, sitemapSize, xslUrl, gzip, callback) {
// Base domain
constructor(urls = [], targetFolder = '.', hostname, cacheTime, sitemapName, sitemapSize, xslUrl, gzip = false, callback) {
this.urls = urls;
this.targetFolder = targetFolder;
this.hostname = hostname;
this.sitemapSize = sitemapSize;
this.xslUrl = xslUrl;
this.callback = callback;
if (sitemapName === undefined) {

@@ -110,9 +105,4 @@ this.sitemapName = 'sitemap';

}
// This limit is defined by Google. See:
// https://sitemaps.org/protocol.php#index
this.sitemapSize = sitemapSize;
this.xslUrl = xslUrl;
this.sitemapId = 0;
this.sitemaps = [];
this.targetFolder = '.';
try {

@@ -123,15 +113,6 @@ if (!fs_1.statSync(targetFolder).isDirectory()) {

}
catch (err) {
catch (e) {
throw new errors_1.UndefinedTargetFolder();
}
this.targetFolder = targetFolder;
// URL list for sitemap
// @ts-ignore
this.urls = urls || [];
if (!Array.isArray(this.urls)) {
// @ts-ignore
this.urls = [this.urls];
}
this.chunks = chunk(this.urls, this.sitemapSize);
this.callback = callback;
this.chunks = utils_1.chunk(urls, this.sitemapSize);
let processesCount = this.chunks.length + 1;

@@ -143,6 +124,6 @@ this.chunks.forEach((chunk, index) => {

let sitemap = sitemap_1.createSitemap({
hostname: this.hostname,
cacheTime: this.cacheTime,
hostname,
cacheTime,
urls: chunk,
xslUrl: this.xslUrl
xslUrl
});

@@ -159,9 +140,9 @@ let stream = fs_1.createWriteStream(targetFolder + '/' + filename);

});
let sitemapUrls = this.sitemaps.map((sitemap) => hostname + '/' + sitemap);
let smConf = { urls: sitemapUrls, xslUrl: this.xslUrl, xmlNs: this.xmlNs };
let xmlString = buildSitemapIndex(smConf);
let stream = fs_1.createWriteStream(targetFolder + '/' +
const stream = fs_1.createWriteStream(targetFolder + '/' +
this.sitemapName + '-index.xml');
stream.once('open', (fd) => {
stream.write(xmlString);
stream.write(buildSitemapIndex({
urls: this.sitemaps.map((sitemap) => hostname + '/' + sitemap),
xslUrl
}));
stream.end();

@@ -168,0 +149,0 @@ processesCount--;

10

dist/lib/sitemap-item.d.ts
import { XMLElement } from 'xmlbuilder';
import { IVideoItem, SitemapItemOptions } from './types';
import { IVideoItem, SitemapItemOptions, ErrorLevel } from './types';
/**
* Item in sitemap
*/
declare class SitemapItem {
export declare class SitemapItem {
conf: SitemapItemOptions;
root: XMLElement;
loc: SitemapItemOptions["url"];

@@ -20,5 +21,5 @@ lastmod: SitemapItemOptions["lastmod"];

ampLink?: SitemapItemOptions["ampLink"];
root: XMLElement;
url: XMLElement;
constructor(conf: SitemapItemOptions);
constructor(conf: SitemapItemOptions, root?: XMLElement, level?: ErrorLevel);
static justItem(conf: SitemapItemOptions, level?: ErrorLevel): string;
/**

@@ -37,2 +38,1 @@ * Create sitemap xml

}
export default SitemapItem;

137

dist/lib/sitemap-item.js
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const utils_1 = require("./utils");
const fs_1 = require("fs");
const xmlbuilder_1 = require("xmlbuilder");
const errors_1 = require("./errors");
const types_1 = require("./types");
function safeDuration(duration) {
if (duration < 0 || duration > 28800) {
throw new errors_1.InvalidVideoDuration();
}
return duration;
}
const allowDeny = /^allow|deny$/;
const validators = {
'price:currency': /^[A-Z]{3}$/,
'price:type': /^rent|purchase|RENT|PURCHASE$/,
'price:resolution': /^HD|hd|sd|SD$/,
'platform:relationship': allowDeny,
'restriction:relationship': allowDeny
};
const utils_1 = require("./utils");
function attrBuilder(conf, keys) {

@@ -34,6 +19,2 @@ if (typeof keys === 'string') {

}
// eslint-disable-next-line
if (validators[key] && !validators[key].test(conf[key])) {
throw new errors_1.InvalidAttrValue(key, conf[key], validators[key]);
}
attrs[keyAr[1]] = conf[key];

@@ -48,53 +29,17 @@ }

class SitemapItem {
constructor(conf) {
constructor(conf, root = xmlbuilder_1.create('root'), level = types_1.ErrorLevel.WARN) {
this.conf = conf;
if (!conf) {
throw new errors_1.NoConfigError();
}
if (!conf.url) {
throw new errors_1.NoURLError();
}
const isSafeUrl = conf.safe;
this.root = root;
utils_1.validateSMIOptions(conf, level);
const { url: loc, lastmod, changefreq, priority } = conf;
// URL of the page
this.loc = conf.url;
let dt;
// If given a file to use for last modified date
if (conf.lastmodfile) {
// console.log('should read stat from file: ' + conf.lastmodfile);
let file = conf.lastmodfile;
let stat = fs_1.statSync(file);
let mtime = stat.mtime;
dt = new Date(mtime);
this.lastmod = utils_1.getTimestampFromDate(dt, conf.lastmodrealtime);
// The date of last modification (YYYY-MM-DD)
}
else if (conf.lastmod) {
// append the timezone offset so that dates are treated as local time.
// Otherwise the Unit tests fail sometimes.
let timezoneOffset = 'UTC-' + (new Date().getTimezoneOffset() / 60) + '00';
timezoneOffset = timezoneOffset.replace('--', '-');
dt = new Date(conf.lastmod + ' ' + timezoneOffset);
this.lastmod = utils_1.getTimestampFromDate(dt, conf.lastmodrealtime);
}
else if (conf.lastmodISO) {
this.lastmod = conf.lastmodISO;
}
this.loc = loc;
// How frequently the page is likely to change
// due to this field is optional no default value is set
// please see: https://www.sitemaps.org/protocol.html
this.changefreq = conf.changefreq;
if (!isSafeUrl && this.changefreq) {
if (types_1.CHANGEFREQ.indexOf(this.changefreq) === -1) {
throw new errors_1.ChangeFreqInvalidError();
}
}
this.changefreq = changefreq;
// The priority of this URL relative to other URLs
// due to this field is optional no default value is set
// please see: https://www.sitemaps.org/protocol.html
this.priority = conf.priority;
if (!isSafeUrl && this.priority) {
if (!(this.priority >= 0.0 && this.priority <= 1.0) || typeof this.priority !== 'number') {
throw new errors_1.PriorityInvalidError();
}
}
this.priority = priority;
this.news = conf.news;

@@ -108,5 +53,9 @@ this.img = conf.img;

this.ampLink = conf.ampLink;
this.root = conf.root || xmlbuilder_1.create('root');
this.url = this.root.element('url');
this.lastmod = lastmod;
}
static justItem(conf, level) {
const smi = new SitemapItem(conf, undefined, level);
return smi.toString();
}
/**

@@ -121,9 +70,2 @@ * Create sitemap xml

const videoxml = this.url.element('video:video');
if (typeof (video) !== 'object' || !video.thumbnail_loc || !video.title || !video.description) {
// has to be an object and include required categories https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190
throw new errors_1.InvalidVideoFormat();
}
if (video.description.length > 2048) {
throw new errors_1.InvalidVideoDescription();
}
videoxml.element('video:thumbnail_loc', video.thumbnail_loc);

@@ -139,3 +81,3 @@ videoxml.element('video:title').cdata(video.title);

if (video.duration) {
videoxml.element('video:duration', safeDuration(video.duration));
videoxml.element('video:duration', video.duration);
}

@@ -145,6 +87,6 @@ if (video.expiration_date) {

}
if (video.rating) {
if (video.rating !== undefined) {
videoxml.element('video:rating', video.rating);
}
if (video.view_count) {
if (video.view_count !== undefined) {
videoxml.element('video:view_count', video.view_count);

@@ -155,18 +97,11 @@ }

}
if (video.family_friendly) {
videoxml.element('video:family_friendly', video.family_friendly);
for (const tag of video.tag) {
videoxml.element('video:tag', tag);
}
if (video.tag) {
if (!Array.isArray(video.tag)) {
videoxml.element('video:tag', video.tag);
}
else {
for (const tag of video.tag) {
videoxml.element('video:tag', tag);
}
}
}
if (video.category) {
videoxml.element('video:category', video.category);
}
if (video.family_friendly) {
videoxml.element('video:family_friendly', video.family_friendly);
}
if (video.restriction) {

@@ -193,2 +128,5 @@ videoxml.element('video:restriction', attrBuilder(video, 'restriction:relationship'), video.restriction);

}
if (video.id) {
videoxml.element('video:id', { type: 'url' }, video.id);
}
}

@@ -210,13 +148,4 @@ buildXML() {

// Image handling
if (!Array.isArray(this.img)) {
// make it an array
this.img = [this.img];
}
this.img.forEach((image) => {
const xmlObj = {};
if (typeof (image) !== 'object') {
// it’s a string
// make it an object
image = { url: image };
}
xmlObj['image:loc'] = image.url;

@@ -239,7 +168,2 @@ if (image.caption) {

else if (this.video && p === 'video') {
// Image handling
if (!Array.isArray(this.video)) {
// make it an array
this.video = [this.video];
}
this.video.forEach(this.buildVideoElement, this);

@@ -281,9 +205,2 @@ }

let newsitem = this.url.element('news:news');
if (!this.news.publication ||
!this.news.publication.name ||
!this.news.publication.language ||
!this.news.publication_date ||
!this.news.title) {
throw new errors_1.InvalidNewsFormat();
}
if (this.news.publication) {

@@ -299,6 +216,2 @@ let publication = newsitem.element('news:publication');

if (this.news.access) {
if (this.news.access !== 'Registration' &&
this.news.access !== 'Subscription') {
throw new errors_1.InvalidNewsAccessValue();
}
newsitem.element('news:access', this.news.access);

@@ -345,3 +258,3 @@ }

}
exports.default = SitemapItem;
exports.SitemapItem = SitemapItem;
//# sourceMappingURL=sitemap-item.js.map

46

dist/lib/sitemap.d.ts

@@ -7,10 +7,5 @@ /// <reference types="node" />

*/
import * as errors from './errors';
import { XMLElement } from 'xmlbuilder';
import SitemapItem from './sitemap-item';
import { ICallback, SitemapItemOptions } from './types';
import { ISitemapItemOptionsLoose, SitemapItemOptions, ErrorLevel } from './types';
import { CompressCallback } from 'zlib';
export { errors };
export * from './sitemap-index';
export declare const version = "2.2.0";
/**

@@ -27,4 +22,4 @@ * Shortcut for `new Sitemap (...)`.

*/
export declare function createSitemap(conf: {
urls?: string | Sitemap["urls"];
export declare function createSitemap({ urls, hostname, cacheTime, xslUrl, xmlNs, level }: {
urls?: (ISitemapItemOptionsLoose | string)[];
hostname?: string;

@@ -34,2 +29,3 @@ cacheTime?: number;

xmlNs?: string;
level?: ErrorLevel;
}): Sitemap;

@@ -40,8 +36,8 @@ export declare class Sitemap {

cacheSetTimestamp: number;
private urls;
cacheTime: number;
cache: string;
root: XMLElement;
hostname?: string;
urls: (string | SitemapItemOptions)[];
cacheResetPeriod: number;
cache: string;
xslUrl?: string;
root: XMLElement;
/**

@@ -55,3 +51,10 @@ * Sitemap constructor

*/
constructor(urls?: string | Sitemap["urls"], hostname?: string, cacheTime?: number, xslUrl?: string, xmlNs?: string);
constructor({ urls, hostname, cacheTime, xslUrl, xmlNs, level }?: {
urls?: (ISitemapItemOptionsLoose | string)[];
hostname?: string;
cacheTime?: number;
xslUrl?: string;
xmlNs?: string;
level?: ErrorLevel;
});
/**

@@ -69,2 +72,3 @@ * Clear sitemap cache

setCache(newCache: string): string;
private _normalizeURL;
/**

@@ -74,13 +78,16 @@ * Add url to sitemap

*/
add(url: string | SitemapItemOptions): number;
add(url: string | ISitemapItemOptionsLoose, level?: ErrorLevel): number;
contains(url: string | ISitemapItemOptionsLoose): boolean;
/**
* Delete url from sitemap
* @param {String} url
* @param {String | SitemapItemOptions} url
* @returns boolean whether the item was removed
*/
del(url: string | SitemapItemOptions): number;
del(url: string | ISitemapItemOptionsLoose): boolean;
/**
* Create sitemap xml
* @param {Function} callback Callback function with one argument — xml
* Alias for toString
*/
toXML(callback?: ICallback<Error, string>): string | void;
toXML(): string;
static normalizeURL(elem: string | ISitemapItemOptionsLoose, root?: XMLElement, hostname?: string): SitemapItemOptions;
static normalizeURLs(urls: (string | ISitemapItemOptionsLoose)[], root?: XMLElement, hostname?: string): Map<string, SitemapItemOptions>;
/**

@@ -94,2 +101,1 @@ * Synchronous alias for toXML()

}
export { SitemapItem };

241

dist/lib/sitemap.js
"use strict";
function __export(m) {
for (var p in m) if (!exports.hasOwnProperty(p)) exports[p] = m[p];
}
var __importStar = (this && this.__importStar) || function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k in mod) if (Object.hasOwnProperty.call(mod, k)) result[k] = mod[k];
result["default"] = mod;
return result;
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });

@@ -22,12 +9,18 @@ /* eslint-disable camelcase, semi */

*/
const errors = __importStar(require("./errors"));
exports.errors = errors;
const xmlbuilder_1 = require("xmlbuilder");
const sitemap_item_1 = __importDefault(require("./sitemap-item"));
exports.SitemapItem = sitemap_item_1.default;
const sitemap_item_1 = require("./sitemap-item");
const types_1 = require("./types");
const zlib_1 = require("zlib");
// remove once we drop node 8
const whatwg_url_1 = require("whatwg-url");
__export(require("./sitemap-index"));
exports.version = '2.2.0';
const url_1 = require("url");
const fs_1 = require("fs");
const utils_1 = require("./utils");
function boolToYESNO(bool) {
if (bool === undefined) {
return bool;
}
if (typeof bool === 'boolean') {
return bool ? types_1.EnumYesNo.yes : types_1.EnumYesNo.no;
}
return bool;
}
/**

@@ -44,6 +37,13 @@ * Shortcut for `new Sitemap (...)`.

*/
function createSitemap(conf) {
function createSitemap({ urls, hostname, cacheTime, xslUrl, xmlNs, level }) {
// cleaner diff
// eslint-disable-next-line @typescript-eslint/no-use-before-define
return new Sitemap(conf.urls, conf.hostname, conf.cacheTime, conf.xslUrl, conf.xmlNs);
return new Sitemap({
urls,
hostname,
cacheTime,
xslUrl,
xmlNs,
level
});
}

@@ -60,3 +60,3 @@ exports.createSitemap = createSitemap;

*/
constructor(urls, hostname, cacheTime, xslUrl, xmlNs) {
constructor({ urls = [], hostname, cacheTime = 0, xslUrl, xmlNs, level = types_1.ErrorLevel.WARN } = {}) {
// This limit is defined by Google. See:

@@ -69,12 +69,4 @@ // https://sitemaps.org/protocol.php#index

this.hostname = hostname;
// Make copy of object
if (urls) {
this.urls = Array.isArray(urls) ? Array.from(urls) : [urls];
}
else {
// URL list for sitemap
this.urls = [];
}
// sitemap cache
this.cacheResetPeriod = cacheTime || 0;
this.cacheTime = cacheTime;
this.cache = '';

@@ -91,2 +83,7 @@ this.xslUrl = xslUrl;

}
urls = Array.from(urls);
this.urls = Sitemap.normalizeURLs(urls, this.root, this.hostname);
for (let [, url] of this.urls) {
utils_1.validateSMIOptions(url, level);
}
}

@@ -104,4 +101,4 @@ /**

let currTimestamp = Date.now();
return !!(this.cacheResetPeriod && this.cache &&
(this.cacheSetTimestamp + this.cacheResetPeriod) >= currTimestamp);
return !!(this.cacheTime && this.cache &&
(this.cacheSetTimestamp + this.cacheTime) >= currTimestamp);
}

@@ -116,2 +113,5 @@ /**

}
_normalizeURL(url) {
return Sitemap.normalizeURL(url, this.root, this.hostname);
}
/**

@@ -121,52 +121,110 @@ * Add url to sitemap

*/
add(url) {
return this.urls.push(url);
add(url, level) {
const smi = this._normalizeURL(url);
utils_1.validateSMIOptions(smi, level);
return this.urls.set(smi.url, smi).size;
}
contains(url) {
return this.urls.has(this._normalizeURL(url).url);
}
/**
* Delete url from sitemap
* @param {String} url
* @param {String | SitemapItemOptions} url
* @returns boolean whether the item was removed
*/
del(url) {
const indexToRemove = [];
let key = '';
if (typeof url === 'string') {
key = url;
return this.urls.delete(this._normalizeURL(url).url);
}
/**
* Alias for toString
*/
toXML() {
return this.toString();
}
static normalizeURL(elem, root, hostname) {
// SitemapItem
// create object with url property
let smi = {
img: [],
video: [],
links: [],
url: ''
};
let smiLoose;
if (typeof elem === 'string') {
smi.url = elem;
smiLoose = { url: elem };
}
else {
// @ts-ignore
key = url.url;
smiLoose = elem;
}
// find
this.urls.forEach((elem, index) => {
if (typeof elem === 'string') {
if (elem === key) {
indexToRemove.push(index);
}
smi.url = (new url_1.URL(smiLoose.url, hostname)).toString();
let img = [];
if (smiLoose.img) {
if (typeof smiLoose.img === 'string') {
// string -> array of objects
smiLoose.img = [{ url: smiLoose.img }];
}
else {
if (elem.url === key) {
indexToRemove.push(index);
}
else if (!Array.isArray(smiLoose.img)) {
// object -> array of objects
smiLoose.img = [smiLoose.img];
}
img = smiLoose.img.map((el) => typeof el === 'string' ? { url: el } : el);
}
// prepend hostname to all image urls
smi.img = img.map((el) => (Object.assign({}, el, { url: (new url_1.URL(el.url, hostname)).toString() })));
let links = [];
if (smiLoose.links) {
links = smiLoose.links;
}
smi.links = links.map((link) => {
return Object.assign({}, link, { url: (new url_1.URL(link.url, hostname)).toString() });
});
// delete
indexToRemove.forEach((elem) => { this.urls.splice(elem, 1); });
return indexToRemove.length;
if (smiLoose.video) {
if (!Array.isArray(smiLoose.video)) {
// make it an array
smiLoose.video = [smiLoose.video];
}
smi.video = smiLoose.video.map((video) => {
const nv = Object.assign({}, video, {
/* eslint-disable-next-line @typescript-eslint/camelcase */
family_friendly: boolToYESNO(video.family_friendly), live: boolToYESNO(video.live),
/* eslint-disable-next-line @typescript-eslint/camelcase */
requires_subscription: boolToYESNO(video.requires_subscription), tag: [], rating: undefined });
if (video.tag !== undefined) {
nv.tag = !Array.isArray(video.tag) ? [video.tag] : video.tag;
}
if (video.rating !== undefined) {
if (typeof video.rating === 'string') {
nv.rating = parseFloat(video.rating);
}
else {
nv.rating = video.rating;
}
}
return nv;
});
}
// If given a file to use for last modified date
if (smiLoose.lastmodfile) {
const { mtime } = fs_1.statSync(smiLoose.lastmodfile);
smi.lastmod = (new Date(mtime)).toISOString();
// The date of last modification (YYYY-MM-DD)
}
else if (smiLoose.lastmodISO) {
smi.lastmod = (new Date(smiLoose.lastmodISO)).toISOString();
}
else if (smiLoose.lastmod) {
smi.lastmod = (new Date(smiLoose.lastmod)).toISOString();
}
smi = Object.assign({}, smiLoose, smi);
return smi;
}
/**
* Create sitemap xml
* @param {Function} callback Callback function with one argument — xml
*/
toXML(callback) {
if (typeof callback === 'undefined') {
return this.toString();
}
process.nextTick(() => {
try {
callback(undefined, this.toString());
}
catch (err) {
callback(err);
}
static normalizeURLs(urls, root, hostname) {
const urlMap = new Map();
urls.forEach((elem) => {
const smio = Sitemap.normalizeURL(elem, root, hostname);
urlMap.set(smio.url, smio);
});
return urlMap;
}

@@ -196,38 +254,5 @@ /**

// TODO: if size > limit: create sitemapindex
this.urls.forEach((elem, index) => {
// SitemapItem
// create object with url property
let smi = (typeof elem === 'string') ? { 'url': elem, root: this.root } : Object.assign({ root: this.root }, elem);
// insert domain name
if (this.hostname) {
smi.url = (new whatwg_url_1.URL(smi.url, this.hostname)).toString();
if (smi.img) {
if (typeof smi.img === 'string') {
// string -> array of objects
smi.img = [{ url: smi.img }];
}
else if (!Array.isArray(smi.img)) {
// object -> array of objects
smi.img = [smi.img];
}
// prepend hostname to all image urls
smi.img.forEach((img) => {
if (typeof img === 'string') {
img = { url: img };
}
img.url = (new whatwg_url_1.URL(img.url, this.hostname)).toString();
});
}
if (smi.links) {
smi.links.forEach((link) => {
link.url = (new whatwg_url_1.URL(link.url, this.hostname)).toString();
});
}
}
else {
smi.url = (new whatwg_url_1.URL(smi.url)).toString();
}
const sitemapItem = new sitemap_item_1.default(smi);
sitemapItem.buildXML();
});
for (let [, smi] of this.urls) {
(new sitemap_item_1.SitemapItem(smi, this.root)).buildXML();
}
return this.setCache(this.root.end());

@@ -234,0 +259,0 @@ }

65

dist/lib/types.d.ts

@@ -1,2 +0,3 @@

import { XMLElement, XMLCData } from 'xmlbuilder';
/// <reference types="node" />
import { URL } from 'url';
export declare enum EnumChangefreq {

@@ -13,4 +14,8 @@ DAILY = "daily",

export declare enum EnumYesNo {
YES = "yes",
NO = "no"
YES = "YES",
NO = "NO",
Yes = "Yes",
No = "No",
yes = "yes",
no = "no"
}

@@ -41,3 +46,3 @@ export declare enum EnumAllowDeny {

}
export interface IVideoItem {
interface IVideoItemBase {
thumbnail_loc: string;

@@ -51,7 +56,4 @@ title: string;

expiration_date?: string;
rating?: string | number;
view_count?: string | number;
publication_date?: string;
family_friendly?: EnumYesNo;
tag?: string | string[];
category?: string;

@@ -66,8 +68,21 @@ restriction?: string;

'price:type'?: string;
requires_subscription?: EnumYesNo;
uploader?: string;
platform?: string;
id?: string;
'platform:relationship'?: EnumAllowDeny;
}
export interface IVideoItem extends IVideoItemBase {
tag: string[];
rating?: number;
family_friendly?: EnumYesNo;
requires_subscription?: EnumYesNo;
live?: EnumYesNo;
}
export interface IVideoItemLoose extends IVideoItemBase {
tag?: string | string[];
rating?: string | number;
family_friendly?: EnumYesNo | boolean;
requires_subscription?: EnumYesNo | boolean;
live?: EnumYesNo | boolean;
}
export interface ILinkItem {

@@ -77,8 +92,9 @@ lang: string;

}
export interface SitemapItemOptions {
safe?: boolean;
lastmodfile?: any;
lastmodrealtime?: boolean;
export interface ISitemapIndexItemOptions {
url: string;
lastmod?: string;
lastmodISO?: string;
}
interface ISitemapItemOptionsBase {
lastmod?: string;
changefreq?: EnumChangefreq;

@@ -88,12 +104,27 @@ fullPrecisionPriority?: boolean;

news?: INewsItem;
img?: string | ISitemapImg | (string | ISitemapImg)[];
links?: ILinkItem[];
expires?: string;
androidLink?: string;
mobile?: boolean | string;
video?: IVideoItem | IVideoItem[];
ampLink?: string;
root?: XMLElement;
url: string;
cdata?: XMLCData;
cdata?: boolean;
}
export interface SitemapItemOptions extends ISitemapItemOptionsBase {
img: ISitemapImg[];
video: IVideoItem[];
links: ILinkItem[];
}
export interface ISitemapItemOptionsLoose extends ISitemapItemOptionsBase {
video?: IVideoItemLoose | IVideoItemLoose[];
img?: string | ISitemapImg | (string | ISitemapImg)[];
links?: ILinkItem[];
lastmodfile?: string | Buffer | URL;
lastmodISO?: string;
lastmodrealtime?: boolean;
}
export declare enum ErrorLevel {
SILENT = "silent",
WARN = "warn",
THROW = "throw"
}
export {};

14

dist/lib/types.js

@@ -26,4 +26,8 @@ "use strict";

(function (EnumYesNo) {
EnumYesNo["YES"] = "yes";
EnumYesNo["NO"] = "no";
EnumYesNo["YES"] = "YES";
EnumYesNo["NO"] = "NO";
EnumYesNo["Yes"] = "Yes";
EnumYesNo["No"] = "No";
EnumYesNo["yes"] = "yes";
EnumYesNo["no"] = "no";
})(EnumYesNo = exports.EnumYesNo || (exports.EnumYesNo = {}));

@@ -35,2 +39,8 @@ var EnumAllowDeny;

})(EnumAllowDeny = exports.EnumAllowDeny || (exports.EnumAllowDeny = {}));
var ErrorLevel;
(function (ErrorLevel) {
ErrorLevel["SILENT"] = "silent";
ErrorLevel["WARN"] = "warn";
ErrorLevel["THROW"] = "throw";
})(ErrorLevel = exports.ErrorLevel || (exports.ErrorLevel = {}));
//# sourceMappingURL=types.js.map

21

dist/lib/utils.d.ts

@@ -1,1 +0,20 @@

export declare function getTimestampFromDate(dt: Date, bRealtime?: boolean): string;
/*!
* Sitemap
* Copyright(c) 2011 Eugene Kalinin
* MIT Licensed
*/
import { SitemapItemOptions, ErrorLevel } from './types';
export declare function validateSMIOptions(conf: SitemapItemOptions, level?: ErrorLevel): SitemapItemOptions;
/**
* Based on lodash's implementation of chunk.
*
* Copyright JS Foundation and other contributors <https://js.foundation/>
*
* Based on Underscore.js, copyright Jeremy Ashkenas,
* DocumentCloud and Investigative Reporters & Editors <http://underscorejs.org/>
*
* This software consists of voluntary contributions made by many
* individuals. For exact contribution history, see the revision history
* available at https://github.com/lodash/lodash
*/
export declare function chunk(array: any[], size?: number): any[];

162

dist/lib/utils.js
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
/*!

@@ -8,19 +7,150 @@ * Sitemap

*/
/* eslint-disable @typescript-eslint/no-var-requires */
const padStart = require('lodash.padstart');
function getTimestampFromDate(dt, bRealtime) {
let timestamp = [dt.getUTCFullYear(), padStart((dt.getUTCMonth() + 1), 2, '0'),
padStart(dt.getUTCDate(), 2, '0')].join('-');
// Indicate that lastmod should include minutes and seconds (and timezone)
if (bRealtime && bRealtime === true) {
timestamp += 'T';
timestamp += [padStart(dt.getUTCHours(), 2, '0'),
padStart(dt.getUTCMinutes(), 2, '0'),
padStart(dt.getUTCSeconds(), 2, '0')
].join(':');
timestamp += 'Z';
Object.defineProperty(exports, "__esModule", { value: true });
const types_1 = require("./types");
const errors_1 = require("./errors");
const allowDeny = /^allow|deny$/;
const validators = {
'price:currency': /^[A-Z]{3}$/,
'price:type': /^rent|purchase|RENT|PURCHASE$/,
'price:resolution': /^HD|hd|sd|SD$/,
'platform:relationship': allowDeny,
'restriction:relationship': allowDeny
};
function validateSMIOptions(conf, level = types_1.ErrorLevel.WARN) {
if (!conf) {
throw new errors_1.NoConfigError();
}
return timestamp;
if (level === types_1.ErrorLevel.SILENT) {
return conf;
}
const { url, changefreq, priority, news, video } = conf;
if (!url) {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.NoURLError();
}
else {
console.warn('URL is required');
}
}
if (changefreq) {
if (types_1.CHANGEFREQ.indexOf(changefreq) === -1) {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.ChangeFreqInvalidError();
}
else {
console.warn(`${url}: changefreq ${changefreq} is not valid`);
}
}
}
if (priority) {
if (!(priority >= 0.0 && priority <= 1.0) || typeof priority !== 'number') {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.PriorityInvalidError();
}
else {
console.warn(`${url}: priority ${priority} is not valid`);
}
}
}
if (news) {
if (news.access &&
news.access !== 'Registration' &&
news.access !== 'Subscription') {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.InvalidNewsAccessValue();
}
else {
console.warn(`${url}: news access ${news.access} is invalid`);
}
}
if (!news.publication ||
!news.publication.name ||
!news.publication.language ||
!news.publication_date ||
!news.title) {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.InvalidNewsFormat();
}
else {
console.warn(`${url}: missing required news property`);
}
}
}
if (video) {
video.forEach((vid) => {
if (vid.duration !== undefined) {
if (vid.duration < 0 || vid.duration > 28800) {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.InvalidVideoDuration();
}
else {
console.warn(`${url}: video duration ${vid.duration} is invalid`);
}
}
}
if (vid.rating !== undefined && (vid.rating < 0 || vid.rating > 5)) {
console.warn(`${url}: video ${vid.title} rating ${vid.rating} must be between 0 and 5 inclusive`);
}
if (typeof (vid) !== 'object' || !vid.thumbnail_loc || !vid.title || !vid.description) {
// has to be an object and include required categories https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.InvalidVideoFormat();
}
else {
console.warn(`${url}: missing required video property`);
}
}
if (vid.description.length > 2048) {
if (level === types_1.ErrorLevel.THROW) {
throw new errors_1.InvalidVideoDescription();
}
else {
console.warn(`${url}: video description is too long`);
}
}
Object.keys(vid).forEach((key) => {
// @ts-ignore
if (validators[key] && !validators[key].test(vid[key])) {
if (level === types_1.ErrorLevel.THROW) {
// @ts-ignore
throw new errors_1.InvalidAttrValue(key, vid[key], validators[key]);
}
else {
// @ts-ignore
console.warn(`${url}: video key ${key} has invalid value: ${vid[key]}`);
}
}
});
});
}
return conf;
}
exports.getTimestampFromDate = getTimestampFromDate;
exports.validateSMIOptions = validateSMIOptions;
/**
* Based on lodash's implementation of chunk.
*
* Copyright JS Foundation and other contributors <https://js.foundation/>
*
* Based on Underscore.js, copyright Jeremy Ashkenas,
* DocumentCloud and Investigative Reporters & Editors <http://underscorejs.org/>
*
* This software consists of voluntary contributions made by many
* individuals. For exact contribution history, see the revision history
* available at https://github.com/lodash/lodash
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
function chunk(array, size = 1) {
size = Math.max(Math.trunc(size), 0);
const length = array ? array.length : 0;
if (!length || size < 1) {
return [];
}
const result = Array(Math.ceil(length / size));
let index = 0, resIndex = 0;
while (index < length) {
result[resIndex++] = array.slice(index, (index += size));
}
return result;
}
exports.chunk = chunk;
//# sourceMappingURL=utils.js.map

55

package.json
{
"name": "sitemap",
"version": "3.2.2",
"version": "4.0.1",
"description": "Sitemap-generating framework",

@@ -21,2 +21,3 @@ "keywords": [

"types": "dist/index.d.ts",
"bin": "./dist/cli.js",
"directories": {

@@ -28,5 +29,8 @@ "lib": "lib",

"prepublishOnly": "sort-package-json && npm run test",
"test": "tsc && jest",
"test-perf": "node ./tests/perf.js",
"test:typecheck": "tsc"
"test": "eslint lib/* ./cli.ts && tsc && jest && npm run test:xmllint",
"test-fast": "jest ./tests/sitemap-item.test.ts ./tests/sitemap-index.test.ts ./tests/sitemap.test.ts ./tests/sitemap-shape.test.ts",
"test-perf": "node ./tests/perf.js > /dev/null",
"test:schema": "node tests/alltags.js | xmllint --schema schema/all.xsd --noout -",
"test:typecheck": "tsc",
"test:xmllint": "if which xmllint; then npm run test:schema; else echo 'skipping xml tests. xmllint not installed'; fi"
},

@@ -52,3 +56,3 @@ "husky": {

"parserOptions": {
"ecmaVersion": 6,
"ecmaVersion": 2018,
"sourceType": "module"

@@ -75,3 +79,6 @@ },

"@typescript-eslint/explicit-member-accessibility": "off",
"@typescript-eslint/interface-name-prefix": "always"
"@typescript-eslint/interface-name-prefix": [
2,
"always"
]
}

@@ -84,2 +91,3 @@ },

"!lib/**/*.d.ts",
"!lib/xmllint.ts",
"!node_modules/"

@@ -97,26 +105,19 @@ ],

"dependencies": {
"lodash.chunk": "^4.2.0",
"lodash.padstart": "^4.6.1",
"whatwg-url": "^7.0.0",
"@types/node": "^12.0.2",
"arg": "^4.1.1",
"xmlbuilder": "^13.0.0"
},
"devDependencies": {
"@babel/core": "^7.4.4",
"@babel/plugin-proposal-class-properties": "^7.4.4",
"@babel/plugin-transform-typescript": "^7.4.5",
"@babel/preset-env": "^7.4.4",
"@babel/core": "^7.5.5",
"@babel/plugin-proposal-class-properties": "^7.5.5",
"@babel/plugin-transform-typescript": "^7.5.5",
"@babel/preset-env": "^7.5.5",
"@babel/preset-typescript": "^7.3.3",
"@types/jest": "^24.0.12",
"@types/lodash.chunk": "^4.2.6",
"@types/lodash.padstart": "^4.6.6",
"@types/node": "^12.0.2",
"@types/url-join": "^4.0.0",
"@types/whatwg-url": "^6.4.0",
"@typescript-eslint/eslint-plugin": "^1.9.0",
"@typescript-eslint/parser": "^1.9.0",
"@types/jest": "^24.0.17",
"@typescript-eslint/eslint-plugin": "^1.13.0",
"@typescript-eslint/parser": "^1.13.0",
"babel-eslint": "^10.0.1",
"babel-polyfill": "^6.26.0",
"babel-preset-minify": "^0.5.0",
"eslint": "^5.0.0",
"husky": "^3.0.0",
"eslint": "^6.1.0",
"husky": "^3.0.3",
"jasmine": "^3.4.0",

@@ -127,9 +128,9 @@ "jest": "^24.8.0",

"stats-lite": "^2.2.0",
"typescript": "^3.4.5"
"typescript": "^3.5.3"
},
"engines": {
"node": ">=6.0.0",
"npm": ">=4.0.0"
"node": ">=8.9.0",
"npm": ">=5.6.0"
},
"License": "MIT"
}

529

README.md

@@ -1,2 +0,2 @@

sitemap.js
sitemap.js [![Build Status](https://travis-ci.org/ekalinin/sitemap.js.svg?branch=master)](https://travis-ci.org/ekalinin/sitemap.js)
==========

@@ -13,3 +13,2 @@

[![Build Status](https://travis-ci.org/ekalinin/sitemap.js.svg?branch=master)](https://travis-ci.org/ekalinin/sitemap.js)

@@ -19,30 +18,25 @@ Table of Contents

* [sitemap.js](#sitemapjs)
* [Table of Contents](#table-of-contents)
* [Installation](#installation)
* [Usage](#usage)
* [Example of using sitemap.js with <a href="https://github.com/visionmedia/express">express</a>:](#example-of-using-sitemapjs-with-express)
* [Example of synchronous sitemap.js usage:](#example-of-synchronous-sitemapjs-usage)
* [Example of dynamic page manipulations into sitemap:](#example-of-dynamic-page-manipulations-into-sitemap)
* [Example of pre-generating sitemap based on existing static files:](#example-of-pre-generating-sitemap-based-on-existing-static-files)
* [Example of images with captions:](#example-of-images-with-captions)
* [Example of indicating alternate language pages:](#example-of-indicating-alternate-language-pages)
* [Example of indicating Android app deep linking:](#example-of-indicating-android-app-deep-linking)
* [Example of Sitemap Styling](#example-of-sitemap-styling)
* [Example of mobile URL](#example-of-mobile-url)
* [Example of using HH:MM:SS in lastmod](#example-of-using-hhmmss-in-lastmod)
* [Example of Sitemap Index as String](#example-of-sitemap-index-as-string)
* [Example of Sitemap Index](#example-of-sitemap-index)
* [Example of overriding default xmlns* attributes in urlset element](#example-of-overriding-default-xmlns-attributes-in-urlset-element)
* [Example of news usage](#example-of-news)
* [Testing](#testing)
* [License](#license)
* [Installation](#installation)
* [Usage](#usage)
* [CLI](#CLI)
* [Example of using sitemap.js with <a href="https://expressjs.com/">express</a>:](#example-of-using-sitemapjs-with-express)
* [Example of dynamic page manipulations into sitemap:](#example-of-dynamic-page-manipulations-into-sitemap)
* [Example of most of the options you can use for sitemap](#example-of-most-of-the-options-you-can-use-for-sitemap)
* [Building just the sitemap index file](#building-just-the-sitemap-index-file)
* [Auto creating sitemap and index files from one large list](#auto-creating-sitemap-and-index-files-from-one-large-list)
* [API](#API)
* [Create Sitemap](#create-sitemap)
* [Sitemap](#sitemap)
* [buildSitemapIndex](#buildsitemapindex)
* [createSitemapIndex](#createsitemapindex)
* [Sitemap Item Options](#sitemap-item-options)
* [ISitemapImage](#ISitemapImage)
* [IVideoItem](#IVideoItem)
* [ILinkItem](#ILinkItem)
* [INewsItem](#INewsItem)
* [License](#license)
TOC created by [gh-md-toc](https://github.com/ekalinin/github-markdown-toc)
Installation
------------
It's recommended to install via [npm](https://github.com/isaacs/npm/):
npm install --save sitemap

@@ -52,12 +46,27 @@

-----
## CLI
Just feed the list of urls into sitemap
npx sitemap < listofurls.txt
Also supports line separated JSON for full configuration
npx sitemap --json < listofurls.txt
Or verify an existing sitemap
npx sitemap --verify sitemap.xml
## As a library
The main functions you want to use in the sitemap module are
```javascript
var sm = require('sitemap')
const { createSitemap } = require('sitemap')
// Creates a sitemap object given the input configuration with URLs
var sitemap = sm.createSitemap({ options });
// Generates XML with a callback function
sitemap.toXML( function(err, xml){ if (!err){ console.log(xml) } });
const sitemap = createSitemap({ options });
// Gives you a string containing the XML data
var xml = sitemap.toString();
const xml = sitemap.toString();
```

@@ -68,24 +77,26 @@

```javascript
var express = require('express')
, sm = require('sitemap');
const express = require('express')
const { createSitemap } = require('sitemap');
var app = express()
, sitemap = sm.createSitemap ({
hostname: 'http://example.com',
cacheTime: 600000, // 600 sec - cache purge period
urls: [
{ url: '/page-1/', changefreq: 'daily', priority: 0.3 },
{ url: '/page-2/', changefreq: 'monthly', priority: 0.7 },
{ url: '/page-3/'}, // changefreq: 'weekly', priority: 0.5
{ url: '/page-4/', img: "http://urlTest.com" }
]
});
const app = express()
const sitemap = createSitemap({
hostname: 'http://example.com',
cacheTime: 600000, // 600 sec - cache purge period
urls: [
{ url: '/page-1/', changefreq: 'daily', priority: 0.3 },
{ url: '/page-2/', changefreq: 'monthly', priority: 0.7 },
{ url: '/page-3/'}, // changefreq: 'weekly', priority: 0.5
{ url: '/page-4/', img: "http://urlTest.com" }
]
});
app.get('/sitemap.xml', function(req, res) {
sitemap.toXML( function (err, xml) {
if (err) {
return res.status(500).end();
}
res.header('Content-Type', 'application/xml');
res.send( xml );
try {
const xml = sitemap.toXML()
res.header('Content-Type', 'application/xml');
res.send( xml );
} catch (e) {
console.error(e)
res.status(500).end()
}
});

@@ -97,34 +108,9 @@ });

### Example of synchronous sitemap.js usage:
### Example of dynamic page manipulations into sitemap:
```javascript
var express = require('express')
, sm = require('sitemap');
var app = express()
, sitemap = sm.createSitemap ({
hostname: 'http://example.com',
cacheTime: 600000, // 600 sec cache period
urls: [
{ url: '/page-1/', changefreq: 'daily', priority: 0.3 },
{ url: '/page-2/', changefreq: 'monthly', priority: 0.7 },
{ url: '/page-3/' } // changefreq: 'weekly', priority: 0.5
]
});
app.get('/sitemap.xml', function(req, res) {
res.header('Content-Type', 'application/xml');
res.send( sitemap.toString() );
const sitemap = createSitemap ({
hostname: 'http://example.com',
cacheTime: 600000
});
app.listen(3000);
```
### Example of dynamic page manipulations into sitemap:
```javascript
var sitemap = sm.createSitemap ({
hostname: 'http://example.com',
cacheTime: 600000
});
sitemap.add({url: '/page-1/'});

@@ -138,226 +124,237 @@ sitemap.add({url: '/page-2/', changefreq: 'monthly', priority: 0.7});

### Example of pre-generating sitemap based on existing static files:
### Example of most of the options you can use for sitemap
```javascript
var sm = require('sitemap')
, fs = require('fs');
const { createSitemap } = require('sitemap');
var sitemap = sm.createSitemap({
hostname: 'http://www.mywebsite.com',
cacheTime: 600000, //600 sec (10 min) cache purge period
urls: [
{ url: '/' , changefreq: 'weekly', priority: 0.8, lastmodrealtime: true, lastmodfile: 'app/assets/index.html' },
{ url: '/page1', changefreq: 'weekly', priority: 0.8, lastmodrealtime: true, lastmodfile: 'app/assets/page1.html' },
{ url: '/page2' , changefreq: 'weekly', priority: 0.8, lastmodrealtime: true, lastmodfile: 'app/templates/page2.hbs' } /* useful to monitor template content files instead of generated static files */
]
const sitemap = createSitemap({
hostname: 'http://www.mywebsite.com',
level: 'warn', // default WARN about bad data
urls: [
{
url: '/page1',
changefreq: 'weekly',
priority: 0.8,
lastmodfile: 'app/assets/page1.html'
},
{
url: '/page2',
changefreq: 'weekly',
priority: 0.8,
/* useful to monitor template content files instead of generated static files */
lastmodfile: 'app/templates/page2.hbs'
},
// each sitemap entry supports many options
// See [Sitemap Item Options](#sitemap-item-options) below for details
{
url: 'http://test.com/page-1/',
img: [
{
url: 'http://test.com/img1.jpg',
caption: 'An image',
title: 'The Title of Image One',
geoLocation: 'London, United Kingdom',
license: 'https://creativecommons.org/licenses/by/4.0/'
},
{
url: 'http://test.com/img2.jpg',
caption: 'Another image',
title: 'The Title of Image Two',
geoLocation: 'London, United Kingdom',
license: 'https://creativecommons.org/licenses/by/4.0/'
}
],
video: [
{
thumbnail_loc: 'http://test.com/tmbn1.jpg',
title: 'A video title',
description: 'This is a video'
},
{
thumbnail_loc: 'http://test.com/tmbn2.jpg',
title: 'A video with an attribute',
description: 'This is another video',
'player_loc': 'http://www.example.com/videoplayer.mp4?video=123',
'player_loc:autoplay': 'ap=1'
}
],
links: [
{ lang: 'en', url: 'http://test.com/page-1/' },
{ lang: 'ja', url: 'http://test.com/page-1/ja/' }
],
androidLink: 'android-app://com.company.test/page-1/',
news: {
publication: {
name: 'The Example Times',
language: 'en'
},
genres: 'PressRelease, Blog',
publication_date: '2008-12-23',
title: 'Companies A, B in Merger Talks',
keywords: 'business, merger, acquisition, A, B',
stock_tickers: 'NASDAQ:A, NASDAQ:B'
}
}
]
});
fs.writeFileSync("app/assets/sitemap.xml", sitemap.toString());
```
### Example of images with captions:
### Building just the sitemap index file
The sitemap index file merely points to other sitemaps
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://test.com/page-1/',
img: [
{
url: 'http://test.com/img1.jpg',
caption: 'An image',
title: 'The Title of Image One',
geoLocation: 'London, United Kingdom',
license: 'https://creativecommons.org/licenses/by/4.0/'
},
{
url: 'http://test.com/img2.jpg',
caption: 'Another image',
title: 'The Title of Image Two',
geoLocation: 'London, United Kingdom',
license: 'https://creativecommons.org/licenses/by/4.0/'
}
]
}]
});
const { buildSitemapIndex } = require('sitemap')
const smi = buildSitemapIndex({
urls: ['https://example.com/sitemap1.xml', 'https://example.com/sitemap2.xml'],
xslUrl: 'https://example.com/style.xsl' // optional
});
```
### Example of videos:
### Auto creating sitemap and index files from one large list
[Description](https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190) specifications. Required fields are thumbnail_loc, title, and description.
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://test.com/page-1/',
video: [
{ thumbnail_loc: 'http://test.com/tmbn1.jpg', title: 'A video title', description: 'This is a video' },
{
thumbnail_loc: 'http://test.com/tmbn2.jpg',
title: 'A video with an attribute',
description: 'This is another video',
'player_loc': 'http://www.example.com/videoplayer.mp4?video=123',
'player_loc:autoplay': 'ap=1'
}
]
}]
});
const { createSitemapIndex } = require('sitemap')
const smi = createSitemapIndex({
cacheTime: 600000,
hostname: 'http://www.sitemap.org',
sitemapName: 'sm-test',
sitemapSize: 1,
targetFolder: require('os').tmpdir(),
urls: ['http://ya.ru', 'http://ya2.ru']
// optional:
// callback: function(err, result) {}
});
```
## API
### Example of indicating alternate language pages:
## Sitemap
[Description](https://support.google.com/webmasters/answer/2620865?hl=en) in
the google's Search Console Help.
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://test.com/page-1/',
changefreq: 'weekly',
priority: 0.3,
links: [
{ lang: 'en', url: 'http://test.com/page-1/', },
{ lang: 'ja', url: 'http://test.com/page-1/ja/', },
]
},]
});
```
### Example of indicating Android app deep linking:
[Description](https://developer.android.com/training/app-indexing/enabling-app-indexing.html#sitemap) in
the google's Search Console Help.
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://test.com/page-1/',
changefreq: 'weekly',
priority: 0.3,
androidLink: 'android-app://com.company.test/page-1/'
}]
});
const { Sitemap } = require('sitemap')
const sm = new Sitemap({
urls: [{url: '/path'}],
hostname: 'http://example.com',
cacheTime: 0, // default
level: 'warn' // default warns if it encounters bad data
})
sm.toString() // returns the xml as a string
```
### Example of Sitemap Styling
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://test.com/page-1/',
changefreq: 'weekly',
priority: 0.3,
links: [
{ lang: 'en', url: 'http://test.com/page-1/', },
{ lang: 'ja', url: 'http://test.com/page-1/ja/', },
]
},],
xslUrl: 'sitemap.xsl'
});
## buildSitemapIndex
Build a sitemap index file
```
const { buildSitemapIndex } = require('sitemap')
const index = buildSitemapIndex({
urls: [{url: 'http://example.com/sitemap-1.xml', lastmod: '2019-07-01'}, 'http://example.com/sitemap-2.xml'],
lastmod: '2019-07-29'
})
```
### Example of mobile URL
[Description](https://support.google.com/webmasters/answer/34648?hl=en) in
the google's Search Console Help.
```javascript
var sitemap = sm.createSitemap({
urls: [{
url: 'http://mobile.test.com/page-1/',
changefreq: 'weekly',
priority: 0.3,
mobile: true
},],
xslUrl: 'sitemap.xsl'
});
## createSitemapIndex
Create several sitemaps and an index automatically from a list of urls
```
const { createSitemapIndex } = require('sitemap')
createSitemapIndex({
urls: [/* list of urls */],
targetFolder: 'absolute path to target folder',
hostname: 'http://example.com',
cacheTime: 600,
sitemapName: 'sitemap',
sitemapSize: 50000, // number of urls to allow in each sitemap
xslUrl: '',// custom xsl url
gzip: false, // whether to gzip the files
callback: // called when complete;
})
```
### Example of using HH:MM:SS in lastmod
## Sitemap Item Options
```javascript
var sm = require('sitemap')
, sitemap = sm.createSitemap({
hostname: 'http://www.mywebsite.com',
urls: [{
url: 'http://mobile.test.com/page-1/',
lastmodISO: '2015-06-27T15:30:00.000Z',
changefreq: 'weekly',
priority: 0.3
}]
});
```
|Option|Type|eg|Description|
|------|----|--|-----------|
|url|string|http://example.com/some/path|The only required property for every sitemap entry|
|lastmod|string|'2019-07-29' or '2019-07-22T05:58:37.037Z'|When the page we as last modified use the W3C Datetime ISO8601 subset https://www.sitemaps.org/protocol.html#xmlTagDefinitions|
|changefreq|string|'weekly'|How frequently the page is likely to change. This value provides general information to search engines and may not correlate exactly to how often they crawl the page. Please note that the value of this tag is considered a hint and not a command. See https://www.sitemaps.org/protocol.html#xmlTagDefinitions for the acceptable values|
|priority|number|0.6|The priority of this URL relative to other URLs on your site. Valid values range from 0.0 to 1.0. This value does not affect how your pages are compared to pages on other sites—it only lets the search engines know which pages you deem most important for the crawlers. The default priority of a page is 0.5. https://www.sitemaps.org/protocol.html#xmlTagDefinitions|
|img|object[]|see [#ISitemapImage](#ISitemapImage)|https://support.google.com/webmasters/answer/178636?hl=en&ref_topic=4581190|
|video|object[]|see [#IVideoItem](#IVideoItem)|https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190|
|links|object[]|see [#ILinkItem](#ILinkItem)|Tell search engines about localized versions https://support.google.com/webmasters/answer/189077|
|news|object|see [#INewsItem](#INewsItem)|https://support.google.com/webmasters/answer/74288?hl=en&ref_topic=4581190|
|ampLink|string|'http://ampproject.org/article.amp.html'||
|mobile|boolean or string|||
|cdata|boolean|true|wrap url in cdata xml escape|
### Example of Sitemap Index as String
## ISitemapImage
```javascript
var sm = require('sitemap')
, smi = sm.buildSitemapIndex({
urls: ['https://example.com/sitemap1.xml', 'https://example.com/sitemap2.xml'],
xslUrl: 'https://example.com/style.xsl' // optional
});
```
Sitemap image
https://support.google.com/webmasters/answer/178636?hl=en&ref_topic=4581190
### Example of Sitemap Index
|Option|Type|eg|Description|
|------|----|--|-----------|
|url|string|'http://example.com/image.jpg'|The URL of the image.|
|caption|string - optional|'Here we did the stuff'|The caption of the image.|
|title|string - optional|'Star Wars EP IV'|The title of the image.|
|geoLocation|string - optional|'Limerick, Ireland'|The geographic location of the image.|
|license|string - optional|'http://example.com/license.txt'|A URL to the license of the image.|
```javascript
var sm = require('sitemap')
, smi = sm.createSitemapIndex({
cacheTime: 600000,
hostname: 'http://www.sitemap.org',
sitemapName: 'sm-test',
sitemapSize: 1,
targetFolder: require('os').tmpdir(),
urls: ['http://ya.ru', 'http://ya2.ru']
// optional:
// callback: function(err, result) {}
});
```
## IVideoItem
### Example of overriding default xmlns* attributes in urlset element
Sitemap video. https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190
Also see 'simple sitemap with dynamic xmlNs' test in [tests/sitemap.js](https://github.com/ekalinin/sitemap.js/blob/master/tests/sitemap.test.js)
|Option|Type|eg|Description|
|------|----|--|-----------|
|thumbnail_loc|string|"https://rtv3-img-roosterteeth.akamaized.net/store/0e841100-289b-4184-ae30-b6a16736960a.jpg/sm/thumb3.jpg"|A URL pointing to the video thumbnail image file|
|title|string|'2018:E6 - GoldenEye: Source'|The title of the video. |
|description|string|'We play gun game in GoldenEye: Source with a good friend of ours. His name is Gruchy. Dan Gruchy.'|A description of the video. Maximum 2048 characters. |
|content_loc|string - optional|"http://streamserver.example.com/video123.mp4"|A URL pointing to the actual video media file. Should be one of the supported formats.HTML is not a supported format. Flash is allowed, but no longer supported on most mobile platforms, and so may be indexed less well. Must not be the same as the <loc> URL.|
|player_loc|string - optional|"https://roosterteeth.com/embed/rouletsplay-2018-goldeneye-source"|A URL pointing to a player for a specific video. Usually this is the information in the src element of an <embed> tag. Must not be the same as the <loc> URL|
|'player_loc:autoplay'|string - optional|'ap=1'|a string the search engine can append as a query param to enable automatic playback|
|duration|number - optional| 600| duration of video in seconds|
|expiration_date| string - optional|"2012-07-16T19:20:30+08:00"|The date after which the video will no longer be available|
|view_count|string - optional|'21000000000'|The number of times the video has been viewed.|
|publication_date| string - optional|"2018-04-27T17:00:00.000Z"|The date the video was first published, in W3C format.|
|category|string - optional|"Baking"|A short description of the broad category that the video belongs to. This is a string no longer than 256 characters.|
|restriction|string - optional|"IE GB US CA"|Whether to show or hide your video in search results from specific countries.|
|restriction:relationship| string - optional|"deny"||
|gallery_loc| string - optional|"https://roosterteeth.com/series/awhu"|Currently not used.|
|gallery_loc:title|string - optional|"awhu series page"|Currently not used.|
|price|string - optional|"1.99"|The price to download or view the video. Omit this tag for free videos.|
|price:resolution|string - optional|"HD"|Specifies the resolution of the purchased version. Supported values are hd and sd.|
|price:currency| string - optional|"USD"|currency [Required] Specifies the currency in ISO 4217 format.|
|price:type|string - optional|"rent"|type [Optional] Specifies the purchase option. Supported values are rent and own. |
|uploader|string - optional|"GrillyMcGrillerson"|The video uploader's name. Only one <video:uploader> is allowed per video. String value, max 255 charactersc.|
|platform|string - optional|"tv"|Whether to show or hide your video in search results on specified platform types. This is a list of space-delimited platform types. See https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190 for more detail|
|platform:relationship|string 'Allow'\|'Deny' - optional|'Allow'||
|id|string - optional|||
|tag|string[] - optional|['Baking']|An arbitrary string tag describing the video. Tags are generally very short descriptions of key concepts associated with a video or piece of content.|
|rating|number - optional|2.5|The rating of the video. Supported values are float numbers i|
|family_friendly|string 'YES'\|'NO' - optional|'YES'||
|requires_subscription|string 'YES'\|'NO' - optional|'YES'|Indicates whether a subscription (either paid or free) is required to view the video. Allowed values are yes or no.|
|live|string 'YES'\|'NO' - optional|'NO'|Indicates whether the video is a live stream. Supported values are yes or no.|
```javascript
var sitemap = sm.createSitemapIndex({
xmlns: 'xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"'
});
```
## ILinkItem
### Example of news
https://support.google.com/webmasters/answer/189077
```javascript
const sm = require('sitemap')
const smi = new sm.SitemapItem({
url: 'http://www.example.org/business/article55.html',
news: {
publication: {
name: 'The Example Times',
language: 'en'
},
genres: 'PressRelease, Blog',
publication_date: '2008-12-23',
title: 'Companies A, B in Merger Talks',
keywords: 'business, merger, acquisition, A, B',
stock_tickers: 'NASDAQ:A, NASDAQ:B'
}
})
```
|Option|Type|eg|Description|
|------|----|--|-----------|
|lang|string|'en'||
|url|string|'http://example.com/en/'||
Testing
-------
## INewsItem
```bash
➥ git clone https://github.com/ekalinin/sitemap.js.git
➥ cd sitemap.js
➥ make env
➥ . env/bin/activate
(env) ➥ make test
./node_modules/expresso/bin/expresso ./tests/sitemap.test.js
https://support.google.com/webmasters/answer/74288?hl=en&ref_topic=4581190
100% 33 tests
|Option|Type|eg|Description|
|------|----|--|-----------|
|access|string - 'Registration' \| 'Subscription'| 'Registration' - optional||
|publication| object|see following options||
|publication['name']| string|'The Example Times'|The <name> is the name of the news publication. It must exactly match the name as it appears on your articles on news.google.com, except for anything in parentheses.|
|publication['language']|string|'en'|he <language> is the language of your publication. Use an ISO 639 language code (2 or 3 letters).|
|genres|string - optional|'PressRelease, Blog'||
|publication_date|string|'2008-12-23'|Article publication date in W3C format, using either the "complete date" (YYYY-MM-DD) format or the "complete date plus hours, minutes, and seconds"|
|title|string|'Companies A, B in Merger Talks'|The title of the news article.|
|keywords|string - optional|"business, merger, acquisition, A, B"||
|stock_tickers|string - optional|"NASDAQ:A, NASDAQ:B"||
```
License

@@ -364,0 +361,0 @@ -------

babel.config.js
dist/index.js.map

Sorry, the diff of this file is not supported yet

dist/lib/errors.js.map

Sorry, the diff of this file is not supported yet

dist/lib/sitemap-index.js.map

Sorry, the diff of this file is not supported yet

dist/lib/sitemap-item.js.map

Sorry, the diff of this file is not supported yet

dist/lib/sitemap.js.map

Sorry, the diff of this file is not supported yet

dist/lib/types.js.map

Sorry, the diff of this file is not supported yet

dist/lib/utils.js.map

Sorry, the diff of this file is not supported yet

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc