link-checker

Link Checker

Link checker for HTML pages which checks href attributes including the anchor in the target.
The Command Line Interface expects a directory on your local file system which will be scanned.

Why did I wrote this tool?

I was using a nice CLI called html-proofer, but was using a preprocessing step in order to get Javadoc and Scaladoc working because of the iframe setup. At some point it didn't scale anymore. Scaladoc link checker with html-proofer took 5 minutes.

link-checker is using cheerio for parsing HTML, which is using the fastest HTML parser for Node.js: htmlparser2. Same Scaladoc which took 5 minutes with html-proofer takes now 5 seconds with link-checker. Also URL transformation for iframes can be turned on on-the-fly via --javadoc. In this mode links like /index.html#com.org.company.product.library.Main@init will check for a HTML in the pathcom/org/company/product/library/Main.html and the anchor init.

FAQ

I need to check links on a website via http(s)

Just use a website-scraper and download all the pages to your file system.

I've used the module with this options:

{
  urls: [urlToScrape],
  directory: outputDirectory,
  recursive: true,
  filenameGenerator: 'bySiteStructure',
  urlFilter: function(url) {
      return url.indexOf(urlToScrape) != -1;
  }
}

Installation

NPM

You can install it via npm

npm install -g link-checker

You can also install it without -g but then you need to put the binary, located in node_modules/.bin/link-checker to your $PATH.

Docker

https://hub.docker.com/r/timaschew/link-checker/

docker pull timaschew/link-checker

Usage

You need to pass exactly one path where to check links
Usage: link-checker path [options]

Options:
  --version             Show version number                            [boolean]
  --allow-hash-href     If `true`, ignores the `href` `#`              [boolean]
  --disable-external    disable checks HTTP links                      [boolean]
  --external-only       check HTTP links only                          [boolean]
  --file-ignore         RegExp to ignore files to scan                   [array]
  --url-ignore          RegExp to ignore URLs                            [array]
  --url-swap            RegExp for URLs which can be replaced on the fly [array]
  --limit-scope         forbid to follow URLs which are out of provided path,
                        like ../somewhere                              [boolean]
  --mkdocs              transforming URLS from foo/#bar to foo/index.html#bar
                                                                       [boolean]
  --javadoc             Enable special URL transforming which allows to check
                        iframe deeplinks for local javadoc and scaladoc[boolean]
  --javadoc-external    Domain or base URL to do URL transformation to check
                        iframe deeplinks                                 [array]
  --http-status-ignore  pass HTTP status code which will be ignore, by default
                        only 2xx are allowed                             [array]
  --json                print errors as JSON                           [boolean]
  --http-redirects      Amount of allowed HTTP redirects            [default: 0]
  --http-timeout        HTTP timeout in milliseconds             [default: 5000]
  --http-always-get     Use always HTTP GET requests, by default HEAD is used
                        for pages without any anchors                  [boolean]
  --warn-name-attr      show warning if name attribute instead of id was used
                        for an anchor                                  [boolean]
  --http-cache          Directory to store the non failing HTTP responses. If
                        none is specified responses won't be cached.    [string]
  --http-cache-max-age  Invalidate the cache after the given period. Allowed
                        values: https://www.npmjs.com/package/ms [default: "1w"]
  -h, --help            Show help                                      [boolean]

Examples:
  link-checker path/to/html/files  checks directory with HTMLfiles for broken
                                   links and anchors

linkcheckerrc configuration

The above configuration can, alternatively or in addition, be provided by a .linkcheckerrc in the project root:

{
    "allow-hash-href": true,
    "disable-external": true,
    ...
}

In addition, this format also provides means to override these settings based on URL regular expression matching:

{
    "overrides": {
        "https://www\\.google.com/#": {
            "allow-hash-href": true,
            "http-status-ignore": [403, 404]
        },
        "marketplace\\.visualstudio\\.com": {
            "http-always-get": true
        }
    }
}