webchk

====== webchk

.. image:: https://img.shields.io/pypi/v/webchk.svg :target: https://pypi.org/project/webchk/

.. image:: https://img.shields.io/travis/amgedr/webchk.svg :target: https://travis-ci.org/amgedr/webchk

.. image:: https://img.shields.io/github/license/amgedr/webchk.svg :target: https://github.com/amgedr/webchk/blob/master/LICENSE

webchk is a command-line tool developed in Python 3 for checking the HTTP status codes and response headers of URLs. It accepts one or more URLs as arguments. Furthermore, a sitemap URL can be passed using the -p option to download its content, extract the URLs and check their statuses.

Installation

webchk is available on PyPI and can be installed using pip with the following command::

$ pip install webchk

Webchk does not require any 3rd party packages to run. So it can also be cloned from GitHub and run as a module::

$ git clone https://github.com/amgedr/webchk.git
$ cd webchk
$ python3 -m webchk

Usage

::

webchk [-h] [-i INPUT] [-o OUTPUT] [-p] [-a] [-l] [-s] [-f] [-v] [urls [urls ...]]

positional arguments: urls

optional arguments: -h, --help show this help message and exit -i INPUT, --input INPUT Read input from a file -o OUTPUT, --output OUTPUT Save output to a file -p, --parse Follow links listed in .xml URLs -l, --list Print URLs without checking them -v, --version Print the version number

Examples

Check a list of URLs from a file (one URL per line)::

    $ webchk -i urls.txt

Check the status of a sitemap file and all the URLs listed in it::

    $ webchk -p http://example.com/sitemap.xml

List the URLs in a file without checking their HTTP status::

    $ webchk -li urls.txt

Check the URLs in a file and .xml files in it::

    $ webchk -pi urls.txt

List the URLs in a file and .xml files in it::

    $ webchk -pli urls.txt

List the URLs in a sitemap without checking their status::

    $ webchk -lp http://example.com/sitemap.xml



=======
History
=======

1.2.0 (2022-02-19)
------------------

* Default HTTP method is now HEAD
* Added --get to switch to HTTP GET
* Added --agent to set the User-Agent
* Added --auth for adding Authorization header
* Prevent timeouts less than 1
* Display results immediately instead of at the end


1.1.0 (2022-02-05)
------------------

* Handle connection related errors
* Improve reporting of HTTP redirects
* Added --timeout or -t to modify respose deadline


1.0.4 (2020-08-29)
------------------

* Fix crash when an URL check times out
* Fix --output command-line option


1.0.3 (2020-07-10)
------------------

* Modify the Python versions tox tests
* Fix failing unit tests


1.0.2 (2020-05-08)
------------------

* Add Python versions 3.7 and 3.8 to the list of tested versions


1.0.1 (2020-05-08)
------------------

* Fixed: Parsing sitemaps enters an endless loop
* Fixed: Parsing a URL that does not exists exits with an unhandled exception


1.0.0 (2018-12-06)
------------------

* Linked to http://codehill.com/projects/webchk/ instead of readthedocs.io


0.3.0 (2018-03-24)
------------------

* Run each check in its own thread


0.2.1 (2017-12-19)
------------------

* Fixed: Status code description not being displayed
* Improved PyPI and GitHub README


0.2.0 (2017-12-14)
------------------

* Code refactoring
* Created setup.py