dutest

An object-oriented API to test doctests using unittest runners.

Module providing classes which extend doctest module so as to achieve better integration with unittest.

It is different from the Pyhton 2.4 doctest unittest API because:

• A new unitest.TestLoader descendant now allows to load instances of TestCases for doctests using unittest-style, supports building complex test suites in a more natural way, and eases the use of specialized instances of TestCase built out of doctest examples.

• Other loaders allow users to extract TestCase instances out of TestCase descendants and doctests (and any other format) in a single step.

• In this case unittest.TestResult instances report whether individual examples have been successfully executed, or otherwise have failed or raised an unexpected exception. Formerly TestResult objects contained the whole report outputted by doctest module.

• Test analysis require no further parsing to retrieve detailed information about failures.

• A whole new unittest API for doctest adds object orientation and eliminates functions with big signatures.

• It is not necessary to use DocTestRunner output streams in order to collect test results.

• A new hierarchy of doctest TestCases is now possible so for example, setUp and tearDown may be redefined across a hierarchy of TestCases instead of providing this methods as parameters to a function (breaking OOP philosophy and logic); or maybe even failures and errors can be represented in a custom way.

• Allows to perform regression testing over tests written using doctest.

• Fixes a minor bug related with specifying different verbosity levels from the command line to unittest.TestProgram (alias main).

• Loads by default test cases for doctests plus those formerly loaded by unittest.TestLoader

It is similar to the Pyhton 2.4 doctest unittest API because:

• Provides integration with TestProgram and unittest test runners.

• Allows to parameterize doctest behavior via doctest options

A fuller explanation can be found in the following article:

"Doctest and unittest... now they'll live happily together", O. Lang
(2008) The Python Papers, Volume 3, Issue 1, pp. 31:51

Note: The contents of this module were first implemented by the module oop.utils.testing contained in PyOOP package_.

.. _PyOOP package: http://pypi.python.org/pypi/PyOOP

What's new in version 0.2.3:

• Implemented Suite Fixture Setup test pattern (i.e. set up a fixture once before all its interactive examples are run and clean up after they all have been run.

• Module is hidden in tracebacks written to instances of TestResult when a failure or an error is detected.

• Support for multiple pattern matching styles has been added in PackageTestLoader trough style parameter. By default standard regular expressions (i.e. style=REGEX) are used.

• Unix filename pattern matching (fnmatch module) is available from now on while using PackageTestLoader. The only thing needed is to specify style=UNIX.

• Multiple functions in previous versions contained arguments having mutable objects as default values. A
  message sent to TiP list about default args_ explains why this is really harmful. This has been fixed. Thanks to Scott David Daniels for reporting the issue with default args_ and Jesse Noller for the pointers about default args_ .

.. _message sent to TiP list about default args: http://lists.idyll.org/pipermail/testing-in-python/2009-April/001617.html

.. _for reporting the issue with default args: http://lists.idyll.org/pipermail/testing-in-python/2009-April/001606.html

.. _the pointers about default args: http://lists.idyll.org/pipermail/testing-in-python/2009-April/001620.html

What's new in version 0.2.2:

• Bug fixed... The class PackageTestLoader loaded the tests defined in a packages' children but not those specified in the top-level module. Now the tests specified in the module supplied in to loadTestsFromModule method are also included in the resulting test suite.

What's new in version 0.2.1:

• PackageTestLoader class added to dutest module. It loads all the tests found throughout a package hierarchy using another loader. The later is used for retrieving the tests contained inside every module matching a specified pattern.

• DocTestLoader class in oop.utils.dutest does not raise ValueError exception if no doctests are found in a module. This is doctest behavior. It was assumed up to this point, but now it has proven to be a headache when these loaders are used together with instances of PackageTestLoader in order to retrieve all the tests defined across a package hierarchy.

What's new in version 0.1.2:

• Bug fixed... DocTestCase instances could not be instantiated because None was supplied in to the initializer as the test method name. This bug remained hidden somehow while executing the test using oop.test.check_oopdbc(), and even when importing oop.utils.dutest (which is actually the same implementation).

What's new in version 0.1.1:

• The class oop.utils.testing.VerboseTestProgram has been moved onto dutest module. Release 0.1.0 did not include it. Thanks to Michal Kwiatkowski for notifying me so [1]_.

• Default test loader and runner have been added to this module as well. The default loader is an instance of MultiTestLoader that combines the suites loaded by unittest.TestLoader and dutest.DocTestLoader.

• dutest.main is an alias for dutest.VerboseTestProgram. This class fixes a minor bug (... IMO) I found while specifying different verbosity levels from the command line to unittest.TestProgram. It also employs by default dutest.defaultTestLoader instead of unittest.defaultTestLoader.

.. [1] [TIP] Fwd: An OO API for doctest / unittest integration... (http://lists.idyll.org/pipermail/testing-in-python/2008-August/000918.html)

What's new in version 0.1.0:

doctest / unittest extensions:

• A whole new unittest API to integrate doctest with unittest runners.

• It allows to report the individual results of each and every interactive example executed during the test run. A separate entry is created in the corresponding TestResult instance containing the expected value and the actual result.

• Since the match made for individual examples is reported one by one by instances of TestResult, it is quite easier to automate post-testing activities (e.g. automated test analysis). Formerly access to this information required extra work (i.e. the full report provided by doctest had to be parsed).

• You do not need to use DocTestRunner output streams to collect test results.

• A new hierarchy of TestCase descendants (extensions of DocTestCase class) is now possible so for example, setUp and tearDown may be redefined across a hierarchy of TestCase s instead of providing this methods as parameters to a function (breaking OOP philosophy and logic); or maybe even to represent failures and errors in a custom way.

• A new TestLoader descendant now allows to load (using unittest-style) TestCase s which check the match made for doctests. It provides integration with TestProgram, supports building complex TestSuite s in a more natural way, and eases the use of specialized instances of TestCase s built out of doctest examples.

• Allows to perform regression testing over tests written using doctest.