🎩 You're Invited:Meet the Socket team at Black Hat in Las Vegas, August 3-6.RSVP →
Sign In

python-subunit

Package Overview
Dependencies
Maintainers
3
Versions
28
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

python-subunit - pypi Package Compare versions

Comparing version
1.4.4
 to
1.4.5
.codespellrc

Sorry, the diff of this file is not supported yet

.git-blame-ignore-revs

Sorry, the diff of this file is not supported yet

+13
.github/dependabot.yaml
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: weekly
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: weekly
+43
.github/workflows/main.yml
name: Tests
on:
push:
branches: [master]
pull_request:
branches: [master]
jobs:
tests:
name: tests-python${{ matrix.python-version }}-${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "pypy3.10"]
os: ["ubuntu-latest"]
steps:
- uses: actions/checkout@v5
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install Deps
run: sudo apt-get install check libcppunit-dev
- name: Install package
run: |
python -m pip install ruff
python -m pip install -U '.[test,docs]'
- name: Build
run: autoreconf -fi && ./configure && make
- name: Run ruff check
run: ruff check python
- name: Format
run: ruff format --check .
- name: Run make check
run: make check
- name: Run make distcheck
run: make distcheck
# Disabled; needs a tuit or two
# - name: Docs build
# run: md2html.py README.md README.html
# if: "matrix.python-version == '3.8'"
# - name: Docs build
# run: md2html README.md README.html
# if: "matrix.python-version != '3.8'"
+35
all_tests.py
#
# subunit: extensions to Python unittest to get test results from subprocesses.
# Copyright (C) 2013 Robert Collins <robertc@robertcollins.net>
#
# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
# license at the users choice. A copy of both licenses are available in the
# project source as Apache-2.0 and BSD. You may not use this file except in
# compliance with one of these two licences.
#
# Unless required by applicable law or agreed to in writing, software
# distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# license you chose for the specific language governing permissions and
# limitations under that license.
#
import unittest
import subunit
class ShellTests(subunit.ExecTestCase):
def test_sourcing(self):
"""./shell/tests/test_source_library.sh"""
def test_functions(self):
"""./shell/tests/test_function_output.sh"""
def test_suite():
result = unittest.TestSuite()
result.addTest(subunit.test_suite())
result.addTest(ShellTests("test_sourcing"))
result.addTest(ShellTests("test_functions"))
return result
Apache-2.0

Sorry, the diff of this file is not supported yet

BSD

Sorry, the diff of this file is not supported yet

+24
disperse.toml
tag-name = "$VERSION"
news-file = "NEWS"
verify-command = "autoreconf -i && ./configure && make distcheck"
tarball-location = []
release-timeout = 5
[[update_version]]
path = "configure.ac"
new-line = "m4_define([SUBUNIT_MAJOR_VERSION], [$MAJOR_VERSION])"
[[update_version]]
path = "configure.ac"
new-line = "m4_define([SUBUNIT_MINOR_VERSION], [$MINOR_VERSION])"
[[update_version]]
path = "configure.ac"
new-line = "m4_define([SUBUNIT_MICRO_VERSION], [$MICRO_VERSION])"
[[update_version]]
path = "python/subunit/__init__.py"
new-line = "__version__ = $STATUS_TUPLED_VERSION"
[launchpad]
project = "subunit"
+1
python/python_subunit.egg-info/dependency_links.txt
+15
python/python_subunit.egg-info/entry_points.txt
[console_scripts]
subunit-1to2 = subunit.filter_scripts.subunit_1to2:main
subunit-2to1 = subunit.filter_scripts.subunit_2to1:main
subunit-filter = subunit.filter_scripts.subunit_filter:main
subunit-ls = subunit.filter_scripts.subunit_ls:main
subunit-notify = subunit.filter_scripts.subunit_notify:main
subunit-output = subunit.filter_scripts.subunit_output:main
subunit-stats = subunit.filter_scripts.subunit_stats:main
subunit-tags = subunit.filter_scripts.subunit_tags:main
subunit2csv = subunit.filter_scripts.subunit2csv:main
subunit2disk = subunit.filter_scripts.subunit2disk:main
subunit2gtk = subunit.filter_scripts.subunit2gtk:main
subunit2junitxml = subunit.filter_scripts.subunit2junitxml:main
subunit2pyunit = subunit.filter_scripts.subunit2pyunit:main
tap2subunit = subunit.filter_scripts.tap2subunit:main
+514
python/python_subunit.egg-info/PKG-INFO
Metadata-Version: 2.4
Name: python-subunit
Version: 1.4.5
Summary: Python implementation of subunit test streaming protocol
Author-email: Robert Collins <subunit-dev@lists.launchpad.net>
License: Apache-2.0 or BSD
Project-URL: Bug Tracker, https://bugs.launchpad.net/subunit
Project-URL: Homepage, http://launchpad.net/subunit
Project-URL: Source Code, https://github.com/testing-cabal/subunit/
Keywords: python,streaming,test
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: COPYING
Requires-Dist: iso8601
Requires-Dist: testtools>=2.7
Provides-Extra: docs
Requires-Dist: docutils; extra == "docs"
Provides-Extra: test
Requires-Dist: fixtures; extra == "test"
Requires-Dist: hypothesis; extra == "test"
Requires-Dist: testscenarios; extra == "test"
Dynamic: license-file
subunit: A streaming protocol for test results
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
See the COPYING file for full details on the licensing of Subunit.
# Subunit
Subunit is a streaming protocol for test results.
There are two major revisions of the protocol. Version 1 was trivially human
readable but had significant defects as far as highly parallel testing was
concerned - it had no room for doing discovery and execution in parallel,
required substantial buffering when multiplexing and was fragile - a corrupt
byte could cause an entire stream to be misparsed. Version 1.1 added
encapsulation of binary streams which mitigated some of the issues but the
core remained.
Version 2 shares many of the good characteristics of Version 1 - it can be
embedded into a regular text stream (e.g. from a build system) and it still
models xUnit style test execution. It also fixes many of the issues with
Version 1 - Version 2 can be multiplexed without excessive buffering (in
time or space), it has a well defined recovery mechanism for dealing with
corrupted streams (e.g. where two processes write to the same stream
concurrently, or where the stream generator suffers a bug).
More details on both protocol version s can be found in the 'Protocol' section
of this document.
Subunit comes with command line filters to process a subunit stream and
language bindings for python, C, C++ and shell. Bindings are easy to write
for other languages.
A number of useful things can be done easily with subunit:
* Test aggregation: Tests run separately can be combined and then
reported/displayed together. For instance, tests from different languages
can be shown as a seamless whole, and tests running on multiple machines
can be aggregated into a single stream through a multiplexer.
* Test archiving: A test run may be recorded and replayed later.
* Test isolation: Tests that may crash or otherwise interact badly with each
other can be run separately and then aggregated, rather than interfering
with each other or requiring an adhoc test->runner reporting protocol.
* Grid testing: subunit can act as the necessary serialisation and
deserialiation to get test runs on distributed machines to be reported in
real time.
Subunit supplies the following filters:
* tap2subunit - convert perl's TestAnythingProtocol to subunit.
* subunit2csv - convert a subunit stream to csv.
* subunit2disk - export a subunit stream to files on disk.
* subunit2pyunit - convert a subunit stream to pyunit test results.
* subunit2gtk - show a subunit stream in GTK.
* subunit2junitxml - convert a subunit stream to JUnit's XML format.
* subunit-diff - compare two subunit streams.
* subunit-filter - filter out tests from a subunit stream.
* subunit-ls - list info about tests present in a subunit stream.
* subunit-stats - generate a summary of a subunit stream.
* subunit-tags - add or remove tags from a stream.
## Integration with other tools
Subunit's language bindings act as integration with various test runners like
'check', 'cppunit', Python's 'unittest'. Beyond that a small amount of glue
(typically a few lines) will allow Subunit to be used in more sophisticated
ways.
### Python
Subunit has excellent Python support: most of the filters and tools are written
in python and there are facilities for using Subunit to increase test isolation
seamlessly within a test suite.
The most common way is to run an existing python test suite and have it output
subunit via the ``subunit.run`` module::
$ python -m subunit.run mypackage.tests.test_suite
For more information on the Python support Subunit offers , please see
``pydoc subunit``, or the source in ``python/subunit/``
### Rust
There is a rust crate [subunit](https://crates.io/crates/subunit) that can be used to parse v1 and v1, and emit v2 subunit streams.
### C
Subunit has C bindings to emit the protocol. The 'check' C unit testing project
has included subunit support in their project for some years now. See
'c/README' for more details.
### C++
The C library is includable and usable directly from C++. A TestListener for
CPPUnit is included in the Subunit distribution. See 'c++/README' for details.
### shell
There are two sets of shell tools. There are filters, which accept a subunit
stream on stdin and output processed data (or a transformed stream) on stdout.
Then there are unittest facilities similar to those for C : shell bindings
consisting of simple functions to output protocol elements, and a patch for
adding subunit output to the 'ShUnit' shell test runner. See 'shell/README' for
details.
#### Filter recipes
To ignore some failing tests whose root cause is already known::
subunit-filter --without 'AttributeError.*flavor'
## The xUnit test model
Subunit implements a slightly modified xUnit test model. The stock standard
model is that there are tests, which have an id(), can be run, and when run
start, emit an outcome (like success or failure) and then finish.
Subunit extends this with the idea of test enumeration (find out about tests
a runner has without running them), tags (allow users to describe tests in
ways the test framework doesn't apply any semantic value to), file attachments
(allow arbitrary data to make analysing a failure easy) and timestamps.
## The protocol
Version 2, or v2 is new and still under development, but is intended to
supersede version 1 in the very near future. Subunit's bundled tools accept
only version 2 and only emit version 2, but the new filters subunit-1to2 and
subunit-2to1 can be used to interoperate with older third party libraries.
### Version 2
Version 2 is a binary protocol consisting of independent packets that can be
embedded in the output from tools like make - as long as each packet has no
other bytes mixed in with it (which 'make -j N>1' has a tendency of doing).
Version 2 is currently in draft form, and early adopters should be willing
to either discard stored results (if protocol changes are made), or bulk
convert them back to v1 and then to a newer edition of v2.
The protocol synchronises at the start of the stream, after a packet, or
after any 0x0A byte. That is, a subunit v2 packet starts after a newline or
directly after the end of the prior packet.
Subunit is intended to be transported over a reliable streaming protocol such
as TCP. As such it does not concern itself with out of order delivery of
packets. However, because of the possibility of corruption due to either
bugs in the sender, or due to mixed up data from concurrent writes to the same
fd when being embedded, subunit strives to recover reasonably gracefully from
damaged data.
A key design goal for Subunit version 2 is to allow processing and multiplexing
without forcing buffering for semantic correctness, as buffering tends to hide
hung or otherwise misbehaving tests. That said, limited time based buffering
for network efficiency is a good idea - this is ultimately implementer
choice. Line buffering is also discouraged for subunit streams, as dropping
into a debugger or other tool may require interactive traffic even if line
buffering would not otherwise be a problem.
In version two there are two conceptual events - a test status event and a file
attachment event. Events may have timestamps, and the path of multiplexers that
an event is routed through is recorded to permit sending actions back to the
source (such as new tests to run or stdin for driving debuggers and other
interactive input). Test status events are used to enumerate tests, to report
tests and test helpers as they run. Tests may have tags, used to allow
tunnelling extra meanings through subunit without requiring parsing of
arbitrary file attachments. Things that are not standalone tests get marked
as such by setting the 'Runnable' flag to false. (For instance, individual
assertions in TAP are not runnable tests, only the top level TAP test script
is runnable).
File attachments are used to provide rich detail about the nature of a failure.
File attachments can also be used to encapsulate stdout and stderr both during
and outside tests.
Most numbers are stored in network byte order - Most Significant Byte first
encoded using a variation of http://www.dlugosz.com/ZIP2/VLI.html. The first
byte's top 2 high order bits encode the total number of octets in the number.
This encoding can encode values from 0 to 2**30-1, enough to encode a
nanosecond. Numbers that are not variable length encoded are still stored in
MSB order.
+--------+--------+---------+------------+
| prefix | octets | max | max |
+========+========+=========+============+
| 00 | 1 | 2**6-1 | 63 |
+--------+--------+---------+------------+
| 01 | 2 | 2**14-1 | 16383 |
+--------+--------+---------+------------+
| 10 | 3 | 2**22-1 | 4194303 |
+--------+--------+---------+------------+
| 11 | 4 | 2**30-1 | 1073741823 |
+--------+--------+---------+------------+
All variable length elements of the packet are stored with a length prefix
number allowing them to be skipped over for consumers that don't need to
interpret them.
UTF-8 strings are with no terminating NUL and should not have any embedded NULs
(implementations SHOULD validate any such strings that they process and take
some remedial action (such as discarding the packet as corrupt).
In short the structure of a packet is:
PACKET := SIGNATURE FLAGS PACKET_LENGTH TIMESTAMP? TESTID? TAGS? MIME?
FILECONTENT? ROUTING_CODE? CRC32
In more detail...
Packets are identified by a single byte signature - 0xB3, which is never legal
in a UTF-8 stream as the first byte of a character. 0xB3 starts with the first
bit set and the second not, which is the UTF-8 signature for a continuation
byte. 0xB3 was chosen as 0x73 ('s' in ASCII') with the top two bits replaced by
the 1 and 0 for a continuation byte.
If subunit packets are being embedded in a non-UTF-8 text stream, where 0x73 is
a legal character, consider either recoding the text to UTF-8, or using
subunit's 'file' packets to embed the text stream in subunit, rather than the
other way around.
Following the signature byte comes a 16-bit flags field, which includes a
4-bit version field - if the version is not 0x2 then the packet cannot be
read. It is recommended to signal an error at this point (e.g. by emitting
a synthetic error packet and returning to the top level loop to look for
new packets, or exiting with an error). If recovery is desired, treat the
packet signature as an opaque byte and scan for a new synchronisation point.
NB: Subunit V1 and V2 packets may legitimately included 0xB3 internally,
as they are an 8-bit safe container format, so recovery from this situation
may involve an arbitrary number of false positives until an actual packet
is encountered : and even then it may still be false, failing after passing
the version check due to coincidence.
Flags are stored in network byte order too.
```asciiart
+------------+------------+------------------------+
| High byte | Low byte |
+------------+------------+------------------------+
| 15 14 13 12 11 10 9 8 | 7 6 5 4 3 2 1 0 |
+------------+------------+------------------------+
| VERSION | feature bits |
+------------+-------------------------------------+
```
Valid version values are:
`0x2` - version 2
Feature bits:
| Bit position | Mask | Description|
|---------|-------------|---------------------------|
| Bit 11 | `0x0800` | Test id present. |
| Bit 10 | `0x0400` | Routing code present. |
| Bit 9 | `0x0200` | Timestamp present. |
| Bit 8 | `0x0100` | Test is 'runnable'. |
| Bit 7 | `0x0080` | Tags are present. |
| Bit 6 | `0x0040` | File content is present. |
| Bit 5 | `0x0020` | File MIME type is present.|
| Bit 4 | `0x0010` | EOF marker. |
| Bit 3 | `0x0008` | Must be zero in version 2.|
Test status gets three bits:
Bit 2 | Bit 1 | Bit 0 - mask 0x0007 - A test status enum lookup:
* 000 - undefined / no test
* 001 - Enumeration / existence
* 002 - In progress
* 003 - Success
* 004 - Unexpected Success
* 005 - Skipped
* 006 - Failed
* 007 - Expected failure
After the flags field is a number field giving the length in bytes for the
entire packet including the signature and the checksum. This length must
be less than 4MiB - 4194303 bytes. The encoding can obviously record a larger
number but one of the goals is to avoid requiring large buffers, or causing
large latency in the packet forward/processing pipeline. Larger file
attachments can be communicated in multiple packets, and the overhead in such a
4MiB packet is approximately 0.2%.
The rest of the packet is a series of optional features as specified by the set
feature bits in the flags field. When absent they are entirely absent.
Forwarding and multiplexing of packets can be done without interpreting the
remainder of the packet until the routing code and checksum (which are both at
the end of the packet). Additionally, routers can often avoid copying or moving
the bulk of the packet, as long as the routing code size increase doesn't force
the length encoding to take up a new byte (which will only happen to packets
less than or equal to 16KiB in length) - large packets are very efficient to
route.
Timestamp when present is a 32 bit unsigned integer for seconds, and a variable
length number for nanoseconds, representing UTC time since Unix Epoch in
seconds and nanoseconds.
Test id when present is a UTF-8 string. The test id should uniquely identify
runnable tests such that they can be selected individually. For tests and other
actions which cannot be individually run (such as test
fixtures/layers/subtests) uniqueness is not required (though being human
meaningful is highly recommended).
Tags when present is a length prefixed vector of UTF-8 strings, one per tag.
There are no restrictions on tag content (other than the restrictions on UTF-8
strings in subunit in general). Tags have no ordering.
When a MIME type is present, it defines the MIME type for the file across all
packets same file (routing code + testid + name uniquely identifies a file,
reset when EOF is flagged). If a file never has a MIME type set, it should be
treated as application/octet-stream.
File content when present is a UTF-8 string for the name followed by the length
in bytes of the content, and then the content octets.
If present routing code is a UTF-8 string. The routing code is used to
determine which test backend a test was running on when doing data analysis,
and to route stdin to the test process if interaction is required.
Multiplexers SHOULD add a routing code if none is present, and prefix any
existing routing code with a routing code ('/' separated) if one is already
present. For example, a multiplexer might label each stream it is multiplexing
with a simple ordinal ('0', '1' etc), and given an incoming packet with route
code '3' from stream '0' would adjust the route code when forwarding the packet
to be '0/3'.
Following the end of the packet is a CRC-32 checksum of the contents of the
packet including the signature.
#### Example packets
Trivial test "foo" enumeration packet, with test id, runnable set,
status=enumeration. Spaces below are to visually break up signature / flags /
length / testid / crc32
b3 2901 0c 03666f6f 08555f1b
### Version 1 (and 1.1)
Version 1 (and 1.1) are mostly human readable protocols.
#### Sample subunit wire contents
The following:
```subunit
test: test foo works
success: test foo works
test: tar a file.
failure: tar a file. [
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
]
a writeln to stdout
When run through subunit2pyunit::
.F
a writeln to stdout
========================
FAILURE: tar a file.
-------------------
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
```
#### Subunit v1 protocol description
This description is being ported to an EBNF style. Currently its only partly in
that style, but should be fairly clear all the same. When in doubt, refer the
source (and ideally help fix up the description!). Generally the protocol is
line orientated and consists of either directives and their parameters, or
when outside a DETAILS region unexpected lines which are not interpreted by
the parser - they should be forwarded unaltered:
```ebnf
test|testing|test:|testing: test LABEL
success|success:|successful|successful: test LABEL
success|success:|successful|successful: test LABEL DETAILS
failure: test LABEL
failure: test LABEL DETAILS
error: test LABEL
error: test LABEL DETAILS
skip[:] test LABEL
skip[:] test LABEL DETAILS
xfail[:] test LABEL
xfail[:] test LABEL DETAILS
uxsuccess[:] test LABEL
uxsuccess[:] test LABEL DETAILS
progress: [+|-]X
progress: push
progress: pop
tags: [-]TAG ...
time: YYYY-MM-DD HH:MM:SSZ
LABEL: UTF8*
NAME: UTF8*
DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' LF UTF8-lines ']' LF
MULTIPART ::= '[ multipart' LF PART* ']' LF
PART ::= PART_TYPE LF NAME LF PART_BYTES LF
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF
LF ::= '\n'
CR ::= '\r'
```
unexpected output on stdout -> stdout.
exit w/0 or last test completing -> error
Tags given outside a test are applied to all following tests
Tags given after a test: line and before the result line for the same test
apply only to that test, and inherit the current global tags.
A '-' before a tag is used to remove tags - e.g. to prevent a global tag
applying to a single test, or to cancel a global tag.
The progress directive is used to provide progress information about a stream
so that stream consumer can provide completion estimates, progress bars and so
on. Stream generators that know how many tests will be present in the stream
should output "progress: COUNT". Stream filters that add tests should output
"progress: +COUNT", and those that remove tests should output
"progress: -COUNT". An absolute count should reset the progress indicators in
use - it indicates that two separate streams from different generators have
been trivially concatenated together, and there is no knowledge of how many
more complete streams are incoming. Smart concatenation could scan each stream
for their count and sum them, or alternatively translate absolute counts into
relative counts inline. It is recommended that outputters avoid absolute counts
unless necessary. The push and pop directives are used to provide local regions
for progress reporting. This fits with hierarchically operating test
environments - such as those that organise tests into suites - the top-most
runner can report on the number of suites, and each suite surround its output
with a (push, pop) pair. Interpreters should interpret a pop as also advancing
the progress of the restored level by one step. Encountering progress
directives between the start and end of a test pair indicates that a previous
test was interrupted and did not cleanly terminate: it should be implicitly
closed with an error (the same as when a stream ends with no closing test
directive for the most recently started test).
The time directive acts as a clock event - it sets the time for all future
events. The value should be a valid ISO8601 time.
The skip, xfail and uxsuccess outcomes are not supported by all testing
environments. In Python the testttools (https://launchpad.net/testtools)
library is used to translate these automatically if an older Python version
that does not support them is in use. See the testtools documentation for the
translation policy.
skip is used to indicate a test was discovered but not executed. xfail is used
to indicate a test that errored in some expected fashion (also know as "TODO"
tests in some frameworks). uxsuccess is used to indicate and unexpected success
where a test though to be failing actually passes. It is complementary to
xfail.
## Hacking on subunit
### Releases
* Update versions in configure.ac and python/subunit/__init__.py.
* Update NEWS.
* Do a make distcheck, which will update Makefile etc.
* Do a PyPI release: PYTHONPATH=python python setup.py sdist bdist_wheel; twine upload -s dist/*
* Upload the regular one to LP.
* Push a tagged commit.
git push -t origin master:master
+10
python/python_subunit.egg-info/requires.txt
iso8601
testtools>=2.7
[docs]
docutils
[test]
fixtures
hypothesis
testscenarios
+65
python/python_subunit.egg-info/SOURCES.txt
.codespellrc
.git-blame-ignore-revs
Apache-2.0
BSD
COPYING
MANIFEST.in
NEWS
README.md
all_tests.py
disperse.toml
pyproject.toml
setup.py
.github/dependabot.yaml
.github/workflows/main.yml
python/python_subunit.egg-info/PKG-INFO
python/python_subunit.egg-info/SOURCES.txt
python/python_subunit.egg-info/dependency_links.txt
python/python_subunit.egg-info/entry_points.txt
python/python_subunit.egg-info/requires.txt
python/python_subunit.egg-info/top_level.txt
python/subunit/__init__.py
python/subunit/_output.py
python/subunit/_to_disk.py
python/subunit/chunked.py
python/subunit/details.py
python/subunit/filters.py
python/subunit/progress_model.py
python/subunit/run.py
python/subunit/test_results.py
python/subunit/v2.py
python/subunit/filter_scripts/__init__.py
python/subunit/filter_scripts/subunit2csv.py
python/subunit/filter_scripts/subunit2disk.py
python/subunit/filter_scripts/subunit2gtk.py
python/subunit/filter_scripts/subunit2junitxml.py
python/subunit/filter_scripts/subunit2pyunit.py
python/subunit/filter_scripts/subunit_1to2.py
python/subunit/filter_scripts/subunit_2to1.py
python/subunit/filter_scripts/subunit_filter.py
python/subunit/filter_scripts/subunit_ls.py
python/subunit/filter_scripts/subunit_notify.py
python/subunit/filter_scripts/subunit_output.py
python/subunit/filter_scripts/subunit_stats.py
python/subunit/filter_scripts/subunit_tags.py
python/subunit/filter_scripts/tap2subunit.py
python/subunit/tests/__init__.py
python/subunit/tests/sample-script.py
python/subunit/tests/sample-two-script.py
python/subunit/tests/test_chunked.py
python/subunit/tests/test_details.py
python/subunit/tests/test_filter_to_disk.py
python/subunit/tests/test_filters.py
python/subunit/tests/test_output_filter.py
python/subunit/tests/test_progress_model.py
python/subunit/tests/test_run.py
python/subunit/tests/test_subunit_filter.py
python/subunit/tests/test_subunit_stats.py
python/subunit/tests/test_subunit_tags.py
python/subunit/tests/test_tap2subunit.py
python/subunit/tests/test_test_protocol.py
python/subunit/tests/test_test_protocol2.py
python/subunit/tests/test_test_results.py
vim/README.md
vim/ftdetect/subunit.vim
vim/syntax/subunit.vim
+1
python/python_subunit.egg-info/top_level.txt
subunit
+482
README.md
subunit: A streaming protocol for test results
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
See the COPYING file for full details on the licensing of Subunit.
# Subunit
Subunit is a streaming protocol for test results.
There are two major revisions of the protocol. Version 1 was trivially human
readable but had significant defects as far as highly parallel testing was
concerned - it had no room for doing discovery and execution in parallel,
required substantial buffering when multiplexing and was fragile - a corrupt
byte could cause an entire stream to be misparsed. Version 1.1 added
encapsulation of binary streams which mitigated some of the issues but the
core remained.
Version 2 shares many of the good characteristics of Version 1 - it can be
embedded into a regular text stream (e.g. from a build system) and it still
models xUnit style test execution. It also fixes many of the issues with
Version 1 - Version 2 can be multiplexed without excessive buffering (in
time or space), it has a well defined recovery mechanism for dealing with
corrupted streams (e.g. where two processes write to the same stream
concurrently, or where the stream generator suffers a bug).
More details on both protocol version s can be found in the 'Protocol' section
of this document.
Subunit comes with command line filters to process a subunit stream and
language bindings for python, C, C++ and shell. Bindings are easy to write
for other languages.
A number of useful things can be done easily with subunit:
* Test aggregation: Tests run separately can be combined and then
reported/displayed together. For instance, tests from different languages
can be shown as a seamless whole, and tests running on multiple machines
can be aggregated into a single stream through a multiplexer.
* Test archiving: A test run may be recorded and replayed later.
* Test isolation: Tests that may crash or otherwise interact badly with each
other can be run separately and then aggregated, rather than interfering
with each other or requiring an adhoc test->runner reporting protocol.
* Grid testing: subunit can act as the necessary serialisation and
deserialiation to get test runs on distributed machines to be reported in
real time.
Subunit supplies the following filters:
* tap2subunit - convert perl's TestAnythingProtocol to subunit.
* subunit2csv - convert a subunit stream to csv.
* subunit2disk - export a subunit stream to files on disk.
* subunit2pyunit - convert a subunit stream to pyunit test results.
* subunit2gtk - show a subunit stream in GTK.
* subunit2junitxml - convert a subunit stream to JUnit's XML format.
* subunit-diff - compare two subunit streams.
* subunit-filter - filter out tests from a subunit stream.
* subunit-ls - list info about tests present in a subunit stream.
* subunit-stats - generate a summary of a subunit stream.
* subunit-tags - add or remove tags from a stream.
## Integration with other tools
Subunit's language bindings act as integration with various test runners like
'check', 'cppunit', Python's 'unittest'. Beyond that a small amount of glue
(typically a few lines) will allow Subunit to be used in more sophisticated
ways.
### Python
Subunit has excellent Python support: most of the filters and tools are written
in python and there are facilities for using Subunit to increase test isolation
seamlessly within a test suite.
The most common way is to run an existing python test suite and have it output
subunit via the ``subunit.run`` module::
$ python -m subunit.run mypackage.tests.test_suite
For more information on the Python support Subunit offers , please see
``pydoc subunit``, or the source in ``python/subunit/``
### Rust
There is a rust crate [subunit](https://crates.io/crates/subunit) that can be used to parse v1 and v1, and emit v2 subunit streams.
### C
Subunit has C bindings to emit the protocol. The 'check' C unit testing project
has included subunit support in their project for some years now. See
'c/README' for more details.
### C++
The C library is includable and usable directly from C++. A TestListener for
CPPUnit is included in the Subunit distribution. See 'c++/README' for details.
### shell
There are two sets of shell tools. There are filters, which accept a subunit
stream on stdin and output processed data (or a transformed stream) on stdout.
Then there are unittest facilities similar to those for C : shell bindings
consisting of simple functions to output protocol elements, and a patch for
adding subunit output to the 'ShUnit' shell test runner. See 'shell/README' for
details.
#### Filter recipes
To ignore some failing tests whose root cause is already known::
subunit-filter --without 'AttributeError.*flavor'
## The xUnit test model
Subunit implements a slightly modified xUnit test model. The stock standard
model is that there are tests, which have an id(), can be run, and when run
start, emit an outcome (like success or failure) and then finish.
Subunit extends this with the idea of test enumeration (find out about tests
a runner has without running them), tags (allow users to describe tests in
ways the test framework doesn't apply any semantic value to), file attachments
(allow arbitrary data to make analysing a failure easy) and timestamps.
## The protocol
Version 2, or v2 is new and still under development, but is intended to
supersede version 1 in the very near future. Subunit's bundled tools accept
only version 2 and only emit version 2, but the new filters subunit-1to2 and
subunit-2to1 can be used to interoperate with older third party libraries.
### Version 2
Version 2 is a binary protocol consisting of independent packets that can be
embedded in the output from tools like make - as long as each packet has no
other bytes mixed in with it (which 'make -j N>1' has a tendency of doing).
Version 2 is currently in draft form, and early adopters should be willing
to either discard stored results (if protocol changes are made), or bulk
convert them back to v1 and then to a newer edition of v2.
The protocol synchronises at the start of the stream, after a packet, or
after any 0x0A byte. That is, a subunit v2 packet starts after a newline or
directly after the end of the prior packet.
Subunit is intended to be transported over a reliable streaming protocol such
as TCP. As such it does not concern itself with out of order delivery of
packets. However, because of the possibility of corruption due to either
bugs in the sender, or due to mixed up data from concurrent writes to the same
fd when being embedded, subunit strives to recover reasonably gracefully from
damaged data.
A key design goal for Subunit version 2 is to allow processing and multiplexing
without forcing buffering for semantic correctness, as buffering tends to hide
hung or otherwise misbehaving tests. That said, limited time based buffering
for network efficiency is a good idea - this is ultimately implementer
choice. Line buffering is also discouraged for subunit streams, as dropping
into a debugger or other tool may require interactive traffic even if line
buffering would not otherwise be a problem.
In version two there are two conceptual events - a test status event and a file
attachment event. Events may have timestamps, and the path of multiplexers that
an event is routed through is recorded to permit sending actions back to the
source (such as new tests to run or stdin for driving debuggers and other
interactive input). Test status events are used to enumerate tests, to report
tests and test helpers as they run. Tests may have tags, used to allow
tunnelling extra meanings through subunit without requiring parsing of
arbitrary file attachments. Things that are not standalone tests get marked
as such by setting the 'Runnable' flag to false. (For instance, individual
assertions in TAP are not runnable tests, only the top level TAP test script
is runnable).
File attachments are used to provide rich detail about the nature of a failure.
File attachments can also be used to encapsulate stdout and stderr both during
and outside tests.
Most numbers are stored in network byte order - Most Significant Byte first
encoded using a variation of http://www.dlugosz.com/ZIP2/VLI.html. The first
byte's top 2 high order bits encode the total number of octets in the number.
This encoding can encode values from 0 to 2**30-1, enough to encode a
nanosecond. Numbers that are not variable length encoded are still stored in
MSB order.
+--------+--------+---------+------------+
| prefix | octets | max | max |
+========+========+=========+============+
| 00 | 1 | 2**6-1 | 63 |
+--------+--------+---------+------------+
| 01 | 2 | 2**14-1 | 16383 |
+--------+--------+---------+------------+
| 10 | 3 | 2**22-1 | 4194303 |
+--------+--------+---------+------------+
| 11 | 4 | 2**30-1 | 1073741823 |
+--------+--------+---------+------------+
All variable length elements of the packet are stored with a length prefix
number allowing them to be skipped over for consumers that don't need to
interpret them.
UTF-8 strings are with no terminating NUL and should not have any embedded NULs
(implementations SHOULD validate any such strings that they process and take
some remedial action (such as discarding the packet as corrupt).
In short the structure of a packet is:
PACKET := SIGNATURE FLAGS PACKET_LENGTH TIMESTAMP? TESTID? TAGS? MIME?
FILECONTENT? ROUTING_CODE? CRC32
In more detail...
Packets are identified by a single byte signature - 0xB3, which is never legal
in a UTF-8 stream as the first byte of a character. 0xB3 starts with the first
bit set and the second not, which is the UTF-8 signature for a continuation
byte. 0xB3 was chosen as 0x73 ('s' in ASCII') with the top two bits replaced by
the 1 and 0 for a continuation byte.
If subunit packets are being embedded in a non-UTF-8 text stream, where 0x73 is
a legal character, consider either recoding the text to UTF-8, or using
subunit's 'file' packets to embed the text stream in subunit, rather than the
other way around.
Following the signature byte comes a 16-bit flags field, which includes a
4-bit version field - if the version is not 0x2 then the packet cannot be
read. It is recommended to signal an error at this point (e.g. by emitting
a synthetic error packet and returning to the top level loop to look for
new packets, or exiting with an error). If recovery is desired, treat the
packet signature as an opaque byte and scan for a new synchronisation point.
NB: Subunit V1 and V2 packets may legitimately included 0xB3 internally,
as they are an 8-bit safe container format, so recovery from this situation
may involve an arbitrary number of false positives until an actual packet
is encountered : and even then it may still be false, failing after passing
the version check due to coincidence.
Flags are stored in network byte order too.
```asciiart
+------------+------------+------------------------+
| High byte | Low byte |
+------------+------------+------------------------+
| 15 14 13 12 11 10 9 8 | 7 6 5 4 3 2 1 0 |
+------------+------------+------------------------+
| VERSION | feature bits |
+------------+-------------------------------------+
```
Valid version values are:
`0x2` - version 2
Feature bits:
| Bit position | Mask | Description|
|---------|-------------|---------------------------|
| Bit 11 | `0x0800` | Test id present. |
| Bit 10 | `0x0400` | Routing code present. |
| Bit 9 | `0x0200` | Timestamp present. |
| Bit 8 | `0x0100` | Test is 'runnable'. |
| Bit 7 | `0x0080` | Tags are present. |
| Bit 6 | `0x0040` | File content is present. |
| Bit 5 | `0x0020` | File MIME type is present.|
| Bit 4 | `0x0010` | EOF marker. |
| Bit 3 | `0x0008` | Must be zero in version 2.|
Test status gets three bits:
Bit 2 | Bit 1 | Bit 0 - mask 0x0007 - A test status enum lookup:
* 000 - undefined / no test
* 001 - Enumeration / existence
* 002 - In progress
* 003 - Success
* 004 - Unexpected Success
* 005 - Skipped
* 006 - Failed
* 007 - Expected failure
After the flags field is a number field giving the length in bytes for the
entire packet including the signature and the checksum. This length must
be less than 4MiB - 4194303 bytes. The encoding can obviously record a larger
number but one of the goals is to avoid requiring large buffers, or causing
large latency in the packet forward/processing pipeline. Larger file
attachments can be communicated in multiple packets, and the overhead in such a
4MiB packet is approximately 0.2%.
The rest of the packet is a series of optional features as specified by the set
feature bits in the flags field. When absent they are entirely absent.
Forwarding and multiplexing of packets can be done without interpreting the
remainder of the packet until the routing code and checksum (which are both at
the end of the packet). Additionally, routers can often avoid copying or moving
the bulk of the packet, as long as the routing code size increase doesn't force
the length encoding to take up a new byte (which will only happen to packets
less than or equal to 16KiB in length) - large packets are very efficient to
route.
Timestamp when present is a 32 bit unsigned integer for seconds, and a variable
length number for nanoseconds, representing UTC time since Unix Epoch in
seconds and nanoseconds.
Test id when present is a UTF-8 string. The test id should uniquely identify
runnable tests such that they can be selected individually. For tests and other
actions which cannot be individually run (such as test
fixtures/layers/subtests) uniqueness is not required (though being human
meaningful is highly recommended).
Tags when present is a length prefixed vector of UTF-8 strings, one per tag.
There are no restrictions on tag content (other than the restrictions on UTF-8
strings in subunit in general). Tags have no ordering.
When a MIME type is present, it defines the MIME type for the file across all
packets same file (routing code + testid + name uniquely identifies a file,
reset when EOF is flagged). If a file never has a MIME type set, it should be
treated as application/octet-stream.
File content when present is a UTF-8 string for the name followed by the length
in bytes of the content, and then the content octets.
If present routing code is a UTF-8 string. The routing code is used to
determine which test backend a test was running on when doing data analysis,
and to route stdin to the test process if interaction is required.
Multiplexers SHOULD add a routing code if none is present, and prefix any
existing routing code with a routing code ('/' separated) if one is already
present. For example, a multiplexer might label each stream it is multiplexing
with a simple ordinal ('0', '1' etc), and given an incoming packet with route
code '3' from stream '0' would adjust the route code when forwarding the packet
to be '0/3'.
Following the end of the packet is a CRC-32 checksum of the contents of the
packet including the signature.
#### Example packets
Trivial test "foo" enumeration packet, with test id, runnable set,
status=enumeration. Spaces below are to visually break up signature / flags /
length / testid / crc32
b3 2901 0c 03666f6f 08555f1b
### Version 1 (and 1.1)
Version 1 (and 1.1) are mostly human readable protocols.
#### Sample subunit wire contents
The following:
```subunit
test: test foo works
success: test foo works
test: tar a file.
failure: tar a file. [
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
]
a writeln to stdout
When run through subunit2pyunit::
.F
a writeln to stdout
========================
FAILURE: tar a file.
-------------------
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
```
#### Subunit v1 protocol description
This description is being ported to an EBNF style. Currently its only partly in
that style, but should be fairly clear all the same. When in doubt, refer the
source (and ideally help fix up the description!). Generally the protocol is
line orientated and consists of either directives and their parameters, or
when outside a DETAILS region unexpected lines which are not interpreted by
the parser - they should be forwarded unaltered:
```ebnf
test|testing|test:|testing: test LABEL
success|success:|successful|successful: test LABEL
success|success:|successful|successful: test LABEL DETAILS
failure: test LABEL
failure: test LABEL DETAILS
error: test LABEL
error: test LABEL DETAILS
skip[:] test LABEL
skip[:] test LABEL DETAILS
xfail[:] test LABEL
xfail[:] test LABEL DETAILS
uxsuccess[:] test LABEL
uxsuccess[:] test LABEL DETAILS
progress: [+|-]X
progress: push
progress: pop
tags: [-]TAG ...
time: YYYY-MM-DD HH:MM:SSZ
LABEL: UTF8*
NAME: UTF8*
DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' LF UTF8-lines ']' LF
MULTIPART ::= '[ multipart' LF PART* ']' LF
PART ::= PART_TYPE LF NAME LF PART_BYTES LF
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF
LF ::= '\n'
CR ::= '\r'
```
unexpected output on stdout -> stdout.
exit w/0 or last test completing -> error
Tags given outside a test are applied to all following tests
Tags given after a test: line and before the result line for the same test
apply only to that test, and inherit the current global tags.
A '-' before a tag is used to remove tags - e.g. to prevent a global tag
applying to a single test, or to cancel a global tag.
The progress directive is used to provide progress information about a stream
so that stream consumer can provide completion estimates, progress bars and so
on. Stream generators that know how many tests will be present in the stream
should output "progress: COUNT". Stream filters that add tests should output
"progress: +COUNT", and those that remove tests should output
"progress: -COUNT". An absolute count should reset the progress indicators in
use - it indicates that two separate streams from different generators have
been trivially concatenated together, and there is no knowledge of how many
more complete streams are incoming. Smart concatenation could scan each stream
for their count and sum them, or alternatively translate absolute counts into
relative counts inline. It is recommended that outputters avoid absolute counts
unless necessary. The push and pop directives are used to provide local regions
for progress reporting. This fits with hierarchically operating test
environments - such as those that organise tests into suites - the top-most
runner can report on the number of suites, and each suite surround its output
with a (push, pop) pair. Interpreters should interpret a pop as also advancing
the progress of the restored level by one step. Encountering progress
directives between the start and end of a test pair indicates that a previous
test was interrupted and did not cleanly terminate: it should be implicitly
closed with an error (the same as when a stream ends with no closing test
directive for the most recently started test).
The time directive acts as a clock event - it sets the time for all future
events. The value should be a valid ISO8601 time.
The skip, xfail and uxsuccess outcomes are not supported by all testing
environments. In Python the testttools (https://launchpad.net/testtools)
library is used to translate these automatically if an older Python version
that does not support them is in use. See the testtools documentation for the
translation policy.
skip is used to indicate a test was discovered but not executed. xfail is used
to indicate a test that errored in some expected fashion (also know as "TODO"
tests in some frameworks). uxsuccess is used to indicate and unexpected success
where a test though to be failing actually passes. It is complementary to
xfail.
## Hacking on subunit
### Releases
* Update versions in configure.ac and python/subunit/__init__.py.
* Update NEWS.
* Do a make distcheck, which will update Makefile etc.
* Do a PyPI release: PYTHONPATH=python python setup.py sdist bdist_wheel; twine upload -s dist/*
* Upload the regular one to LP.
* Push a tagged commit.
git push -t origin master:master
vim/ftdetect/subunit.vim

Sorry, the diff of this file is not supported yet

+31
vim/README.md
# Subunit Vim Syntax Highlighting
This directory contains Vim syntax highlighting for Subunit v1 test protocol streams.
## Installation
### Option 1: Manual Installation
Copy the files to your Vim configuration directory:
```bash
cp -r syntax ~/.vim/
cp -r ftdetect ~/.vim/
```
### Option 2: Using a Plugin Manager
If you use a Vim plugin manager like vim-plug, add to your `.vimrc`:
```vim
Plug 'path/to/subunit/vim'
```
### Option 3: Pathogen
If you use Pathogen:
```bash
cd ~/.vim/bundle
ln -s /path/to/subunit/vim subunit
```
vim/syntax/subunit.vim

Sorry, the diff of this file is not supported yet

+20
-2
NEWS

@@ -5,5 +5,21 @@ ---------------------

NEXT (In development)
1.4.5 (2025-11-10)
---------------------
BUG FIXES
~~~~~~~~~
* Drop support for Python 3.9.
(Jelmer Vernooij)
* Drop use of Python26 and Python27 test cases from testtools.
These are no longer relevant and no longer provided by
modern versions of testtools. (Jelmer Vernooij)
IMPROVEMENTS
~~~~~~~~~~~~
* Add support for Python 3.14.
(Jelmer Vernooij)
1.4.4 (2023-11-17)

@@ -27,2 +43,4 @@ ------------------

* Drop support for Python 3.7 (Jelmer Vernooij)
1.4.3 (2023-09-17)

@@ -338,3 +356,3 @@ ---------------------

With the ability to encapsulate multiple non-test streams, another significant
cange is that filters which emit subunit now encapsulate any non-subunit they
change is that filters which emit subunit now encapsulate any non-subunit they
encounter, labelling it 'stdout'. This permits multiplexing such streams and

@@ -341,0 +359,0 @@ detangling the stdout streams from each input.

+71
-74
PKG-INFO

@@ -1,27 +0,32 @@

Metadata-Version: 2.1
Metadata-Version: 2.4
Name: python-subunit
Version: 1.4.4
Version: 1.4.5
Summary: Python implementation of subunit test streaming protocol
Home-page: http://launchpad.net/subunit
Author: Robert Collins
Author-email: subunit-dev@lists.launchpad.net
Author-email: Robert Collins <subunit-dev@lists.launchpad.net>
License: Apache-2.0 or BSD
Project-URL: Bug Tracker, https://bugs.launchpad.net/subunit
Project-URL: Homepage, http://launchpad.net/subunit
Project-URL: Source Code, https://github.com/testing-cabal/subunit/
Keywords: python test streaming
Keywords: python,streaming,test
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.7
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: COPYING
Requires-Dist: iso8601
Requires-Dist: testtools>=2.7
Provides-Extra: docs
Requires-Dist: docutils; extra == "docs"
Provides-Extra: test
License-File: COPYING
Requires-Dist: fixtures; extra == "test"
Requires-Dist: hypothesis; extra == "test"
Requires-Dist: testscenarios; extra == "test"
Dynamic: license-file

@@ -45,4 +50,3 @@

Subunit
-------
# Subunit

@@ -81,3 +85,3 @@ Subunit is a streaming protocol for test results.

* Test isolation: Tests that may crash or otherwise interact badly with each
other can be run seperately and then aggregated, rather than interfering
other can be run separately and then aggregated, rather than interfering
with each other or requiring an adhoc test->runner reporting protocol.

@@ -101,5 +105,5 @@ * Grid testing: subunit can act as the necessary serialisation and

Integration with other tools
----------------------------
## Integration with other tools
Subunit's language bindings act as integration with various test runners like

@@ -110,4 +114,3 @@ 'check', 'cppunit', Python's 'unittest'. Beyond that a small amount of glue

Python
======
### Python

@@ -126,5 +129,9 @@ Subunit has excellent Python support: most of the filters and tools are written

C
=
### Rust
There is a rust crate [subunit](https://crates.io/crates/subunit) that can be used to parse v1 and v1, and emit v2 subunit streams.
### C
Subunit has C bindings to emit the protocol. The 'check' C unit testing project

@@ -134,10 +141,9 @@ has included subunit support in their project for some years now. See

C++
===
### C++
The C library is includable and usable directly from C++. A TestListener for
CPPUnit is included in the Subunit distribution. See 'c++/README' for details.
shell
=====
### shell

@@ -152,4 +158,3 @@ There are two sets of shell tools. There are filters, which accept a subunit

Filter recipes
--------------
#### Filter recipes

@@ -160,5 +165,4 @@ To ignore some failing tests whose root cause is already known::

## The xUnit test model
The xUnit test model
--------------------

@@ -174,12 +178,11 @@ Subunit implements a slightly modified xUnit test model. The stock standard

The protocol
------------
## The protocol
Version 2, or v2 is new and still under development, but is intended to
supercede version 1 in the very near future. Subunit's bundled tools accept
supersede version 1 in the very near future. Subunit's bundled tools accept
only version 2 and only emit version 2, but the new filters subunit-1to2 and
subunit-2to1 can be used to interoperate with older third party libraries.
Version 2
=========
### Version 2

@@ -207,3 +210,3 @@ Version 2 is a binary protocol consisting of independent packets that can be

hung or otherwise misbehaving tests. That said, limited time based buffering
for network efficiency is a good idea - this is ultimately implementator
for network efficiency is a good idea - this is ultimately implementer
choice. Line buffering is also discouraged for subunit streams, as dropping

@@ -288,2 +291,3 @@ into a debugger or other tool may require interactive traffic even if line

```asciiart
+------------+------------+------------------------+

@@ -296,27 +300,20 @@ | High byte | Low byte |

+------------+-------------------------------------+
```
Valid version values are:
0x2 - version 2
`0x2` - version 2
Feature bits:
+---------+-------------+---------------------------+
| Bit 11 | mask 0x0800 | Test id present. |
+---------+-------------+---------------------------+
| Bit 10 | mask 0x0400 | Routing code present. |
+---------+-------------+---------------------------+
| Bit 9 | mask 0x0200 | Timestamp present. |
+---------+-------------+---------------------------+
| Bit 8 | mask 0x0100 | Test is 'runnable'. |
+---------+-------------+---------------------------+
| Bit 7 | mask 0x0080 | Tags are present. |
+---------+-------------+---------------------------+
| Bit 6 | mask 0x0040 | File content is present. |
+---------+-------------+---------------------------+
| Bit 5 | mask 0x0020 | File MIME type is present.|
+---------+-------------+---------------------------+
| Bit 4 | mask 0x0010 | EOF marker. |
+---------+-------------+---------------------------+
| Bit 3 | mask 0x0008 | Must be zero in version 2.|
+---------+-------------+---------------------------+
| Bit position | Mask | Description|
|---------|-------------|---------------------------|
| Bit 11 | `0x0800` | Test id present. |
| Bit 10 | `0x0400` | Routing code present. |
| Bit 9 | `0x0200` | Timestamp present. |
| Bit 8 | `0x0100` | Test is 'runnable'. |
| Bit 7 | `0x0080` | Tags are present. |
| Bit 6 | `0x0040` | File content is present. |
| Bit 5 | `0x0020` | File MIME type is present.|
| Bit 4 | `0x0010` | EOF marker. |
| Bit 3 | `0x0008` | Must be zero in version 2.|

@@ -390,4 +387,3 @@ Test status gets three bits:

Example packets
~~~~~~~~~~~~~~~
#### Example packets

@@ -401,12 +397,12 @@ Trivial test "foo" enumeration packet, with test id, runnable set,

Version 1 (and 1.1)
===================
### Version 1 (and 1.1)
Version 1 (and 1.1) are mostly human readable protocols.
Sample subunit wire contents
----------------------------
#### Sample subunit wire contents
The following::
The following:
```subunit
test: test foo works

@@ -433,7 +429,6 @@ success: test foo works

foo.c:34 WARNING foo is not defined.
```
#### Subunit v1 protocol description
Subunit v1 protocol description
===============================
This description is being ported to an EBNF style. Currently its only partly in

@@ -444,4 +439,5 @@ that style, but should be fairly clear all the same. When in doubt, refer the

when outside a DETAILS region unexpected lines which are not interpreted by
the parser - they should be forwarded unaltered::
the parser - they should be forwarded unaltered:
```ebnf
test|testing|test:|testing: test LABEL

@@ -469,7 +465,10 @@ success|success:|successful|successful: test LABEL

DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' CR UTF8-lines ']' CR
MULTIPART ::= '[ multipart' CR PART* ']' CR
PART ::= PART_TYPE CR NAME CR PART_BYTES CR
BRACKETED ::= '[' LF UTF8-lines ']' LF
MULTIPART ::= '[ multipart' LF PART* ']' LF
PART ::= PART_TYPE LF NAME LF PART_BYTES LF
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF
LF ::= '\n'
CR ::= '\r'
```

@@ -522,7 +521,5 @@ unexpected output on stdout -> stdout.

Hacking on subunit
------------------
## Hacking on subunit
Releases
========
### Releases

@@ -529,0 +526,0 @@ * Update versions in configure.ac and python/subunit/__init__.py.

+60
-2
pyproject.toml
[build-system]
requires = ["setuptools>=43.0.0"]
build-backend = "setuptools.build_meta"
requires = ["iso8601", "setuptools>=61.2", "testtools"]
[tool.ruff]
line-length = 120
target-version = "py37"
[project]
authors = [
{ name = "Robert Collins", email = "subunit-dev@lists.launchpad.net" },
]
classifiers = [
"Intended Audience :: Developers",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python",
"Topic :: Software Development :: Testing",
]
dependencies = ["iso8601", "testtools>=2.7"]
description = "Python implementation of subunit test streaming protocol"
dynamic = ["version"]
keywords = ["python", "streaming", "test"]
license = { text = "Apache-2.0 or BSD" }
name = "python-subunit"
readme = "README.md"
requires-python = ">=3.10"
[project.urls]
"Bug Tracker" = "https://bugs.launchpad.net/subunit"
Homepage = "http://launchpad.net/subunit"
"Source Code" = "https://github.com/testing-cabal/subunit/"
[project.optional-dependencies]
"docs" = ["docutils"]
"test" = ["fixtures", "hypothesis", "testscenarios"]
[tool.setuptools.packages.find]
include = ["subunit*"]
where = ["python"]
[project.scripts]
"subunit-1to2" = "subunit.filter_scripts.subunit_1to2:main"
"subunit-2to1" = "subunit.filter_scripts.subunit_2to1:main"
"subunit-filter" = "subunit.filter_scripts.subunit_filter:main"
"subunit-ls" = "subunit.filter_scripts.subunit_ls:main"
"subunit-notify" = "subunit.filter_scripts.subunit_notify:main"
"subunit-output" = "subunit.filter_scripts.subunit_output:main"
"subunit-stats" = "subunit.filter_scripts.subunit_stats:main"
"subunit-tags" = "subunit.filter_scripts.subunit_tags:main"
"subunit2csv" = "subunit.filter_scripts.subunit2csv:main"
"subunit2disk" = "subunit.filter_scripts.subunit2disk:main"
"subunit2gtk" = "subunit.filter_scripts.subunit2gtk:main"
"subunit2junitxml" = "subunit.filter_scripts.subunit2junitxml:main"
"subunit2pyunit" = "subunit.filter_scripts.subunit2pyunit:main"
"tap2subunit" = "subunit.filter_scripts.tap2subunit:main"
[tool.setuptools.dynamic]
version = { attr = "subunit.version_string" }
[tool.setuptools]
license-files = ["COPYING"]
+192
-186
python/subunit/__init__.py

@@ -52,7 +52,4 @@ #

When used the value of details should be a dict from ``string`` to
``testtools.content.Content`` objects. This is a draft API being worked on with
the Python Testing In Python mail list, with the goal of permitting a common
way to provide additional data beyond a traceback, such as captured data from
disk, logging messages etc. The reference for this API is in testtools (0.9.0
and newer).
``testtools.content.Content`` objects. This API permits providing additional
data beyond a traceback, such as captured data from disk, logging messages etc.

@@ -128,12 +125,8 @@ The ``tags(new_tags, gone_tags)`` method is called (if present) to add or

import iso8601
from testtools import ExtendedToOriginalDecorator, content, content_type
from testtools.compat import _b, _u
from testtools.content import TracebackContent
try:
from testtools.testresult.real import _StringException
RemoteException = _StringException
except ImportError:
raise ImportError ("testtools.testresult.real does not contain "
"_StringException, check your version.")
from testtools.testresult.real import _StringException as RemoteException
from testtools import CopyStreamResult, testresult

@@ -156,22 +149,25 @@

__version__ = (1, 4, 4, 'final', 0)
__version__ = (1, 4, 5, "final", 0)
version_string = ".".join(map(str, __version__[:3]))
__all__ = [
'join_dir',
'tags_to_new_gone',
'content',
'content_type',
'TestProtocolServer',
'TestProtocolClient',
'RemoteError',
'RemotedTestCase',
'IsolatedTestCase',
'IsolatedTestSuite',
'run_isolated',
'TAP2SubUnit',
'tag_stream',
'ProtocolTestCase',
'make_stream_binary',
'read_test_list',
'TestResultStats',
"join_dir",
"tags_to_new_gone",
"content",
"content_type",
"TestProtocolServer",
"TestProtocolClient",
"RemoteError",
"RemotedTestCase",
"IsolatedTestCase",
"IsolatedTestSuite",
"run_isolated",
"TAP2SubUnit",
"tag_stream",
"ProtocolTestCase",
"make_stream_binary",
"read_test_list",
"TestResultStats",
]

@@ -187,2 +183,3 @@

import subunit.tests
return subunit.tests.test_suite()

@@ -210,3 +207,3 @@

for tag in tags:
if tag[0] == '-':
if tag[0] == "-":
gone_tags.add(tag[1:])

@@ -228,3 +225,3 @@ else:

def read(self, len=0):
return _b('')
return b""

@@ -237,15 +234,15 @@

self.parser = parser
self._test_sym = (_b('test'), _b('testing'))
self._colon_sym = _b(':')
self._error_sym = (_b('error'),)
self._failure_sym = (_b('failure'),)
self._progress_sym = (_b('progress'),)
self._skip_sym = _b('skip')
self._success_sym = (_b('success'), _b('successful'))
self._tags_sym = (_b('tags'),)
self._time_sym = (_b('time'),)
self._xfail_sym = (_b('xfail'),)
self._uxsuccess_sym = (_b('uxsuccess'),)
self._start_simple = _u(" [")
self._start_multipart = _u(" [ multipart")
self._test_sym = (b"test", b"testing")
self._colon_sym = b":"
self._error_sym = (b"error",)
self._failure_sym = (b"failure",)
self._progress_sym = (b"progress",)
self._skip_sym = b"skip"
self._success_sym = (b"success", b"successful")
self._tags_sym = (b"tags",)
self._time_sym = (b"time",)
self._xfail_sym = (b"xfail",)
self._uxsuccess_sym = (b"uxsuccess",)
self._start_simple = " ["
self._start_multipart = " [ multipart"

@@ -308,3 +305,3 @@ def addError(self, offset, line):

"""Connection lost."""
self.parser._lostConnectionInTest(_u('unknown state of '))
self.parser._lostConnectionInTest("unknown state of ")

@@ -326,3 +323,3 @@ def startTest(self, offset, line):

"""
test_name = line[offset:-1].decode('utf8')
test_name = line[offset:-1].decode("utf8")
if self.parser.current_test_description == test_name:

@@ -335,9 +332,7 @@ self.parser._state = self.parser._outside_test

self.parser.subunitLineReceived(line)
elif self.parser.current_test_description + self._start_simple == \
test_name:
elif self.parser.current_test_description + self._start_simple == test_name:
self.parser._state = details_state
details_state.set_simple()
self.parser.subunitLineReceived(line)
elif self.parser.current_test_description + self._start_multipart == \
test_name:
elif self.parser.current_test_description + self._start_multipart == test_name:
self.parser._state = details_state

@@ -350,18 +345,14 @@ details_state.set_multipart()

def _error(self):
self.parser.client.addError(self.parser._current_test,
details={})
self.parser.client.addError(self.parser._current_test, details={})
def addError(self, offset, line):
"""An 'error:' directive has been read."""
self._outcome(offset, line, self._error,
self.parser._reading_error_details)
self._outcome(offset, line, self._error, self.parser._reading_error_details)
def _xfail(self):
self.parser.client.addExpectedFailure(self.parser._current_test,
details={})
self.parser.client.addExpectedFailure(self.parser._current_test, details={})
def addExpectedFail(self, offset, line):
"""An 'xfail:' directive has been read."""
self._outcome(offset, line, self._xfail,
self.parser._reading_xfail_details)
self._outcome(offset, line, self._xfail, self.parser._reading_xfail_details)

@@ -373,4 +364,3 @@ def _uxsuccess(self):

"""A 'uxsuccess:' directive has been read."""
self._outcome(offset, line, self._uxsuccess,
self.parser._reading_uxsuccess_details)
self._outcome(offset, line, self._uxsuccess, self.parser._reading_uxsuccess_details)

@@ -382,4 +372,3 @@ def _failure(self):

"""A 'failure:' directive has been read."""
self._outcome(offset, line, self._failure,
self.parser._reading_failure_details)
self._outcome(offset, line, self._failure, self.parser._reading_failure_details)

@@ -391,4 +380,3 @@ def _skip(self):

"""A 'skip:' directive has been read."""
self._outcome(offset, line, self._skip,
self.parser._reading_skip_details)
self._outcome(offset, line, self._skip, self.parser._reading_skip_details)

@@ -400,8 +388,7 @@ def _succeed(self):

"""A 'success:' directive has been read."""
self._outcome(offset, line, self._succeed,
self.parser._reading_success_details)
self._outcome(offset, line, self._succeed, self.parser._reading_success_details)
def lostConnection(self):
"""Connection lost."""
self.parser._lostConnectionInTest(_u(''))
self.parser._lostConnectionInTest("")

@@ -418,3 +405,3 @@

self.parser._state = self.parser._in_test
test_name = line[offset:-1].decode('utf8')
test_name = line[offset:-1].decode("utf8")
self.parser._current_test = RemotedTestCase(test_name)

@@ -443,4 +430,3 @@ self.parser.current_test_description = test_name

"""Connection lost."""
self.parser._lostConnectionInTest(_u('%s report of ') %
self._outcome_label())
self.parser._lostConnectionInTest("%s report of " % self._outcome_label())

@@ -464,4 +450,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addFailure(self.parser._current_test,
details=self.details_parser.get_details())
self.parser.client.addFailure(self.parser._current_test, details=self.details_parser.get_details())

@@ -476,4 +461,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addError(self.parser._current_test,
details=self.details_parser.get_details())
self.parser.client.addError(self.parser._current_test, details=self.details_parser.get_details())

@@ -488,4 +472,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addExpectedFailure(self.parser._current_test,
details=self.details_parser.get_details())
self.parser.client.addExpectedFailure(self.parser._current_test, details=self.details_parser.get_details())

@@ -500,4 +483,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addUnexpectedSuccess(self.parser._current_test,
details=self.details_parser.get_details())
self.parser.client.addUnexpectedSuccess(self.parser._current_test, details=self.details_parser.get_details())

@@ -512,4 +494,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addSkip(self.parser._current_test,
details=self.details_parser.get_details("skip"))
self.parser.client.addSkip(self.parser._current_test, details=self.details_parser.get_details("skip"))

@@ -524,4 +505,3 @@ def _outcome_label(self):

def _report_outcome(self):
self.parser.client.addSuccess(self.parser._current_test,
details=self.details_parser.get_details("success"))
self.parser.client.addSuccess(self.parser._current_test, details=self.details_parser.get_details("success"))

@@ -568,5 +548,5 @@ def _outcome_label(self):

# Avoid casts on every call
self._plusminus = _b('+-')
self._push_sym = _b('push')
self._pop_sym = _b('pop')
self._plusminus = b"+-"
self._push_sym = b"push"
self._pop_sym = b"pop"

@@ -592,3 +572,3 @@ def _handleProgress(self, offset, line):

"""Process a tags command."""
tags = line[offset:].decode('utf8').split()
tags = line[offset:].decode("utf8").split()
new_tags, gone_tags = tags_to_new_gone(tags)

@@ -602,4 +582,3 @@ self.client.tags(new_tags, gone_tags)

except TypeError:
raise TypeError(_u("Failed to parse %r, got %r")
% (line, sys.exc_info()[1]))
raise TypeError("Failed to parse %r, got %r" % (line, sys.exc_info()[1]))
self.client.time(event_time)

@@ -612,4 +591,3 @@

def _lostConnectionInTest(self, state_string):
error_string = _u("lost connection during %stest '%s'") % (
state_string, self.current_test_description)
error_string = "lost connection during %stest '%s'" % (state_string, self.current_test_description)
self.client.addError(self._current_test, RemoteError(error_string))

@@ -666,10 +644,10 @@ self.client.stopTest(self._current_test)

self._stream = stream
self._progress_fmt = _b("progress: ")
self._bytes_eol = _b("\n")
self._progress_plus = _b("+")
self._progress_push = _b("push")
self._progress_pop = _b("pop")
self._empty_bytes = _b("")
self._start_simple = _b(" [\n")
self._end_simple = _b("]\n")
self._progress_fmt = b"progress: "
self._bytes_eol = b"\n"
self._progress_plus = b"+"
self._progress_push = b"push"
self._progress_pop = b"pop"
self._empty_bytes = b""
self._start_simple = b" [\n"
self._end_simple = b"]\n"

@@ -725,4 +703,3 @@ def addError(self, test, error=None, details=None):

def _addOutcome(self, outcome, test, error=None, details=None,
error_permitted=True):
def _addOutcome(self, outcome, test, error=None, details=None, error_permitted=True):
"""Report a failure in test test.

@@ -743,4 +720,4 @@

details must be supplied. If False then error must not be supplied
and details is still optional. """
self._stream.write(_b("%s: " % outcome) + self._test_id(test))
and details is still optional."""
self._stream.write(("%s: " % outcome).encode() + self._test_id(test))
if error_permitted:

@@ -760,3 +737,3 @@ if error is None and details is None:

else:
self._stream.write(_b("\n"))
self._stream.write(b"\n")
if details is not None or error is not None:

@@ -770,4 +747,4 @@ self._stream.write(self._end_simple)

else:
self._stream.write(_b("skip: %s [\n" % test.id()))
self._stream.write(_b("%s\n" % reason))
self._stream.write(("skip: %s [\n" % test.id()).encode())
self._stream.write(("%s\n" % reason).encode())
self._stream.write(self._end_simple)

@@ -790,4 +767,3 @@

"""
self._addOutcome("uxsuccess", test, details=details,
error_permitted=False)
self._addOutcome("uxsuccess", test, details=details, error_permitted=False)
if self.failfast:

@@ -798,4 +774,4 @@ self.stop()

result = test.id()
if type(result) is not bytes:
result = result.encode('utf8')
if not isinstance(result, bytes):
result = result.encode("utf8")
return result

@@ -806,3 +782,3 @@

super(TestProtocolClient, self).startTest(test)
self._stream.write(_b("test: ") + self._test_id(test) + _b("\n"))
self._stream.write(b"test: " + self._test_id(test) + b"\n")
self._stream.flush()

@@ -826,3 +802,3 @@

prefix = self._progress_plus
offset = _b(str(offset))
offset = str(offset).encode()
elif whence == PROGRESS_PUSH:

@@ -836,5 +812,4 @@ prefix = self._empty_bytes

prefix = self._empty_bytes
offset = _b(str(offset))
self._stream.write(self._progress_fmt + prefix + offset +
self._bytes_eol)
offset = str(offset).encode()
self._stream.write(self._progress_fmt + prefix + offset + self._bytes_eol)

@@ -845,5 +820,5 @@ def tags(self, new_tags, gone_tags):

return
tags = set([tag.encode('utf8') for tag in new_tags])
tags.update([_b("-") + tag.encode('utf8') for tag in gone_tags])
tag_line = _b("tags: ") + _b(" ").join(tags) + _b("\n")
tags = set([tag.encode("utf8") for tag in new_tags])
tags.update([b"-" + tag.encode("utf8") for tag in gone_tags])
tag_line = b"tags: " + b" ".join(tags) + b"\n"
self._stream.write(tag_line)

@@ -857,5 +832,6 @@

time = a_datetime.astimezone(iso8601.UTC)
self._stream.write(_b("time: %04d-%02d-%02d %02d:%02d:%02d.%06dZ\n" % (
time.year, time.month, time.day, time.hour, time.minute,
time.second, time.microsecond)))
self._stream.write(
b"time: %04d-%02d-%02d %02d:%02d:%02d.%06dZ\n"
% (time.year, time.month, time.day, time.hour, time.minute, time.second, time.microsecond)
)

@@ -867,14 +843,15 @@ def _write_details(self, details):

"""
self._stream.write(_b(" [ multipart\n"))
self._stream.write(b" [ multipart\n")
for name, content in sorted(details.items()): # noqa: F402
self._stream.write(_b("Content-Type: %s/%s" %
(content.content_type.type, content.content_type.subtype)))
self._stream.write(
("Content-Type: %s/%s" % (content.content_type.type, content.content_type.subtype)).encode()
)
parameters = content.content_type.parameters
if parameters:
self._stream.write(_b(";"))
self._stream.write(b";")
param_strs = []
for param, value in sorted(parameters.items()):
param_strs.append("%s=%s" % (param, value))
self._stream.write(_b(",".join(param_strs)))
self._stream.write(_b("\n%s\n" % name))
self._stream.write(",".join(param_strs).encode())
self._stream.write(b"\n" + name.encode() + b"\n")
encoder = chunked.Encoder(self._stream)

@@ -887,7 +864,15 @@ list(map(encoder.write, content.iter_bytes()))

def addDuration(self, test, duration):
"""Called to add a test duration.
def RemoteError(description=_u("")):
return (_StringException, _StringException(description), None)
:param test: The test that completed.
:param duration: The duration of the test as a float in seconds.
"""
pass
def RemoteError(description=""):
return (RemoteException, RemoteException(description), None)
class RemotedTestCase(unittest.TestCase):

@@ -903,3 +888,3 @@ """A class to represent test cases run in child processes.

def __eq__ (self, other):
def __eq__(self, other):
try:

@@ -911,8 +896,7 @@ return self.__description == other.__description

def __init__(self, description):
"""Create a psuedo test case with description description."""
"""Create a pseudo test case with description description."""
self.__description = description
def error(self, label):
raise NotImplementedError("%s on RemotedTestCases is not permitted." %
label)
raise NotImplementedError("%s on RemotedTestCases is not permitted." % label)

@@ -935,4 +919,3 @@ def setUp(self):

def __repr__(self):
return "<%s description='%s'>" % \
(self._strclass(), self.__description)
return "<%s description='%s'>" % (self._strclass(), self.__description)

@@ -943,3 +926,3 @@ def run(self, result=None):

result.startTest(self)
result.addError(self, RemoteError(_u("Cannot run RemotedTestCases.\n")))
result.addError(self, RemoteError("Cannot run RemotedTestCases.\n"))
result.stopTest(self)

@@ -955,11 +938,10 @@

def __init__(self, methodName='runTest'):
def __init__(self, methodName="runTest"):
"""Create an instance of the class that will use the named test
method when executed. Raises a ValueError if the instance does
not have a method with the specified name.
method when executed. Raises a ValueError if the instance does
not have a method with the specified name.
"""
unittest.TestCase.__init__(self, methodName)
testMethod = getattr(self, methodName)
self.script = join_dir(sys.modules[self.__class__.__module__].__file__,
testMethod.__doc__)
self.script = join_dir(sys.modules[self.__class__.__module__].__file__, testMethod.__doc__)

@@ -980,4 +962,3 @@ def countTestCases(self):

protocol = TestProtocolServer(result)
process = subprocess.Popen(self.script, shell=True,
stdout=subprocess.PIPE)
process = subprocess.Popen(self.script, shell=True, stdout=subprocess.PIPE)
make_stream_binary(process.stdout)

@@ -1018,4 +999,3 @@ output = process.communicate()[0]

def run_isolated(klass, self, result):
"""Run a test suite or case in a subprocess, using the run method on klass.
"""
"""Run a test suite or case in a subprocess, using the run method on klass."""
c2pread, c2pwrite = os.pipe()

@@ -1036,4 +1016,3 @@ # fixme - error -> result

# to filter it to escape ]'s.
### XXX: test and write that bit.
stream = os.fdopen(1, 'wb')
stream = os.fdopen(1, "wb")
result = TestProtocolClient(stream)

@@ -1051,6 +1030,5 @@ klass.run(self, result)

protocol = TestProtocolServer(result)
fileobj = os.fdopen(c2pread, 'rb')
fileobj = os.fdopen(c2pread, "rb")
protocol.readFrom(fileobj)
os.waitpid(pid, 0)
# TODO return code evaluation.
return result

@@ -1070,3 +1048,3 @@

output = StreamResultToBytes(output_stream)
UTF8_TEXT = 'text/plain; charset=UTF8'
UTF8_TEXT = "text/plain; charset=UTF8"
BEFORE_PLAN = 0

@@ -1082,7 +1060,14 @@ AFTER_PLAN = 1

result = None
def missing_test(plan_start):
output.status(test_id='test %d' % plan_start,
test_status='fail', runnable=False,
mime_type=UTF8_TEXT, eof=True, file_name="tap meta",
file_bytes=b"test missing from TAP output")
output.status(
test_id="test %d" % plan_start,
test_status="fail",
runnable=False,
mime_type=UTF8_TEXT,
eof=True,
file_name="tap meta",
file_bytes=b"test missing from TAP output",
)
def _emit_test():

@@ -1093,5 +1078,5 @@ "write out a test"

if log:
log_bytes = b'\n'.join(log_line.encode('utf8') for log_line in log)
log_bytes = b"\n".join(log_line.encode("utf8") for log_line in log)
mime_type = UTF8_TEXT
file_name = 'tap comment'
file_name = "tap comment"
eof = True

@@ -1104,5 +1089,12 @@ else:

del log[:]
output.status(test_id=test_name, test_status=result,
file_bytes=log_bytes, mime_type=mime_type, eof=eof,
file_name=file_name, runnable=False)
output.status(
test_id=test_name,
test_status=result,
file_bytes=log_bytes,
mime_type=mime_type,
eof=eof,
file_name=file_name,
runnable=False,
)
for line in tap:

@@ -1118,10 +1110,15 @@ if state == BEFORE_PLAN:

state = SKIP_STREAM
output.status(test_id='file skip', test_status='skip',
file_bytes=comment.encode('utf8'), eof=True,
file_name='tap comment')
output.status(
test_id="file skip",
test_status="skip",
file_bytes=comment.encode("utf8"),
eof=True,
file_name="tap comment",
)
continue
# not a plan line, or have seen one before
match = re.match(
r"(ok|not ok)(?:\s+(\d+)?)?(?:\s+([^#]*[^#\s]+)\s*)?"
r"(?:\s+#\s+(TODO|SKIP|skip|todo)(?:\s+(.*))?)?\n", line)
r"(ok|not ok)(?:\s+(\d+)?)?(?:\s+([^#]*[^#\s]+)\s*)?" r"(?:\s+#\s+(TODO|SKIP|skip|todo)(?:\s+(.*))?)?\n",
line,
)
if match:

@@ -1131,15 +1128,15 @@ # new test, emit current one.

status, number, description, directive, directive_comment = match.groups()
if status == 'ok':
result = 'success'
if status == "ok":
result = "success"
else:
result = "fail"
if description is None:
description = ''
description = ""
else:
description = ' ' + description
description = " " + description
if directive is not None:
if directive.upper() == 'TODO':
result = 'xfail'
elif directive.upper() == 'SKIP':
result = 'skip'
if directive.upper() == "TODO":
result = "xfail"
elif directive.upper() == "SKIP":
result = "skip"
if directive_comment is not None:

@@ -1157,7 +1154,7 @@ log.append(directive_comment)

if match:
reason, = match.groups()
(reason,) = match.groups()
if reason is None:
extra = ''
extra = ""
else:
extra = ' %s' % reason
extra = " %s" % reason
_emit_test()

@@ -1173,4 +1170,3 @@ test_name = "Bail out!%s" % extra

# Should look at buffering status and binding this to the prior result.
output.status(file_bytes=line.encode('utf8'), file_name='stdout',
mime_type=UTF8_TEXT)
output.status(file_bytes=line.encode("utf8"), file_name="stdout", mime_type=UTF8_TEXT)
_emit_test()

@@ -1208,6 +1204,7 @@ while plan_start <= plan_stop:

new_tags, gone_tags = tags_to_new_gone(tags)
source = ByteStreamToStreamResult(original, non_subunit_name='stdout')
source = ByteStreamToStreamResult(original, non_subunit_name="stdout")
class Tagger(CopyStreamResult):
def status(self, **kwargs):
tags = kwargs.get('test_tags')
tags = kwargs.get("test_tags")
if not tags:

@@ -1218,6 +1215,7 @@ tags = set()

if tags:
kwargs['test_tags'] = tags
kwargs["test_tags"] = tags
else:
kwargs['test_tags'] = None
kwargs["test_tags"] = None
super(Tagger, self).status(**kwargs)
output = Tagger([StreamResultToBytes(filtered)])

@@ -1334,3 +1332,11 @@ source.run(output)

def addDuration(self, test, duration):
"""Called to add a test duration.
:param test: The test that completed.
:param duration: The duration of the test as a float in seconds.
"""
pass
def read_test_list(path):

@@ -1342,5 +1348,4 @@ """Read a list of test ids from a file on disk.

"""
with open(path, 'r') as f:
return [line.split('#')[0].rstrip() for line in f.readlines()
if line.split('#')[0]]
with open(path, "r") as f:
return [line.split("#")[0].rstrip() for line in f.readlines() if line.split("#")[0]]

@@ -1350,3 +1355,3 @@

"""Ensure that a stream will be binary safe. See _make_binary_on_windows.
:return: A binary version of the same stream (some streams cannot be

@@ -1368,2 +1373,3 @@ 'fixed' but can be unwrapped).

import msvcrt
msvcrt.setmode(fileno, os.O_BINARY)

@@ -1383,5 +1389,5 @@

try:
stream.write(_b(''))
stream.write(b"")
except TypeError:
return stream.buffer
return stream
+36
-39
python/subunit/_output.py

@@ -25,12 +25,14 @@ # subunit: extensions to python unittest to get test results from subprocesses.

_FINAL_ACTIONS = frozenset([
'exists',
'fail',
'skip',
'success',
'uxsuccess',
'xfail',
])
_ALL_ACTIONS = _FINAL_ACTIONS.union(['inprogress'])
_CHUNK_SIZE=3670016 # 3.5 MiB
_FINAL_ACTIONS = frozenset(
[
"exists",
"fail",
"skip",
"success",
"uxsuccess",
"xfail",
]
)
_ALL_ACTIONS = _FINAL_ACTIONS.union(["inprogress"])
_CHUNK_SIZE = 3670016 # 3.5 MiB

@@ -58,4 +60,4 @@

)
parser.set_default('tags', None)
parser.set_default('test_id', None)
parser.set_default("tags", None)
parser.set_default("test_id", None)

@@ -65,4 +67,3 @@ status_commands = OptionGroup(

"Status Commands",
"These options report the status of a test. TEST_ID must be a string "
"that uniquely identifies the test."
"These options report the status of a test. TEST_ID must be a string that uniquely identifies the test.",
)

@@ -78,3 +79,3 @@ for action_name in _ALL_ACTIONS:

metavar="TEST_ID",
help="Report a test status."
help="Report a test status.",
)

@@ -87,6 +88,6 @@ parser.add_option_group(status_commands)

"These options control attaching data to a result stream. They can "
"either be specified with a status command, in which case the file "
"is attached to the test status, or by themselves, in which case "
"the file is attached to the stream (and not associated with any "
"test id)."
"either be specified with a status command, in which case the file "
"is attached to the test status, or by themselves, in which case "
"the file is attached to the stream (and not associated with any "
"test id).",
)

@@ -96,5 +97,5 @@ file_commands.add_option(

help="Attach a file to the result stream for this test. If '-' is "
"specified, stdin will be read instead. In this case, the file "
"name will be set to 'stdin' (but can still be overridden with "
"the --file-name option)."
"specified, stdin will be read instead. In this case, the file "
"name will be set to 'stdin' (but can still be overridden with "
"the --file-name option).",
)

@@ -104,13 +105,13 @@ file_commands.add_option(

help="The name to give this file attachment. If not specified, the "
"name of the file on disk will be used, or 'stdin' in the case "
"where '-' was passed to the '--attach-file' argument. This option"
" may only be specified when '--attach-file' is specified.",
)
"name of the file on disk will be used, or 'stdin' in the case "
"where '-' was passed to the '--attach-file' argument. This option"
" may only be specified when '--attach-file' is specified.",
)
file_commands.add_option(
"--mimetype",
help="The mime type to send with this file. This is only used if the "
"--attach-file argument is used. This argument is optional. If it "
"is not specified, the file will be sent without a mime type. This "
"option may only be specified when '--attach-file' is specified.",
default=None
"--attach-file argument is used. This argument is optional. If it "
"is not specified, the file will be sent without a mime type. This "
"option may only be specified when '--attach-file' is specified.",
default=None,
)

@@ -120,7 +121,3 @@ parser.add_option_group(file_commands)

parser.add_option(
"--tag",
help="Specifies a tag. May be used multiple times",
action="append",
dest="tags",
default=[]
"--tag", help="Specifies a tag. May be used multiple times", action="append", dest="tags", default=[]
)

@@ -134,9 +131,9 @@

if options.attach_file:
if options.attach_file == '-':
if options.attach_file == "-":
if not options.file_name:
options.file_name = 'stdin'
options.file_name = "stdin"
options.attach_file = make_stream_binary(sys.stdin)
else:
try:
options.attach_file = open(options.attach_file, 'rb')
options.attach_file = open(options.attach_file, "rb")
except IOError as e:

@@ -185,3 +182,3 @@ parser.error("Cannot open %s (%s)" % (options.attach_file, e.strerror))

write_status = partial(write_status, file_name=filename, file_bytes=this_file_hunk)
if next_file_hunk == b'':
if next_file_hunk == b"":
write_status = partial(write_status, eof=True)

@@ -188,0 +185,0 @@ is_last_packet = True

+25
-26
python/subunit/_to_disk.py

@@ -8,3 +8,3 @@ # subunit: extensions to python unittest to get test results from subprocesses.

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -43,3 +43,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

if not candidate.startswith(realroot):
sub = sub.replace('/', '_').replace('\\', '_')
sub = sub.replace("/", "_").replace("\\", "_")
return _allocate_path(root, sub)

@@ -51,3 +51,3 @@

attempt += 1
probe = '%s-%s' % (candidate, attempt)
probe = "%s-%s" % (candidate, attempt)
return probe

@@ -63,3 +63,3 @@

raise
return io.open(name, 'wb')
return io.open(name, "wb")

@@ -80,20 +80,19 @@

def export(self, test_dict):
id = test_dict['id']
tags = sorted(test_dict['tags'])
details = test_dict['details']
status = test_dict['status']
start, stop = test_dict['timestamps']
id = test_dict["id"]
tags = sorted(test_dict["tags"])
details = test_dict["details"]
status = test_dict["status"]
start, stop = test_dict["timestamps"]
test_summary = {}
test_summary['id'] = id
test_summary['tags'] = tags
test_summary['status'] = status
test_summary['details'] = sorted(details.keys())
test_summary['start'] = _json_time(start)
test_summary['stop'] = _json_time(stop)
test_summary["id"] = id
test_summary["tags"] = tags
test_summary["status"] = status
test_summary["details"] = sorted(details.keys())
test_summary["start"] = _json_time(start)
test_summary["stop"] = _json_time(stop)
root = _allocate_path(self._directory, id)
with _open_path(root, 'test.json') as f:
maybe_str = json.dumps(
test_summary, sort_keys=True, ensure_ascii=False)
with _open_path(root, "test.json") as f:
maybe_str = json.dumps(test_summary, sort_keys=True, ensure_ascii=False)
if not isinstance(maybe_str, bytes):
maybe_str = maybe_str.encode('utf-8')
maybe_str = maybe_str.encode("utf-8")
f.write(maybe_str)

@@ -113,3 +112,4 @@ for name, detail in details.items():

description="Export a subunit stream to files on disk.",
epilog=dedent("""\
epilog=dedent(
"""\
Creates a directory per test id, a JSON file with test

@@ -122,6 +122,6 @@ metadata within that directory, and each attachment

Exits 0 if the export was completed, or non-zero otherwise.
"""))
parser.add_option(
"-d", "--directory", help="Root directory to export to.",
default=".")
"""
),
)
parser.add_option("-d", "--directory", help="Root directory to export to.", default=".")
options, args = parser.parse_args(argv)

@@ -131,3 +131,3 @@ if len(args) > 1:

if len(args):
source = io.open(args[0], 'rb')
source = io.open(args[0], "rb")
else:

@@ -139,2 +139,1 @@ source = stdin

return 0
+11
-13
python/subunit/chunked.py

@@ -20,5 +20,4 @@ #

from testtools.compat import _b
empty = b""
empty = _b('')

@@ -50,7 +49,7 @@ class Decoder(object):

self.strict = strict
self._match_chars = _b("0123456789abcdefABCDEF\r\n")
self._slash_n = _b('\n')
self._slash_r = _b('\r')
self._slash_rn = _b('\r\n')
self._slash_nr = _b('\n\r')
self._match_chars = b"0123456789abcdefABCDEF\r\n"
self._slash_n = b"\n"
self._slash_r = b"\r"
self._slash_rn = b"\r\n"
self._slash_nr = b"\n\r"

@@ -85,5 +84,4 @@ def close(self):

else:
self.output.write(self.buffered_bytes[0][:self.body_length])
self.buffered_bytes[0] = \
self.buffered_bytes[0][self.body_length:]
self.output.write(self.buffered_bytes[0][: self.body_length])
self.buffered_bytes[0] = self.buffered_bytes[0][self.body_length :]
self.body_length = 0

@@ -98,3 +96,3 @@ self.state = self._read_length

for pos in range(len(bytes)):
byte = bytes[pos:pos+1]
byte = bytes[pos : pos + 1]
if byte not in self._match_chars:

@@ -171,3 +169,3 @@ break

self.buffer_size = 0
self.output.write(_b("%X\r\n" % (buffer_size + extra_len)))
self.output.write(b"%X\r\n" % (buffer_size + extra_len))
if buffer_size:

@@ -190,2 +188,2 @@ self.output.write(empty.join(buffered_bytes))

self.flush()
self.output.write(_b("0\r\n"))
self.output.write(b"0\r\n")
+16
-21
python/subunit/details.py

@@ -22,9 +22,8 @@ #

from testtools import content, content_type
from testtools.compat import _b
from subunit import chunked
end_marker = _b("]\n")
quoted_marker = _b(" ]")
empty = _b('')
end_marker = b"]\n"
quoted_marker = b" ]"
empty = b""

@@ -40,3 +39,3 @@

def __init__(self, state):
self._message = _b("")
self._message = b""
self._state = state

@@ -60,14 +59,11 @@

# or something like that.
result['traceback'] = content.Content(
content_type.ContentType("text", "x-traceback",
{"charset": "utf8"}),
lambda:[self._message])
result["traceback"] = content.Content(
content_type.ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [self._message]
)
else:
if style == 'skip':
name = 'reason'
if style == "skip":
name = "reason"
else:
name = 'message'
result[name] = content.Content(
content_type.ContentType("text", "plain"),
lambda:[self._message])
name = "message"
result[name] = content.Content(content_type.ContentType("text", "plain"), lambda: [self._message])
return result

@@ -92,5 +88,5 @@

# TODO error handling
field, value = line[:-1].decode('utf8').split(' ', 1)
field, value = line[:-1].decode("utf8").split(" ", 1)
try:
main, sub = value.split('/')
main, sub = value.split("/")
except ValueError:

@@ -102,3 +98,3 @@ raise ValueError("Invalid MIME type %r" % value)

def _get_name(self, line):
self._name = line[:-1].decode('utf8')
self._name = line[:-1].decode("utf8")
self._body = BytesIO()

@@ -112,6 +108,5 @@ self._chunk_parser = chunked.Decoder(self._body)

# Line based use always ends on no residue.
assert residue == empty, 'residue: %r' % (residue,)
assert residue == empty, "residue: %r" % (residue,)
body = self._body
self._details[self._name] = content.Content(
self._content_type, lambda:[body.getvalue()])
self._details[self._name] = content.Content(self._content_type, lambda: [body.getvalue()])
self._chunk_parser.close()

@@ -118,0 +113,0 @@ self._parse_state = self._look_for_content

+2
-3
python/subunit/filter_scripts/subunit_1to2.py

@@ -36,8 +36,7 @@ #!/usr/bin/env python3

(options, args) = parser.parse_args()
run_tests_from_stream(find_stream(sys.stdin, args),
ExtendedToStreamDecorator(StreamResultToBytes(sys.stdout)))
run_tests_from_stream(find_stream(sys.stdin, args), ExtendedToStreamDecorator(StreamResultToBytes(sys.stdout)))
sys.exit(0)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+3
-4
python/subunit/filter_scripts/subunit_2to1.py

@@ -37,8 +37,7 @@ #!/usr/bin/env python3

(options, args) = parser.parse_args()
case = ByteStreamToStreamResult(
find_stream(sys.stdin, args), non_subunit_name='stdout')
case = ByteStreamToStreamResult(find_stream(sys.stdin, args), non_subunit_name="stdout")
result = StreamToExtendedDecorator(TestProtocolClient(sys.stdout))
result = StreamResultRouter(result)
cat = CatFiles(sys.stdout)
result.add_rule(cat, 'test_id', test_id=None)
result.add_rule(cat, "test_id", test_id=None)
result.startTestRun()

@@ -50,3 +49,3 @@ case.run(result)

if __name__ == '__main__':
if __name__ == "__main__":
main()
+85
-62
python/subunit/filter_scripts/subunit_filter.py

@@ -37,4 +37,3 @@ #!/usr/bin/env python3

from subunit.filters import filter_by_result, find_stream
from subunit.test_results import (TestResultFilter, and_predicates,
make_tag_filter)
from subunit.test_results import TestResultFilter, and_predicates, make_tag_filter

@@ -44,48 +43,68 @@

parser = OptionParser(description=__doc__)
parser.add_option("--error", action="store_false",
help="include errors", default=False, dest="error")
parser.add_option("-e", "--no-error", action="store_true",
help="exclude errors", dest="error")
parser.add_option("--failure", action="store_false",
help="include failures", default=False, dest="failure")
parser.add_option("-f", "--no-failure", action="store_true",
help="exclude failures", dest="failure")
parser.add_option("--passthrough", action="store_false",
help="Forward non-subunit input as 'stdout'.", default=False,
dest="no_passthrough")
parser.add_option("--no-passthrough", action="store_true",
help="Discard all non subunit input.", default=False,
dest="no_passthrough")
parser.add_option("-s", "--success", action="store_false",
help="include successes", dest="success")
parser.add_option("--no-success", action="store_true",
help="exclude successes", default=True, dest="success")
parser.add_option("--no-skip", action="store_true",
help="exclude skips", dest="skip")
parser.add_option("--xfail", action="store_false",
help="include expected failures", default=True, dest="xfail")
parser.add_option("--no-xfail", action="store_true",
help="exclude expected failures", default=True, dest="xfail")
parser.add_option("--error", action="store_false", help="include errors", default=False, dest="error")
parser.add_option("-e", "--no-error", action="store_true", help="exclude errors", dest="error")
parser.add_option("--failure", action="store_false", help="include failures", default=False, dest="failure")
parser.add_option("-f", "--no-failure", action="store_true", help="exclude failures", dest="failure")
parser.add_option(
"--with-tag", type=str,
help="include tests with these tags", action="append", dest="with_tags")
"--passthrough",
action="store_false",
help="Forward non-subunit input as 'stdout'.",
default=False,
dest="no_passthrough",
)
parser.add_option(
"--without-tag", type=str,
help="exclude tests with these tags", action="append", dest="without_tags")
parser.add_option("-m", "--with", type=str,
"--no-passthrough",
action="store_true",
help="Discard all non subunit input.",
default=False,
dest="no_passthrough",
)
parser.add_option("-s", "--success", action="store_false", help="include successes", dest="success")
parser.add_option("--no-success", action="store_true", help="exclude successes", default=True, dest="success")
parser.add_option("--no-skip", action="store_true", help="exclude skips", dest="skip")
parser.add_option("--xfail", action="store_false", help="include expected failures", default=True, dest="xfail")
parser.add_option("--no-xfail", action="store_true", help="exclude expected failures", default=True, dest="xfail")
parser.add_option("--with-tag", type=str, help="include tests with these tags", action="append", dest="with_tags")
parser.add_option(
"--without-tag", type=str, help="exclude tests with these tags", action="append", dest="without_tags"
)
parser.add_option(
"-m",
"--with",
type=str,
help="regexp to include (case-sensitive by default)",
action="append", dest="with_regexps")
parser.add_option("--fixup-expected-failures", type=str,
action="append",
dest="with_regexps",
)
parser.add_option(
"--fixup-expected-failures",
type=str,
help="File with list of test ids that are expected to fail; on failure "
"their result will be changed to xfail; on success they will be "
"changed to error.", dest="fixup_expected_failures", action="append")
parser.add_option("--without", type=str,
"their result will be changed to xfail; on success they will be "
"changed to error.",
dest="fixup_expected_failures",
action="append",
)
parser.add_option(
"--without",
type=str,
help="regexp to exclude (case-sensitive by default)",
action="append", dest="without_regexps")
parser.add_option("-F", "--only-genuine-failures", action="callback",
action="append",
dest="without_regexps",
)
parser.add_option(
"-F",
"--only-genuine-failures",
action="callback",
callback=only_genuine_failures_callback,
help="Only pass through failures and exceptions.")
parser.add_option("--rename", action="append", nargs=2,
help="Apply specified regex subsitutions to test names.",
dest="renames", default=[])
help="Only pass through failures and exceptions.",
)
parser.add_option(
"--rename",
action="append",
nargs=2,
help="Apply specified regex substitutions to test names.",
dest="renames",
default=[],
)
return parser

@@ -95,6 +114,6 @@

def only_genuine_failures_callback(option, opt, value, parser):
parser.rargs.insert(0, '--no-passthrough')
parser.rargs.insert(0, '--no-xfail')
parser.rargs.insert(0, '--no-skip')
parser.rargs.insert(0, '--no-success')
parser.rargs.insert(0, "--no-passthrough")
parser.rargs.insert(0, "--no-xfail")
parser.rargs.insert(0, "--no-skip")
parser.rargs.insert(0, "--no-success")

@@ -123,2 +142,3 @@

return True
return check_regexps

@@ -129,5 +149,6 @@

def rename(name):
for (from_pattern, to_pattern) in patterns:
for from_pattern, to_pattern in patterns:
name = re.sub(from_pattern, to_pattern, name)
return name
return rename

@@ -141,13 +162,15 @@

fixup_expected_failures.update(read_test_list(path))
return StreamToExtendedDecorator(TestResultFilter(
ExtendedToStreamDecorator(
StreamResultToBytes(output)),
filter_error=options.error,
filter_failure=options.failure,
filter_success=options.success,
filter_skip=options.skip,
filter_xfail=options.xfail,
filter_predicate=predicate,
fixup_expected_failures=fixup_expected_failures,
rename=_compile_rename(options.renames)))
return StreamToExtendedDecorator(
TestResultFilter(
ExtendedToStreamDecorator(StreamResultToBytes(output)),
filter_error=options.error,
filter_failure=options.failure,
filter_success=options.success,
filter_skip=options.skip,
filter_xfail=options.xfail,
filter_predicate=predicate,
fixup_expected_failures=fixup_expected_failures,
rename=_compile_rename(options.renames),
)
)

@@ -159,4 +182,3 @@

regexp_filter = _make_regexp_filter(
options.with_regexps, options.without_regexps)
regexp_filter = _make_regexp_filter(options.with_regexps, options.without_regexps)
tag_filter = make_tag_filter(options.with_tags, options.without_tags)

@@ -171,7 +193,8 @@ filter_predicate = and_predicates([regexp_filter, tag_filter])

protocol_version=2,
input_stream=find_stream(sys.stdin, args))
input_stream=find_stream(sys.stdin, args),
)
sys.exit(0)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+18
-11
python/subunit/filter_scripts/subunit_ls.py

@@ -31,13 +31,20 @@ #!/usr/bin/env python

parser = OptionParser(description=__doc__)
parser.add_option("--times", action="store_true",
parser.add_option(
"--times",
action="store_true",
help="list the time each test took (requires a timestamped stream)",
default=False)
parser.add_option("--exists", action="store_true",
help="list tests that are reported as existing (as well as ran)",
default=False)
parser.add_option("--no-passthrough", action="store_true",
help="Hide all non subunit input.", default=False, dest="no_passthrough")
default=False,
)
parser.add_option(
"--exists", action="store_true", help="list tests that are reported as existing (as well as ran)", default=False
)
parser.add_option(
"--no-passthrough",
action="store_true",
help="Hide all non subunit input.",
default=False,
dest="no_passthrough",
)
(options, args) = parser.parse_args()
test = ByteStreamToStreamResult(
find_stream(sys.stdin, args), non_subunit_name="stdout")
test = ByteStreamToStreamResult(find_stream(sys.stdin, args), non_subunit_name="stdout")
result = TestIdPrintingResult(sys.stdout, options.times, options.exists)

@@ -47,3 +54,3 @@ if not options.no_passthrough:

cat = CatFiles(sys.stdout)
result.add_rule(cat, 'test_id', test_id=None)
result.add_rule(cat, "test_id", test_id=None)
summary = StreamSummary()

@@ -61,3 +68,3 @@ result = CopyStreamResult([result, summary])

if __name__ == '__main__':
if __name__ == "__main__":
main()
+5
-5
python/subunit/filter_scripts/subunit_notify.py

@@ -9,3 +9,3 @@ #!/usr/bin/env python3

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -23,3 +23,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

gi.require_version('Gtk', '3.0')
gi.require_version("Gtk", "3.0")
from gi.repository import Notify # noqa: E402

@@ -52,7 +52,7 @@ from testtools import StreamToExtendedDecorator # noqa: E402

run_filter_script(
lambda output:StreamToExtendedDecorator(TestResultStats(output)),
__doc__, notify_of_result, protocol_version=2)
lambda output: StreamToExtendedDecorator(TestResultStats(output)), __doc__, notify_of_result, protocol_version=2
)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+1
-1
python/subunit/filter_scripts/subunit_output.py

@@ -28,3 +28,3 @@ #!/usr/bin/env python3

if __name__ == '__main__':
if __name__ == "__main__":
main()
+10
-5
python/subunit/filter_scripts/subunit_stats.py

@@ -9,3 +9,3 @@ #!/usr/bin/env python3

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -33,8 +33,13 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

r.decorated.formatStats()
run_filter_script(
lambda output:StreamToExtendedDecorator(result),
__doc__, show_stats, protocol_version=2, passthrough_subunit=False)
lambda output: StreamToExtendedDecorator(result),
__doc__,
show_stats,
protocol_version=2,
passthrough_subunit=False,
)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+2
-2
python/subunit/filter_scripts/subunit_tags.py

@@ -9,3 +9,3 @@ #!/usr/bin/env python3

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -33,3 +33,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

if __name__ == '__main__':
if __name__ == "__main__":
main()
+2
-3
python/subunit/filter_scripts/subunit2csv.py

@@ -26,7 +26,6 @@ #!/usr/bin/env python3

def main():
run_filter_script(lambda output: StreamToExtendedDecorator(
CsvResult(output)), __doc__, protocol_version=2)
run_filter_script(lambda output: StreamToExtendedDecorator(CsvResult(output)), __doc__, protocol_version=2)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+1
-1
python/subunit/filter_scripts/subunit2disk.py

@@ -28,3 +28,3 @@ #!/usr/bin/env python3

if __name__ == '__main__':
if __name__ == "__main__":
main()
+88
-25
python/subunit/filter_scripts/subunit2gtk.py

@@ -54,8 +54,12 @@ #!/usr/bin/env python3

gi.require_version('Gtk', '3.0')
from gi.repository import GObject, Gtk # noqa: E402
gi.require_version("Gtk", "3.0")
from gi.repository import GObject, Gtk # noqa: E402
from testtools import StreamToExtendedDecorator # noqa: E402
from subunit import (PROGRESS_POP, PROGRESS_PUSH, PROGRESS_SET, # noqa: E402
ByteStreamToStreamResult)
from subunit import ( # noqa: E402
PROGRESS_POP,
PROGRESS_PUSH,
PROGRESS_SET,
ByteStreamToStreamResult,
)
from subunit.progress_model import ProgressModel # noqa: E402

@@ -65,3 +69,2 @@

class GTKTestResult(unittest.TestResult):
def __init__(self):

@@ -110,26 +113,80 @@ super(GTKTestResult, self).__init__()

label = Gtk.Label(label="Run:")
table.attach(label, 0, 1, 1, 2, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
label,
0,
1,
1,
2,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
label.show()
self.run_label = Gtk.Label(label="N/A")
table.attach(self.run_label, 1, 2, 1, 2, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
self.run_label,
1,
2,
1,
2,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
self.run_label.show()
label = Gtk.Label(label="OK:")
table.attach(label, 0, 1, 2, 3, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
label,
0,
1,
2,
3,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
label.show()
self.ok_label = Gtk.Label(label="N/A")
table.attach(self.ok_label, 1, 2, 2, 3, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
self.ok_label,
1,
2,
2,
3,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
self.ok_label.show()
label = Gtk.Label(label="Not OK:")
table.attach(label, 0, 1, 3, 4, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
label,
0,
1,
3,
4,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
label.show()
self.not_ok_label = Gtk.Label(label="N/A")
table.attach(self.not_ok_label, 1, 2, 3, 4, Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL, 5, 5)
table.attach(
self.not_ok_label,
1,
2,
3,
4,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
Gtk.AttachOptions.EXPAND | Gtk.AttachOptions.FILL,
5,
5,
)
self.not_ok_label.show()

@@ -153,11 +210,8 @@

width = self.progress_model.width()
percentage = (pos / float(width))
percentage = pos / float(width)
self.pbar.set_fraction(percentage)
def stopTestRun(self):
try:
super(GTKTestResult, self).stopTestRun()
except AttributeError:
pass
GObject.idle_add(self.pbar.set_text, 'Finished')
super(GTKTestResult, self).stopTestRun()
GObject.idle_add(self.pbar.set_text, "Finished")

@@ -204,2 +258,10 @@ def addError(self, test, err):

def addDuration(self, test, duration):
"""Called to add a test duration.
:param test: The test that completed.
:param duration: The duration of the test as a float in seconds.
"""
pass
def update_counts(self):

@@ -215,3 +277,3 @@ self.run_label.set_text(str(self.testsRun))

result = StreamToExtendedDecorator(GTKTestResult())
test = ByteStreamToStreamResult(sys.stdin, non_subunit_name='stdout')
test = ByteStreamToStreamResult(sys.stdin, non_subunit_name="stdout")
# Get setup

@@ -225,2 +287,3 @@ while Gtk.events_pending():

result.stopTestRun()
t = threading.Thread(target=run_and_finish)

@@ -238,3 +301,3 @@ t.daemon = True

if __name__ == '__main__':
if __name__ == "__main__":
main()
+6
-7
python/subunit/filter_scripts/subunit2junitxml.py

@@ -19,3 +19,2 @@ #!/usr/bin/env python3

import sys

@@ -30,4 +29,6 @@

except ImportError:
sys.stderr.write("python-junitxml (https://launchpad.net/pyjunitxml or "
"http://pypi.python.org/pypi/junitxml) is required for this filter.")
sys.stderr.write(
"python-junitxml (https://launchpad.net/pyjunitxml or "
"http://pypi.python.org/pypi/junitxml) is required for this filter."
)
raise

@@ -37,8 +38,6 @@

def main():
run_filter_script(
lambda output: StreamToExtendedDecorator(
JUnitXmlResult(output)), __doc__, protocol_version=2)
run_filter_script(lambda output: StreamToExtendedDecorator(JUnitXmlResult(output)), __doc__, protocol_version=2)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+21
-17
python/subunit/filter_scripts/subunit2pyunit.py

@@ -24,4 +24,3 @@ #!/usr/bin/env python

from testtools import (DecorateTestCaseResult, StreamResultRouter,
StreamToExtendedDecorator)
from testtools import DecorateTestCaseResult, StreamResultRouter, StreamToExtendedDecorator

@@ -35,11 +34,14 @@ from subunit import ByteStreamToStreamResult

parser = OptionParser(description=__doc__)
parser.add_option("--no-passthrough", action="store_true",
help="Hide all non subunit input.",
default=False, dest="no_passthrough")
parser.add_option("--progress", action="store_true",
help="Use bzrlib's test reporter (requires bzrlib)",
default=False)
parser.add_option(
"--no-passthrough",
action="store_true",
help="Hide all non subunit input.",
default=False,
dest="no_passthrough",
)
parser.add_option(
"--progress", action="store_true", help="Use breezy 's test reporter (requires breezy)", default=False
)
(options, args) = parser.parse_args()
test = ByteStreamToStreamResult(
find_stream(sys.stdin, args), non_subunit_name='stdout')
test = ByteStreamToStreamResult(find_stream(sys.stdin, args), non_subunit_name="stdout")

@@ -50,10 +52,12 @@ def wrap_result(result):

result = StreamResultRouter(result)
result.add_rule(CatFiles(sys.stdout), 'test_id', test_id=None)
result.add_rule(CatFiles(sys.stdout), "test_id", test_id=None)
return result
test = DecorateTestCaseResult(test, wrap_result,
before_run=methodcaller('startTestRun'),
after_run=methodcaller('stopTestRun'))
test = DecorateTestCaseResult(
test, wrap_result, before_run=methodcaller("startTestRun"), after_run=methodcaller("stopTestRun")
)
if options.progress:
from bzrlib import ui
from bzrlib.tests import TextTestRunner
from breezy import ui
from breezy.tests import TextTestRunner
ui.ui_factory = ui.make_ui_for_terminal(None, sys.stdout, sys.stderr)

@@ -70,3 +74,3 @@ runner = TextTestRunner()

if __name__ == '__main__':
if __name__ == "__main__":
main()
+2
-2
python/subunit/filter_scripts/tap2subunit.py

@@ -9,3 +9,3 @@ #!/usr/bin/env python3

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -33,3 +33,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

if __name__ == '__main__':
if __name__ == "__main__":
main()
+50
-38
python/subunit/filters.py

@@ -22,4 +22,3 @@ # subunit: extensions to python unittest to get test results from subprocesses.

from subunit import (ByteStreamToStreamResult, DiscardStream, ProtocolTestCase,
StreamResultToBytes)
from subunit import ByteStreamToStreamResult, DiscardStream, ProtocolTestCase, StreamResultToBytes
from subunit.test_results import CatFiles

@@ -31,17 +30,22 @@

parser.add_option(
"--no-passthrough", action="store_true",
help="Hide all non subunit input.", default=False,
dest="no_passthrough")
"--no-passthrough",
action="store_true",
help="Hide all non subunit input.",
default=False,
dest="no_passthrough",
)
parser.add_option("-o", "--output-to", help="Send the output to this path rather than stdout.")
parser.add_option(
"-o", "--output-to",
help="Send the output to this path rather than stdout.")
parser.add_option(
"-f", "--forward", action="store_true", default=False,
help="Forward subunit stream on stdout. When set, received "
"non-subunit output will be encapsulated in subunit.")
"-f",
"--forward",
action="store_true",
default=False,
help="Forward subunit stream on stdout. When set, received non-subunit output will be encapsulated in subunit.",
)
return parser
def run_tests_from_stream(input_stream, result, passthrough_stream=None,
forward_stream=None, protocol_version=1, passthrough_subunit=True):
def run_tests_from_stream(
input_stream, result, passthrough_stream=None, forward_stream=None, protocol_version=1, passthrough_subunit=True
):
"""Run tests from a subunit input stream through 'result'.

@@ -71,7 +75,5 @@

"""
if 1==protocol_version:
test = ProtocolTestCase(
input_stream, passthrough=passthrough_stream,
forward=forward_stream)
elif 2==protocol_version:
if 1 == protocol_version:
test = ProtocolTestCase(input_stream, passthrough=passthrough_stream, forward=forward_stream)
elif 2 == protocol_version:
# In all cases we encapsulate unknown inputs.

@@ -85,3 +87,3 @@ if forward_stream is not None:

router = StreamResultRouter(forward_result)
router.add_rule(StreamResult(), 'test_id', test_id=None)
router.add_rule(StreamResult(), "test_id", test_id=None)
result = CopyStreamResult([router, result])

@@ -99,5 +101,4 @@ else:

result = StreamResultRouter(result)
result.add_rule(passthrough_result, 'test_id', test_id=None)
test = ByteStreamToStreamResult(input_stream,
non_subunit_name='stdout')
result.add_rule(passthrough_result, "test_id", test_id=None)
test = ByteStreamToStreamResult(input_stream, non_subunit_name="stdout")
else:

@@ -110,5 +111,11 @@ raise Exception("Unknown protocol version.")

def filter_by_result(result_factory, output_path, passthrough, forward,
input_stream=sys.stdin, protocol_version=1,
passthrough_subunit=True):
def filter_by_result(
result_factory,
output_path,
passthrough,
forward,
input_stream=sys.stdin,
protocol_version=1,
passthrough_subunit=True,
):
"""Filter an input stream using a test result.

@@ -134,3 +141,3 @@

else:
if 1==protocol_version:
if 1 == protocol_version:
passthrough_stream = DiscardStream()

@@ -142,3 +149,3 @@ else:

forward_stream = sys.stdout
elif 1==protocol_version:
elif 1 == protocol_version:
forward_stream = DiscardStream()

@@ -151,3 +158,3 @@ else:

else:
output_to = open(output_path, 'w')
output_to = open(output_path, "w")

@@ -157,5 +164,9 @@ try:

run_tests_from_stream(
input_stream, result, passthrough_stream, forward_stream,
input_stream,
result,
passthrough_stream,
forward_stream,
protocol_version=protocol_version,
passthrough_subunit=passthrough_subunit)
passthrough_subunit=passthrough_subunit,
)
finally:

@@ -167,4 +178,3 @@ if output_path:

def run_filter_script(result_factory, description, post_run_hook=None,
protocol_version=1, passthrough_subunit=True):
def run_filter_script(result_factory, description, post_run_hook=None, protocol_version=1, passthrough_subunit=True):
"""Main function for simple subunit filter scripts.

@@ -188,10 +198,12 @@

result = filter_by_result(
result_factory, options.output_to, not options.no_passthrough,
options.forward, protocol_version=protocol_version,
result_factory,
options.output_to,
not options.no_passthrough,
options.forward,
protocol_version=protocol_version,
passthrough_subunit=passthrough_subunit,
input_stream=find_stream(sys.stdin, args))
input_stream=find_stream(sys.stdin, args),
)
if post_run_hook:
post_run_hook(result)
if not hasattr(result, 'wasSuccessful'):
result = result.decorated
if result.wasSuccessful():

@@ -213,4 +225,4 @@ sys.exit(0)

if argv:
return open(argv[0], 'rb')
return open(argv[0], "rb")
else:
return stdin
+4
-4
python/subunit/progress_model.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -20,5 +20,6 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

class ProgressModel(object):
"""A model of progress indicators as subunit defines it.
Instances of this class represent a single logical operation that is

@@ -45,3 +46,3 @@ progressing. The operation may have many steps, and some of those steps may

"""Create a ProgressModel.
The new model has no progress data at all - it will claim a summary

@@ -109,2 +110,1 @@ width of zero and position of 0.

return task[1]
+22
-26
python/subunit/run.py

@@ -5,3 +5,3 @@ #!/usr/bin/python3

# Copyright (C) Jelmer Vernooij <jelmer@samba.org> 2007
#
#
# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause

@@ -11,3 +11,3 @@ # license at the users choice. A copy of both licenses are available in the

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -22,3 +22,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

$ python -m subunit.run mylib.tests.test_suite
$ python -m subunit.run mylib.tests.test_suite
"""

@@ -31,4 +31,3 @@

from testtools import ExtendedToStreamDecorator
from testtools.run import (BUFFEROUTPUT, CATCHBREAK, FAILFAST, USAGE_AS_MAIN,
TestProgram, list_test)
from testtools.run import BUFFEROUTPUT, CATCHBREAK, FAILFAST, USAGE_AS_MAIN, TestProgram, list_test

@@ -40,4 +39,3 @@ from subunit import StreamResultToBytes

class SubunitTestRunner(object):
def __init__(self, verbosity=None, failfast=None, buffer=None, stream=None,
stdout=None, tb_locals=False):
def __init__(self, verbosity=None, failfast=None, buffer=None, stream=None, stdout=None, tb_locals=False):
"""Create a TestToolsTestRunner.

@@ -82,5 +80,6 @@

if errors:
failed_descr = '\n'.join(errors).encode('utf8')
result.status(file_name="import errors", runnable=False,
file_bytes=failed_descr, mime_type="text/plain;charset=utf8")
failed_descr = "\n".join(errors).encode("utf8")
result.status(
file_name="import errors", runnable=False, file_bytes=failed_descr, mime_type="text/plain;charset=utf8"
)
sys.exit(2)

@@ -95,3 +94,3 @@

if fileno is not None:
stream = os.fdopen(fileno, 'wb', 0)
stream = os.fdopen(fileno, "wb", 0)
else:

@@ -101,3 +100,3 @@ stream = self.stream

for test_id in test_ids:
result.status(test_id=test_id, test_status='exists')
result.status(test_id=test_id, test_status="exists")
return result, errors

@@ -107,3 +106,2 @@

class SubunitTestProgram(TestProgram):
USAGE = USAGE_AS_MAIN

@@ -113,16 +111,15 @@

if msg:
print (msg)
usage = {'progName': self.progName, 'catchbreak': '', 'failfast': '',
'buffer': ''}
print(msg)
usage = {"progName": self.progName, "catchbreak": "", "failfast": "", "buffer": ""}
if self.failfast is not False:
usage['failfast'] = FAILFAST
usage["failfast"] = FAILFAST
if self.catchbreak is not False:
usage['catchbreak'] = CATCHBREAK
usage["catchbreak"] = CATCHBREAK
if self.buffer is not False:
usage['buffer'] = BUFFEROUTPUT
usage["buffer"] = BUFFEROUTPUT
usage_text = self.USAGE % usage
usage_lines = usage_text.split('\n')
usage_lines = usage_text.split("\n")
usage_lines.insert(2, "Run a test suite with a subunit reporter.")
usage_lines.insert(3, "")
print('\n'.join(usage_lines))
print("\n".join(usage_lines))
sys.exit(2)

@@ -140,12 +137,11 @@

# on non-ttys.
if hasattr(stdout, 'fileno'):
if hasattr(stdout, "fileno"):
# Patch stdout to be unbuffered, so that pdb works well on 2.6/2.7.
binstdout = io.open(stdout.fileno(), 'wb', 0)
binstdout = io.open(stdout.fileno(), "wb", 0)
sys.stdout = io.TextIOWrapper(binstdout, encoding=sys.stdout.encoding)
stdout = sys.stdout
SubunitTestProgram(module=None, argv=argv, testRunner=runner,
stdout=stdout, exit=False)
SubunitTestProgram(module=None, argv=argv, testRunner=runner, stdout=stdout, exit=False)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+95
-225
python/subunit/test_results.py

@@ -23,4 +23,3 @@ #

import testtools
from testtools import StreamResult
from testtools.content import TracebackContent, text_content
from testtools import StreamResult, TestResultDecorator, TestByTestResult

@@ -31,82 +30,2 @@ import iso8601

# NOT a TestResult, because we are implementing the interface, not inheriting
# it.
class TestResultDecorator:
"""General pass-through decorator.
This provides a base that other TestResults can inherit from to
gain basic forwarding functionality. It also takes care of
handling the case where the target doesn't support newer methods
or features by degrading them.
"""
# XXX: Since lp:testtools r250, this is in testtools. Once it's released,
# we should gut this and just use that.
def __init__(self, decorated):
"""Create a TestResultDecorator forwarding to decorated."""
# Make every decorator degrade gracefully.
self.decorated = testtools.ExtendedToOriginalDecorator(decorated)
def startTest(self, test):
return self.decorated.startTest(test)
def startTestRun(self):
return self.decorated.startTestRun()
def stopTest(self, test):
return self.decorated.stopTest(test)
def stopTestRun(self):
return self.decorated.stopTestRun()
def addError(self, test, err=None, details=None):
return self.decorated.addError(test, err, details=details)
def addFailure(self, test, err=None, details=None):
return self.decorated.addFailure(test, err, details=details)
def addSuccess(self, test, details=None):
return self.decorated.addSuccess(test, details=details)
def addSkip(self, test, reason=None, details=None):
return self.decorated.addSkip(test, reason, details=details)
def addExpectedFailure(self, test, err=None, details=None):
return self.decorated.addExpectedFailure(test, err, details=details)
def addUnexpectedSuccess(self, test, details=None):
return self.decorated.addUnexpectedSuccess(test, details=details)
def _get_failfast(self):
return getattr(self.decorated, 'failfast', False)
def _set_failfast(self, value):
self.decorated.failfast = value
failfast = property(_get_failfast, _set_failfast)
def progress(self, offset, whence):
return self.decorated.progress(offset, whence)
def wasSuccessful(self):
return self.decorated.wasSuccessful()
@property
def shouldStop(self):
return self.decorated.shouldStop
def stop(self):
return self.decorated.stop()
@property
def testsRun(self):
return self.decorated.testsRun
def tags(self, new_tags, gone_tags):
return self.decorated.tags(new_tags, gone_tags)
def time(self, a_datetime):
return self.decorated.time(a_datetime)
class HookedTestResultDecorator(TestResultDecorator):

@@ -116,2 +35,4 @@ """A TestResult which calls a hook on every event."""

def __init__(self, decorated):
# Wrap with ExtendedToOriginalDecorator to handle unittest TestResults
decorated = testtools.ExtendedToOriginalDecorator(decorated)
self.super = super()

@@ -181,3 +102,16 @@ self.super.__init__(decorated)

def addDuration(self, test, duration):
self._before_event()
if hasattr(self.decorated, "addDuration"):
return self.decorated.addDuration(test, duration)
def _get_failfast(self):
return getattr(self.decorated, "failfast", False)
def _set_failfast(self, value):
self.decorated.failfast = value
failfast = property(_get_failfast, _set_failfast)
class AutoTimingTestResultDecorator(HookedTestResultDecorator):

@@ -222,3 +156,2 @@ """Decorate a TestResult to add time events to a test run.

class TagsMixin:
def __init__(self):

@@ -318,3 +251,2 @@ self._clear_tags()

"""Return a predicate that is true iff all predicates are true."""
# XXX: Should probably be in testtools to be better used by matchers. jml
return lambda *args, **kwargs: all(p(*args, **kwargs) for p in predicates)

@@ -340,8 +272,6 @@

class _PredicateFilter(TestResultDecorator, TagsMixin):
def __init__(self, result, predicate):
super().__init__(result)
self._clear_tags()
self.decorated = TimeCollapsingDecorator(
TagCollapsingDecorator(self.decorated))
self.decorated = TimeCollapsingDecorator(TagCollapsingDecorator(self.decorated))
self._predicate = predicate

@@ -356,9 +286,7 @@ # The current test (for filtering tags)

def filter_predicate(self, test, outcome, error, details):
return self._predicate(
test, outcome, error, details, self._get_active_tags())
return self._predicate(test, outcome, error, details, self._get_active_tags())
def addError(self, test, err=None, details=None):
if (self.filter_predicate(test, 'error', err, details)):
self._buffered_calls.append(
('addError', [test, err], {'details': details}))
if self.filter_predicate(test, "error", err, details):
self._buffered_calls.append(("addError", [test, err], {"details": details}))
else:

@@ -368,5 +296,4 @@ self._filtered()

def addFailure(self, test, err=None, details=None):
if (self.filter_predicate(test, 'failure', err, details)):
self._buffered_calls.append(
('addFailure', [test, err], {'details': details}))
if self.filter_predicate(test, "failure", err, details):
self._buffered_calls.append(("addFailure", [test, err], {"details": details}))
else:

@@ -376,5 +303,4 @@ self._filtered()

def addSkip(self, test, reason=None, details=None):
if (self.filter_predicate(test, 'skip', reason, details)):
self._buffered_calls.append(
('addSkip', [test, reason], {'details': details}))
if self.filter_predicate(test, "skip", reason, details):
self._buffered_calls.append(("addSkip", [test, reason], {"details": details}))
else:

@@ -384,5 +310,4 @@ self._filtered()

def addExpectedFailure(self, test, err=None, details=None):
if self.filter_predicate(test, 'expectedfailure', err, details):
self._buffered_calls.append(
('addExpectedFailure', [test, err], {'details': details}))
if self.filter_predicate(test, "expectedfailure", err, details):
self._buffered_calls.append(("addExpectedFailure", [test, err], {"details": details}))
else:

@@ -392,9 +317,7 @@ self._filtered()

def addUnexpectedSuccess(self, test, details=None):
self._buffered_calls.append(
('addUnexpectedSuccess', [test], {'details': details}))
self._buffered_calls.append(("addUnexpectedSuccess", [test], {"details": details}))
def addSuccess(self, test, details=None):
if (self.filter_predicate(test, 'success', None, details)):
self._buffered_calls.append(
('addSuccess', [test], {'details': details}))
if self.filter_predicate(test, "success", None, details):
self._buffered_calls.append(("addSuccess", [test], {"details": details}))
else:

@@ -415,3 +338,3 @@ self._filtered()

self._current_test_filtered = False
self._buffered_calls.append(('startTest', [test], {}))
self._buffered_calls.append(("startTest", [test], {}))

@@ -436,3 +359,3 @@ def stopTest(self, test):

if self._current_test is not None:
self._buffered_calls.append(('tags', [new_tags, gone_tags], {}))
self._buffered_calls.append(("tags", [new_tags, gone_tags], {}))
else:

@@ -446,3 +369,3 @@ return super().tags(new_tags, gone_tags)

if id.startswith("subunit.RemotedTestCase."):
return id[len("subunit.RemotedTestCase."):]
return id[len("subunit.RemotedTestCase.") :]
return id

@@ -463,6 +386,14 @@

def __init__(self, result, filter_error=False, filter_failure=False,
filter_success=True, filter_skip=False, filter_xfail=False,
filter_predicate=None, fixup_expected_failures=None,
rename=None):
def __init__(
self,
result,
filter_error=False,
filter_failure=False,
filter_success=True,
filter_skip=False,
filter_xfail=False,
filter_predicate=None,
fixup_expected_failures=None,
rename=None,
):
"""Create a FilterResult object filtering to result.

@@ -488,28 +419,23 @@

if filter_error:
predicates.append(
lambda t, outcome, e, d, tags: outcome != 'error')
predicates.append(lambda t, outcome, e, d, tags: outcome != "error")
if filter_failure:
predicates.append(
lambda t, outcome, e, d, tags: outcome != 'failure')
predicates.append(lambda t, outcome, e, d, tags: outcome != "failure")
if filter_success:
predicates.append(
lambda t, outcome, e, d, tags: outcome != 'success')
predicates.append(lambda t, outcome, e, d, tags: outcome != "success")
if filter_skip:
predicates.append(
lambda t, outcome, e, d, tags: outcome != 'skip')
predicates.append(lambda t, outcome, e, d, tags: outcome != "skip")
if filter_xfail:
predicates.append(
lambda t, outcome, e, d, tags: outcome != 'expectedfailure')
predicates.append(lambda t, outcome, e, d, tags: outcome != "expectedfailure")
if filter_predicate is not None:
def compat(test, outcome, error, details, tags):
# 0.0.7 and earlier did not support the 'tags' parameter.
try:
return filter_predicate(
test, outcome, error, details, tags)
return filter_predicate(test, outcome, error, details, tags)
except TypeError:
return filter_predicate(test, outcome, error, details)
predicates.append(compat)
predicate = and_predicates(predicates)
super().__init__(
_PredicateFilter(result, predicate))
super().__init__(_PredicateFilter(result, predicate))
if fixup_expected_failures is None:

@@ -526,4 +452,3 @@ self._fixup_expected_failures = frozenset()

else:
super().addError(
test, err=err, details=details)
super().addError(test, err=err, details=details)

@@ -535,4 +460,3 @@ def addFailure(self, test, err=None, details=None):

else:
super().addFailure(
test, err=err, details=details)
super().addFailure(test, err=err, details=details)

@@ -547,3 +471,3 @@ def addSuccess(self, test, details=None):

def _failure_expected(self, test):
return (test.id() in self._fixup_expected_failures)
return test.id() in self._fixup_expected_failures

@@ -554,3 +478,2 @@ def _apply_renames(self, test):

new_id = self._rename_fn(test.id())
# TODO(jelmer): Isn't there a cleaner way of doing this?
setattr(test, "id", lambda: new_id)

@@ -606,5 +529,5 @@ return test

seconds += duration.microseconds / 1000000.0
self._stream.write(test_id + ' %0.3f\n' % seconds)
self._stream.write(test_id + " %0.3f\n" % seconds)
else:
self._stream.write(test_id + '\n')
self._stream.write(test_id + "\n")

@@ -614,5 +537,15 @@ def startTest(self, test):

def status(self, test_id=None, test_status=None, test_tags=None,
runnable=True, file_name=None, file_bytes=None, eof=False,
mime_type=None, route_code=None, timestamp=None):
def status(
self,
test_id=None,
test_status=None,
test_tags=None,
runnable=True,
file_name=None,
file_bytes=None,
eof=False,
mime_type=None,
route_code=None,
timestamp=None,
):
if not test_id:

@@ -622,6 +555,6 @@ return

self.time(timestamp)
if test_status=='exists':
if test_status == "exists":
if self.show_exists:
self.reportTest(test_id, 0)
elif test_status in ('inprogress', None):
elif test_status in ("inprogress", None):
self._active_tests[test_id] = self._time()

@@ -653,2 +586,10 @@ else:

def addDuration(self, test, duration):
"""Called to add a test duration.
:param test: The test that completed.
:param duration: The duration of the test as a float in seconds.
"""
pass
def stopTestRun(self):

@@ -659,84 +600,3 @@ for test_id in list(self._active_tests.keys()):

class TestByTestResult(testtools.TestResult):
"""Call something every time a test completes."""
# XXX: In testtools since lp:testtools r249. Once that's released, just
# import that.
def __init__(self, on_test):
"""Construct a ``TestByTestResult``.
:param on_test: A callable that take a test case, a status (one of
"success", "failure", "error", "skip", or "xfail"), a start time
(a ``datetime`` with timezone), a stop time, an iterable of tags,
and a details dict. Is called at the end of each test (i.e. on
``stopTest``) with the accumulated values for that test.
"""
super().__init__()
self._on_test = on_test
def startTest(self, test):
super().startTest(test)
self._start_time = self._now()
# There's no supported (i.e. tested) behaviour that relies on these
# being set, but it makes me more comfortable all the same. -- jml
self._status = None
self._details = None
self._stop_time = None
def stopTest(self, test):
self._stop_time = self._now()
super().stopTest(test)
self._on_test(
test=test,
status=self._status,
start_time=self._start_time,
stop_time=self._stop_time,
# current_tags is new in testtools 0.9.13.
tags=getattr(self, 'current_tags', None),
details=self._details)
def _err_to_details(self, test, err, details):
if details:
return details
return {'traceback': TracebackContent(err, test)}
def addSuccess(self, test, details=None):
super().addSuccess(test)
self._status = 'success'
self._details = details
def addFailure(self, test, err=None, details=None):
super().addFailure(test, err, details)
self._status = 'failure'
self._details = self._err_to_details(test, err, details)
def addError(self, test, err=None, details=None):
super().addError(test, err, details)
self._status = 'error'
self._details = self._err_to_details(test, err, details)
def addSkip(self, test, reason=None, details=None):
super().addSkip(test, reason, details)
self._status = 'skip'
if details is None:
details = {'reason': text_content(reason)}
elif reason:
# XXX: What if details already has 'reason' key?
details['reason'] = text_content(reason)
self._details = details
def addExpectedFailure(self, test, err=None, details=None):
super().addExpectedFailure(test, err, details)
self._status = 'xfail'
self._details = self._err_to_details(test, err, details)
def addUnexpectedSuccess(self, test, details=None):
super().addUnexpectedSuccess(test, details)
self._status = 'success'
self._details = details
class CsvResult(TestByTestResult):
def __init__(self, stream):

@@ -751,3 +611,3 @@ super().__init__(self._on_test)

super().startTestRun()
self._write_row(['test', 'status', 'start_time', 'stop_time'])
self._write_row(["test", "status", "start_time", "stop_time"])

@@ -761,7 +621,17 @@

def status(self, test_id=None, test_status=None, test_tags=None,
runnable=True, file_name=None, file_bytes=None, eof=False,
mime_type=None, route_code=None, timestamp=None):
def status(
self,
test_id=None,
test_status=None,
test_tags=None,
runnable=True,
file_name=None,
file_bytes=None,
eof=False,
mime_type=None,
route_code=None,
timestamp=None,
):
if file_name is not None:
self.stream.write(file_bytes)
self.stream.flush()
+17
-9
python/subunit/tests/__init__.py

@@ -29,8 +29,18 @@ #

from subunit.tests import (test_chunked, test_details, test_filter_to_disk, # noqa: E402
test_filters, test_output_filter,
test_progress_model, test_run, test_subunit_filter,
test_subunit_stats, test_subunit_tags,
test_tap2subunit, test_test_protocol,
test_test_protocol2, test_test_results)
from subunit.tests import ( # noqa: E402
test_chunked,
test_details,
test_filter_to_disk,
test_filters,
test_output_filter,
test_progress_model,
test_run,
test_subunit_filter,
test_subunit_stats,
test_subunit_tags,
test_tap2subunit,
test_test_protocol,
test_test_protocol2,
test_test_results,
)

@@ -53,5 +63,3 @@

result.addTest(loader.loadTestsFromModule(test_run))
result.addTests(
generate_scenarios(loader.loadTestsFromModule(test_output_filter))
)
result.addTests(generate_scenarios(loader.loadTestsFromModule(test_output_filter)))
return result
+2
-1
python/subunit/tests/sample-script.py

@@ -7,5 +7,6 @@ #!/usr/bin/env python3

import os
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
if len(sys.argv) == 2:
# subunit.tests.test_test_protocol.TestExecTestCase.test_sample_method_args
# subunit.tests.test_test_protocol.TestExecTestCase.test_sample_method_args
# uses this code path to be sure that the arguments were passed to

@@ -12,0 +13,0 @@ # sample-script.py

+45
-52
python/subunit/tests/test_chunked.py

@@ -21,3 +21,2 @@ #

from testtools.compat import _b

@@ -28,3 +27,2 @@ import subunit.chunked

class TestDecode(unittest.TestCase):
def setUp(self):

@@ -39,50 +37,50 @@ unittest.TestCase.setUp(self)

def test_close_body_short_errors(self):
self.assertEqual(None, self.decoder.write(_b('2\r\na')))
self.assertEqual(None, self.decoder.write(b"2\r\na"))
self.assertRaises(ValueError, self.decoder.close)
def test_close_body_buffered_data_errors(self):
self.assertEqual(None, self.decoder.write(_b('2\r')))
self.assertEqual(None, self.decoder.write(b"2\r"))
self.assertRaises(ValueError, self.decoder.close)
def test_close_after_finished_stream_safe(self):
self.assertEqual(None, self.decoder.write(_b('2\r\nab')))
self.assertEqual(_b(''), self.decoder.write(_b('0\r\n')))
self.assertEqual(None, self.decoder.write(b"2\r\nab"))
self.assertEqual(b"", self.decoder.write(b"0\r\n"))
self.decoder.close()
def test_decode_nothing(self):
self.assertEqual(_b(''), self.decoder.write(_b('0\r\n')))
self.assertEqual(_b(''), self.output.getvalue())
self.assertEqual(b"", self.decoder.write(b"0\r\n"))
self.assertEqual(b"", self.output.getvalue())
def test_decode_serialised_form(self):
self.assertEqual(None, self.decoder.write(_b("F\r\n")))
self.assertEqual(None, self.decoder.write(_b("serialised\n")))
self.assertEqual(_b(''), self.decoder.write(_b("form0\r\n")))
self.assertEqual(None, self.decoder.write(b"F\r\n"))
self.assertEqual(None, self.decoder.write(b"serialised\n"))
self.assertEqual(b"", self.decoder.write(b"form0\r\n"))
def test_decode_short(self):
self.assertEqual(_b(''), self.decoder.write(_b('3\r\nabc0\r\n')))
self.assertEqual(_b('abc'), self.output.getvalue())
self.assertEqual(b"", self.decoder.write(b"3\r\nabc0\r\n"))
self.assertEqual(b"abc", self.output.getvalue())
def test_decode_combines_short(self):
self.assertEqual(_b(''), self.decoder.write(_b('6\r\nabcdef0\r\n')))
self.assertEqual(_b('abcdef'), self.output.getvalue())
self.assertEqual(b"", self.decoder.write(b"6\r\nabcdef0\r\n"))
self.assertEqual(b"abcdef", self.output.getvalue())
def test_decode_excess_bytes_from_write(self):
self.assertEqual(_b('1234'), self.decoder.write(_b('3\r\nabc0\r\n1234')))
self.assertEqual(_b('abc'), self.output.getvalue())
self.assertEqual(b"1234", self.decoder.write(b"3\r\nabc0\r\n1234"))
self.assertEqual(b"abc", self.output.getvalue())
def test_decode_write_after_finished_errors(self):
self.assertEqual(_b('1234'), self.decoder.write(_b('3\r\nabc0\r\n1234')))
self.assertRaises(ValueError, self.decoder.write, _b(''))
self.assertEqual(b"1234", self.decoder.write(b"3\r\nabc0\r\n1234"))
self.assertRaises(ValueError, self.decoder.write, b"")
def test_decode_hex(self):
self.assertEqual(_b(''), self.decoder.write(_b('A\r\n12345678900\r\n')))
self.assertEqual(_b('1234567890'), self.output.getvalue())
self.assertEqual(b"", self.decoder.write(b"A\r\n12345678900\r\n"))
self.assertEqual(b"1234567890", self.output.getvalue())
def test_decode_long_ranges(self):
self.assertEqual(None, self.decoder.write(_b('10000\r\n')))
self.assertEqual(None, self.decoder.write(_b('1' * 65536)))
self.assertEqual(None, self.decoder.write(_b('10000\r\n')))
self.assertEqual(None, self.decoder.write(_b('2' * 65536)))
self.assertEqual(_b(''), self.decoder.write(_b('0\r\n')))
self.assertEqual(_b('1' * 65536 + '2' * 65536), self.output.getvalue())
self.assertEqual(None, self.decoder.write(b"10000\r\n"))
self.assertEqual(None, self.decoder.write(b"1" * 65536))
self.assertEqual(None, self.decoder.write(b"10000\r\n"))
self.assertEqual(None, self.decoder.write(b"2" * 65536))
self.assertEqual(b"", self.decoder.write(b"0\r\n"))
self.assertEqual(b"1" * 65536 + b"2" * 65536, self.output.getvalue())

@@ -93,6 +91,6 @@ def test_decode_newline_nonstrict(self):

self.decoder = subunit.chunked.Decoder(self.output, strict=False)
self.assertEqual(None, self.decoder.write(_b('a\n')))
self.assertEqual(None, self.decoder.write(_b('abcdeabcde')))
self.assertEqual(_b(''), self.decoder.write(_b('0\n')))
self.assertEqual(_b('abcdeabcde'), self.output.getvalue())
self.assertEqual(None, self.decoder.write(b"a\n"))
self.assertEqual(None, self.decoder.write(b"abcdeabcde"))
self.assertEqual(b"", self.decoder.write(b"0\n"))
self.assertEqual(b"abcdeabcde", self.output.getvalue())

@@ -102,16 +100,12 @@ def test_decode_strict_newline_only(self):

# From <http://pad.lv/505078>
self.assertRaises(ValueError,
self.decoder.write, _b('a\n'))
self.assertRaises(ValueError, self.decoder.write, b"a\n")
def test_decode_strict_multiple_crs(self):
self.assertRaises(ValueError,
self.decoder.write, _b('a\r\r\n'))
self.assertRaises(ValueError, self.decoder.write, b"a\r\r\n")
def test_decode_short_header(self):
self.assertRaises(ValueError,
self.decoder.write, _b('\n'))
self.assertRaises(ValueError, self.decoder.write, b"\n")
class TestEncode(unittest.TestCase):
def setUp(self):

@@ -124,30 +118,29 @@ unittest.TestCase.setUp(self)

self.encoder.close()
self.assertEqual(_b('0\r\n'), self.output.getvalue())
self.assertEqual(b"0\r\n", self.output.getvalue())
def test_encode_empty(self):
self.encoder.write(_b(''))
self.encoder.write(b"")
self.encoder.close()
self.assertEqual(_b('0\r\n'), self.output.getvalue())
self.assertEqual(b"0\r\n", self.output.getvalue())
def test_encode_short(self):
self.encoder.write(_b('abc'))
self.encoder.write(b"abc")
self.encoder.close()
self.assertEqual(_b('3\r\nabc0\r\n'), self.output.getvalue())
self.assertEqual(b"3\r\nabc0\r\n", self.output.getvalue())
def test_encode_combines_short(self):
self.encoder.write(_b('abc'))
self.encoder.write(_b('def'))
self.encoder.write(b"abc")
self.encoder.write(b"def")
self.encoder.close()
self.assertEqual(_b('6\r\nabcdef0\r\n'), self.output.getvalue())
self.assertEqual(b"6\r\nabcdef0\r\n", self.output.getvalue())
def test_encode_over_9_is_in_hex(self):
self.encoder.write(_b('1234567890'))
self.encoder.write(b"1234567890")
self.encoder.close()
self.assertEqual(_b('A\r\n12345678900\r\n'), self.output.getvalue())
self.assertEqual(b"A\r\n12345678900\r\n", self.output.getvalue())
def test_encode_long_ranges_not_combined(self):
self.encoder.write(_b('1' * 65536))
self.encoder.write(_b('2' * 65536))
self.encoder.write(b"1" * 65536)
self.encoder.write(b"2" * 65536)
self.encoder.close()
self.assertEqual(_b('10000\r\n' + '1' * 65536 + '10000\r\n' +
'2' * 65536 + '0\r\n'), self.output.getvalue())
self.assertEqual(b"10000\r\n" + b"1" * 65536 + b"10000\r\n" + b"2" * 65536 + b"0\r\n", self.output.getvalue())
+26
-38
python/subunit/tests/test_details.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -20,3 +20,2 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

from testtools.compat import _b

@@ -27,19 +26,18 @@ from subunit import content, content_type, details

class TestSimpleDetails(unittest.TestCase):
def test_lineReceived(self):
parser = details.SimpleDetailsParser(None)
parser.lineReceived(_b("foo\n"))
parser.lineReceived(_b("bar\n"))
self.assertEqual(_b("foo\nbar\n"), parser._message)
parser.lineReceived(b"foo\n")
parser.lineReceived(b"bar\n")
self.assertEqual(b"foo\nbar\n", parser._message)
def test_lineReceived_escaped_bracket(self):
parser = details.SimpleDetailsParser(None)
parser.lineReceived(_b("foo\n"))
parser.lineReceived(_b(" ]are\n"))
parser.lineReceived(_b("bar\n"))
self.assertEqual(_b("foo\n]are\nbar\n"), parser._message)
parser.lineReceived(b"foo\n")
parser.lineReceived(b" ]are\n")
parser.lineReceived(b"bar\n")
self.assertEqual(b"foo\n]are\nbar\n", parser._message)
def test_get_message(self):
parser = details.SimpleDetailsParser(None)
self.assertEqual(_b(""), parser.get_message())
self.assertEqual(b"", parser.get_message())

@@ -49,12 +47,9 @@ def test_get_details(self):

expected = {}
expected['traceback'] = content.Content(
content_type.ContentType("text", "x-traceback",
{'charset': 'utf8'}),
lambda:[_b("")])
expected["traceback"] = content.Content(
content_type.ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b""]
)
found = parser.get_details()
self.assertEqual(expected.keys(), found.keys())
self.assertEqual(expected['traceback'].content_type,
found['traceback'].content_type)
self.assertEqual(_b('').join(expected['traceback'].iter_bytes()),
_b('').join(found['traceback'].iter_bytes()))
self.assertEqual(expected["traceback"].content_type, found["traceback"].content_type)
self.assertEqual(b"".join(expected["traceback"].iter_bytes()), b"".join(found["traceback"].iter_bytes()))

@@ -64,5 +59,3 @@ def test_get_details_skip(self):

expected = {}
expected['reason'] = content.Content(
content_type.ContentType("text", "plain"),
lambda:[_b("")])
expected["reason"] = content.Content(content_type.ContentType("text", "plain"), lambda: [b""])
found = parser.get_details("skip")

@@ -74,5 +67,3 @@ self.assertEqual(expected, found)

expected = {}
expected['message'] = content.Content(
content_type.ContentType("text", "plain"),
lambda:[_b("")])
expected["message"] = content.Content(content_type.ContentType("text", "plain"), lambda: [b""])
found = parser.get_details("success")

@@ -83,3 +74,2 @@ self.assertEqual(expected, found)

class TestMultipartDetails(unittest.TestCase):
def test_get_message_is_None(self):

@@ -95,16 +85,14 @@ parser = details.MultipartDetailsParser(None)

parser = details.MultipartDetailsParser(None)
parser.lineReceived(_b("Content-Type: text/plain\n"))
parser.lineReceived(_b("something\n"))
parser.lineReceived(_b("F\r\n"))
parser.lineReceived(_b("serialised\n"))
parser.lineReceived(_b("form0\r\n"))
parser.lineReceived(b"Content-Type: text/plain\n")
parser.lineReceived(b"something\n")
parser.lineReceived(b"F\r\n")
parser.lineReceived(b"serialised\n")
parser.lineReceived(b"form0\r\n")
expected = {}
expected['something'] = content.Content(
content_type.ContentType("text", "plain"),
lambda:[_b("serialised\nform")])
expected["something"] = content.Content(
content_type.ContentType("text", "plain"), lambda: [b"serialised\nform"]
)
found = parser.get_details()
self.assertEqual(expected.keys(), found.keys())
self.assertEqual(expected['something'].content_type,
found['something'].content_type)
self.assertEqual(_b('').join(expected['something'].iter_bytes()),
_b('').join(found['something'].iter_bytes()))
self.assertEqual(expected["something"].content_type, found["something"].content_type)
self.assertEqual(b"".join(expected["something"].iter_bytes()), b"".join(found["something"].iter_bytes()))
+9
-11
python/subunit/tests/test_filter_to_disk.py

@@ -28,5 +28,4 @@ #

class SmokeTest(TestCase):
def test_smoke(self):
output = os.path.join(self.useFixture(TempDir()).path, 'output')
output = os.path.join(self.useFixture(TempDir()).path, "output")
stdin = io.BytesIO()

@@ -37,14 +36,13 @@ stdout = io.StringIO()

writer.status(
'foo', 'success', {'tag'}, file_name='fred',
file_bytes=b'abcdefg', eof=True, mime_type='text/plain')
"foo", "success", {"tag"}, file_name="fred", file_bytes=b"abcdefg", eof=True, mime_type="text/plain"
)
writer.stopTestRun()
stdin.seek(0)
_to_disk.to_disk(['-d', output], stdin=stdin, stdout=stdout)
_to_disk.to_disk(["-d", output], stdin=stdin, stdout=stdout)
self.expectThat(
os.path.join(output, 'foo/test.json'),
os.path.join(output, "foo/test.json"),
FileContains(
'{"details": ["fred"], "id": "foo", "start": null, '
'"status": "success", "stop": null, "tags": ["tag"]}'))
self.expectThat(
os.path.join(output, 'foo/fred'),
FileContains('abcdefg'))
'{"details": ["fred"], "id": "foo", "start": null, "status": "success", "stop": null, "tags": ["tag"]}'
),
)
self.expectThat(os.path.join(output, "foo/fred"), FileContains("abcdefg"))
+6
-8
python/subunit/tests/test_filters.py

@@ -26,20 +26,18 @@ #

class TestReadTestList(TestCase):
def test_read_list(self):
with NamedTemporaryFile() as f:
f.write(b'foo\nbar\n# comment\nother # comment\n')
f.write(b"foo\nbar\n# comment\nother # comment\n")
f.flush()
self.assertEqual(read_test_list(f.name), ['foo', 'bar', 'other'])
self.assertEqual(read_test_list(f.name), ["foo", "bar", "other"])
class TestFindStream(TestCase):
def test_no_argv(self):
self.assertEqual('foo', find_stream('foo', []))
self.assertEqual("foo", find_stream("foo", []))
def test_opens_file(self):
f = NamedTemporaryFile()
f.write(b'foo')
f.write(b"foo")
f.flush()
stream = find_stream('bar', [f.name])
self.assertEqual(b'foo', stream.read())
stream = find_stream("bar", [f.name])
self.assertEqual(b"foo", stream.read())
+255
-264
python/subunit/tests/test_output_filter.py

@@ -27,9 +27,7 @@ #

from testtools import TestCase
from testtools.matchers import (Equals, Matcher, MatchesAny, MatchesListwise,
Mismatch, raises)
from testtools.matchers import Equals, Matcher, MatchesAny, MatchesListwise, Mismatch, raises
from testtools.testresult.doubles import StreamResult
import subunit._output as _o
from subunit._output import (_ALL_ACTIONS, _FINAL_ACTIONS,
generate_stream_results, parse_arguments)
from subunit._output import _ALL_ACTIONS, _FINAL_ACTIONS, generate_stream_results, parse_arguments

@@ -51,7 +49,4 @@

class TestStatusArgParserTests(TestCase):
scenarios = [(cmd, dict(command=cmd, option="--" + cmd)) for cmd in _ALL_ACTIONS]
scenarios = [
(cmd, dict(command=cmd, option='--' + cmd)) for cmd in _ALL_ACTIONS
]
def test_can_parse_all_commands_with_test_id(self):

@@ -66,5 +61,3 @@ test_id = self.getUniqueString()

with NamedTemporaryFile() as tmp_file:
args = safe_parse_arguments(
args=[self.option, 'foo', '--attach-file', tmp_file.name]
)
args = safe_parse_arguments(args=[self.option, "foo", "--attach-file", tmp_file.name])
self.assertThat(args.attach_file.name, Equals(tmp_file.name))

@@ -75,3 +68,3 @@

args = safe_parse_arguments(
args=[self.option, 'foo', '--attach-file', tmp_file.name, '--mimetype', "text/plain"]
args=[self.option, "foo", "--attach-file", tmp_file.name, "--mimetype", "text/plain"]
)

@@ -82,18 +75,12 @@ self.assertThat(args.mimetype, Equals("text/plain"))

with NamedTemporaryFile() as tmp_file:
args = safe_parse_arguments(
args=[self.option, 'foo', '--attach-file', tmp_file.name, '--file-name', "foo"]
)
args = safe_parse_arguments(args=[self.option, "foo", "--attach-file", tmp_file.name, "--file-name", "foo"])
self.assertThat(args.file_name, Equals("foo"))
def test_all_commands_accept_tags_argument(self):
args = safe_parse_arguments(
args=[self.option, 'foo', '--tag', "foo", "--tag", "bar", "--tag", "baz"]
)
args = safe_parse_arguments(args=[self.option, "foo", "--tag", "foo", "--tag", "bar", "--tag", "baz"])
self.assertThat(args.tags, Equals(["foo", "bar", "baz"]))
def test_attach_file_with_hyphen_opens_stdin(self):
self.patch(_o.sys, 'stdin', TextIOWrapper(BytesIO(b"Hello")))
args = safe_parse_arguments(
args=[self.option, "foo", "--attach-file", "-"]
)
self.patch(_o.sys, "stdin", TextIOWrapper(BytesIO(b"Hello")))
args = safe_parse_arguments(args=[self.option, "foo", "--attach-file", "-"])

@@ -103,5 +90,3 @@ self.assertThat(args.attach_file.read(), Equals(b"Hello"))

def test_attach_file_with_hyphen_sets_filename_to_stdin(self):
args = safe_parse_arguments(
args=[self.option, "foo", "--attach-file", "-"]
)
args = safe_parse_arguments(args=[self.option, "foo", "--attach-file", "-"])

@@ -111,5 +96,3 @@ self.assertThat(args.file_name, Equals("stdin"))

def test_can_override_stdin_filename(self):
args = safe_parse_arguments(
args=[self.option, "foo", "--attach-file", "-", '--file-name', 'foo']
)
args = safe_parse_arguments(args=[self.option, "foo", "--attach-file", "-", "--file-name", "foo"])

@@ -121,15 +104,10 @@ self.assertThat(args.file_name, Equals("foo"))

return safe_parse_arguments(args=[self.option])
self.assertThat(
fn,
raises(RuntimeError('argument %s: must specify a single TEST_ID.' % self.option))
)
self.assertThat(fn, raises(RuntimeError("argument %s: must specify a single TEST_ID." % self.option)))
class ArgParserTests(TestCase):
def test_can_parse_attach_file_without_test_id(self):
with NamedTemporaryFile() as tmp_file:
args = safe_parse_arguments(
args=["--attach-file", tmp_file.name]
)
args = safe_parse_arguments(args=["--attach-file", tmp_file.name])
self.assertThat(args.attach_file.name, Equals(tmp_file.name))

@@ -143,26 +121,20 @@

return safe_parse_arguments(["--fail", "foo", "--skip", "bar"])
self.assertThat(
fn,
raises(RuntimeError('argument --skip: Only one status may be specified at once.'))
)
self.assertThat(fn, raises(RuntimeError("argument --skip: Only one status may be specified at once.")))
def test_cannot_specify_mimetype_without_attach_file(self):
def fn():
return safe_parse_arguments(["--mimetype", "foo"])
self.assertThat(
fn,
raises(RuntimeError('Cannot specify --mimetype without --attach-file'))
)
self.assertThat(fn, raises(RuntimeError("Cannot specify --mimetype without --attach-file")))
def test_cannot_specify_filename_without_attach_file(self):
def fn():
return safe_parse_arguments(["--file-name", "foo"])
self.assertThat(
fn,
raises(RuntimeError('Cannot specify --file-name without --attach-file'))
)
self.assertThat(fn, raises(RuntimeError("Cannot specify --file-name without --attach-file")))
def test_can_specify_tags_without_status_command(self):
args = safe_parse_arguments(['--tag', 'foo'])
self.assertEqual(['foo'], args.tags)
args = safe_parse_arguments(["--tag", "foo"])
self.assertEqual(["foo"], args.tags)

@@ -172,10 +144,12 @@ def test_must_specify_tags_with_tags_options(self):

return safe_parse_arguments(["--fail", "foo", "--tag"])
self.assertThat(
fn,
MatchesAny(
raises(RuntimeError('--tag option requires 1 argument')),
raises(RuntimeError('--tag option requires an argument')),
)
raises(RuntimeError("--tag option requires 1 argument")),
raises(RuntimeError("--tag option requires an argument")),
),
)
def get_result_for(commands):

@@ -205,7 +179,4 @@ """Get a result object from *commands.

class StatusStreamResultTests(TestCase):
scenarios = [(s, dict(status=s, option="--" + s)) for s in _ALL_ACTIONS]
scenarios = [
(s, dict(status=s, option='--' + s)) for s in _ALL_ACTIONS
]
_dummy_timestamp = datetime.datetime(2013, 1, 1, 0, 0, 0, 0, UTC)

@@ -215,3 +186,3 @@

super().setUp()
self.patch(_o, 'create_timestamp', lambda: self._dummy_timestamp)
self.patch(_o, "create_timestamp", lambda: self._dummy_timestamp)
self.test_id = self.getUniqueString()

@@ -223,3 +194,3 @@

len(result._events),
Equals(3) # startTestRun and stopTestRun are also called, making 3 total.
Equals(3), # startTestRun and stopTestRun are also called, making 3 total.
)

@@ -230,13 +201,7 @@

self.assertThat(
result._events[1],
MatchesStatusCall(test_status=self.status)
)
self.assertThat(result._events[1], MatchesStatusCall(test_status=self.status))
def test_all_commands_generate_tags(self):
result = get_result_for([self.option, self.test_id, '--tag', 'hello', '--tag', 'world'])
self.assertThat(
result._events[1],
MatchesStatusCall(test_tags={'hello', 'world'})
)
result = get_result_for([self.option, self.test_id, "--tag", "hello", "--tag", "world"])
self.assertThat(result._events[1], MatchesStatusCall(test_tags={"hello", "world"}))

@@ -246,6 +211,3 @@ def test_all_commands_generate_timestamp(self):

self.assertThat(
result._events[1],
MatchesStatusCall(timestamp=self._dummy_timestamp)
)
self.assertThat(result._events[1], MatchesStatusCall(timestamp=self._dummy_timestamp))

@@ -255,31 +217,32 @@ def test_all_commands_generate_correct_test_id(self):

self.assertThat(
result._events[1],
MatchesStatusCall(test_id=self.test_id)
)
self.assertThat(result._events[1], MatchesStatusCall(test_id=self.test_id))
def test_file_is_sent_in_single_packet(self):
with temp_file_contents(b"Hello") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)
def test_can_read_binary_files(self):
with temp_file_contents(b"\xDE\xAD\xBE\xEF") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
with temp_file_contents(b"\xde\xad\xbe\xef") as f:
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_bytes=b"\xDE\xAD\xBE\xEF", eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_bytes=b"\xde\xad\xbe\xef", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -289,24 +252,28 @@

with temp_file_contents(b"") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_bytes=b"", file_name=f.name, eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_bytes=b"", file_name=f.name, eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)
def test_can_read_stdin(self):
self.patch(_o.sys, 'stdin', TextIOWrapper(BytesIO(b"\xFE\xED\xFA\xCE")))
result = get_result_for([self.option, self.test_id, '--attach-file', '-'])
self.patch(_o.sys, "stdin", TextIOWrapper(BytesIO(b"\xfe\xed\xfa\xce")))
result = get_result_for([self.option, self.test_id, "--attach-file", "-"])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_bytes=b"\xFE\xED\xFA\xCE", file_name='stdin', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_bytes=b"\xfe\xed\xfa\xce", file_name="stdin", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -316,11 +283,13 @@

with temp_file_contents(b"Hello") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -330,11 +299,13 @@

with temp_file_contents(b"Hello") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_status=self.status, file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_status=self.status, file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -344,16 +315,18 @@

with temp_file_contents(b"Hello") as f:
self.patch(_o, '_CHUNK_SIZE', 1)
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
self.patch(_o, "_CHUNK_SIZE", 1)
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'H', eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'e', eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'l', eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'l', eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b'o', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"H", eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"e", eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"l", eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"l", eof=False),
MatchesStatusCall(test_id=self.test_id, file_bytes=b"o", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -363,20 +336,24 @@

with temp_file_contents(b"Hi") as f:
self.patch(_o, '_CHUNK_SIZE', 1)
result = get_result_for([
self.option,
self.test_id,
'--attach-file',
f.name,
'--mimetype',
'text/plain',
])
self.patch(_o, "_CHUNK_SIZE", 1)
result = get_result_for(
[
self.option,
self.test_id,
"--attach-file",
f.name,
"--mimetype",
"text/plain",
]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=self.test_id, mime_type='text/plain', file_bytes=b'H', eof=False),
MatchesStatusCall(test_id=self.test_id, mime_type=None, file_bytes=b'i', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=self.test_id, mime_type="text/plain", file_bytes=b"H", eof=False),
MatchesStatusCall(test_id=self.test_id, mime_type=None, file_bytes=b"i", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -386,22 +363,26 @@

with temp_file_contents(b"Hi") as f:
self.patch(_o, '_CHUNK_SIZE', 1)
result = get_result_for([
self.option,
self.test_id,
'--attach-file',
f.name,
'--tag',
'foo',
'--tag',
'bar',
])
self.patch(_o, "_CHUNK_SIZE", 1)
result = get_result_for(
[
self.option,
self.test_id,
"--attach-file",
f.name,
"--tag",
"foo",
"--tag",
"bar",
]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=self.test_id, test_tags={'foo', 'bar'}),
MatchesStatusCall(test_id=self.test_id, test_tags=None),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=self.test_id, test_tags={"foo", "bar"}),
MatchesStatusCall(test_id=self.test_id, test_tags=None),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -411,18 +392,22 @@

with temp_file_contents(b"Hi") as f:
self.patch(_o, '_CHUNK_SIZE', 1)
result = get_result_for([
self.option,
self.test_id,
'--attach-file',
f.name,
])
self.patch(_o, "_CHUNK_SIZE", 1)
result = get_result_for(
[
self.option,
self.test_id,
"--attach-file",
f.name,
]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=self.test_id, timestamp=self._dummy_timestamp),
MatchesStatusCall(test_id=self.test_id, timestamp=None),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=self.test_id, timestamp=self._dummy_timestamp),
MatchesStatusCall(test_id=self.test_id, timestamp=None),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -432,9 +417,11 @@

with temp_file_contents(b"Hi") as f:
self.patch(_o, '_CHUNK_SIZE', 1)
result = get_result_for([
self.option,
self.test_id,
'--attach-file',
f.name,
])
self.patch(_o, "_CHUNK_SIZE", 1)
result = get_result_for(
[
self.option,
self.test_id,
"--attach-file",
f.name,
]
)

@@ -451,8 +438,10 @@ # 'inprogress' status should be on the first packet only, all other

result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
first_call,
last_call,
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
first_call,
last_call,
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -463,17 +452,15 @@

specified_file_name = self.getUniqueString()
result = get_result_for([
self.option,
self.test_id,
'--attach-file',
f.name,
'--file-name',
specified_file_name])
result = get_result_for(
[self.option, self.test_id, "--attach-file", f.name, "--file-name", specified_file_name]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_name=specified_file_name, file_bytes=b'Hello'),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_name=specified_file_name, file_bytes=b"Hello"),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -483,11 +470,13 @@

with temp_file_contents(b"Hello") as f:
result = get_result_for([self.option, self.test_id, '--attach-file', f.name])
result = get_result_for([self.option, self.test_id, "--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_name=f.name, file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_name=f.name, file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -497,14 +486,15 @@

class FileDataTests(TestCase):
def test_can_attach_file_without_test_id(self):
with temp_file_contents(b"Hello") as f:
result = get_result_for(['--attach-file', f.name])
result = get_result_for(["--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_id=None, file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_id=None, file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -514,11 +504,13 @@

with temp_file_contents(b"Hello") as f:
result = get_result_for(['--attach-file', f.name])
result = get_result_for(["--attach-file", f.name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_name=f.name, file_bytes=b'Hello', eof=True),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_name=f.name, file_bytes=b"Hello", eof=True),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -529,16 +521,13 @@

specified_file_name = self.getUniqueString()
result = get_result_for([
'--attach-file',
f.name,
'--file-name',
specified_file_name
])
result = get_result_for(["--attach-file", f.name, "--file-name", specified_file_name])
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_name=specified_file_name, file_bytes=b'Hello'),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_name=specified_file_name, file_bytes=b"Hello"),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -548,33 +537,41 @@

_dummy_timestamp = datetime.datetime(2013, 1, 1, 0, 0, 0, 0, UTC)
self.patch(_o, 'create_timestamp', lambda: _dummy_timestamp)
self.patch(_o, "create_timestamp", lambda: _dummy_timestamp)
with temp_file_contents(b"Hello") as f:
self.getUniqueString()
result = get_result_for([
'--attach-file',
f.name,
])
result = get_result_for(
[
"--attach-file",
f.name,
]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(file_bytes=b'Hello', timestamp=_dummy_timestamp),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(file_bytes=b"Hello", timestamp=_dummy_timestamp),
MatchesStatusCall(call="stopTestRun"),
]
),
)
def test_can_specify_tags_without_test_status(self):
result = get_result_for([
'--tag',
'foo',
])
result = get_result_for(
[
"--tag",
"foo",
]
)
self.assertThat(
result._events,
MatchesListwise([
MatchesStatusCall(call='startTestRun'),
MatchesStatusCall(test_tags={'foo'}),
MatchesStatusCall(call='stopTestRun'),
])
MatchesListwise(
[
MatchesStatusCall(call="startTestRun"),
MatchesStatusCall(test_tags={"foo"}),
MatchesStatusCall(call="stopTestRun"),
]
),
)

@@ -584,24 +581,20 @@

class MatchesStatusCall(Matcher):
_position_lookup = {
'call': 0,
'test_id': 1,
'test_status': 2,
'test_tags': 3,
'runnable': 4,
'file_name': 5,
'file_bytes': 6,
'eof': 7,
'mime_type': 8,
'route_code': 9,
'timestamp': 10,
"call": 0,
"test_id": 1,
"test_status": 2,
"test_tags": 3,
"runnable": 4,
"file_name": 5,
"file_bytes": 6,
"eof": 7,
"mime_type": 8,
"route_code": 9,
"timestamp": 10,
}
def __init__(self, **kwargs):
unknown_kwargs = list(filter(
lambda k: k not in self._position_lookup,
kwargs
))
unknown_kwargs = list(filter(lambda k: k not in self._position_lookup, kwargs))
if unknown_kwargs:
raise ValueError("Unknown keywords: %s" % ','.join(unknown_kwargs))
raise ValueError("Unknown keywords: %s" % ",".join(unknown_kwargs))
self._filters = kwargs

@@ -614,5 +607,3 @@

if call_tuple[pos] != v:
return Mismatch(
"Value for key is {!r}, not {!r}".format(call_tuple[pos], v)
)
return Mismatch("Value for key is {!r}, not {!r}".format(call_tuple[pos], v))
except IndexError:

@@ -619,0 +610,0 @@ return Mismatch("Key %s is not present." % k)

+1
-2
python/subunit/tests/test_progress_model.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -24,3 +24,2 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

class TestProgressModel(unittest.TestCase):
def assertProgressSummary(self, pos, total, progress):

@@ -27,0 +26,0 @@ """Assert that a progress model has reached a particular point."""

+34
-24
python/subunit/tests/test_run.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -22,3 +22,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

from testtools import PlaceHolder, TestCase
from testtools.compat import _b
from testtools.matchers import StartsWith

@@ -33,7 +33,6 @@ from testtools.testresult.doubles import StreamResult

class TestSubunitTestRunner(TestCase):
def test_includes_timing_output(self):
bytestream = io.BytesIO()
runner = SubunitTestRunner(stream=bytestream)
test = PlaceHolder('name')
test = PlaceHolder("name")
runner.run(test)

@@ -43,4 +42,3 @@ bytestream.seek(0)

subunit.ByteStreamToStreamResult(bytestream).run(eventstream)
timestamps = [event[-1] for event in eventstream._events
if event is not None]
timestamps = [event[-1] for event in eventstream._events if event is not None]
self.assertNotEqual([], timestamps)

@@ -51,4 +49,4 @@

runner = SubunitTestRunner(stream=bytestream)
test1 = PlaceHolder('name1')
test2 = PlaceHolder('name2')
test1 = PlaceHolder("name1")
test2 = PlaceHolder("name2")
case = unittest.TestSuite([test1, test2])

@@ -59,6 +57,9 @@ runner.run(case)

subunit.ByteStreamToStreamResult(bytestream).run(eventstream)
self.assertEqual([
('status', 'name1', 'exists'),
('status', 'name2', 'exists'),
], [event[:3] for event in eventstream._events[:2]])
self.assertEqual(
[
("status", "name1", "exists"),
("status", "name2", "exists"),
],
[event[:3] for event in eventstream._events[:2]],
)

@@ -68,5 +69,7 @@ def test_list_errors_if_errors_from_list_test(self):

runner = SubunitTestRunner(stream=bytestream)
def list_test(test):
return [], ['failed import']
self.patch(run, 'list_test', list_test)
return [], ["failed import"]
self.patch(run, "list_test", list_test)
exc = self.assertRaises(SystemExit, runner.list, None)

@@ -78,8 +81,11 @@ self.assertEqual((2,), exc.args)

runner = SubunitTestRunner(stream=bytestream)
def list_test(test):
return [], []
class Loader:
errors = ['failed import']
errors = ["failed import"]
loader = Loader()
self.patch(run, 'list_test', list_test)
self.patch(run, "list_test", list_test)
exc = self.assertRaises(SystemExit, runner.list, None, loader=loader)

@@ -90,3 +96,3 @@ self.assertEqual((2,), exc.args)

def test_fail(self):
1/0
1 / 0

@@ -97,8 +103,9 @@ def test_exits_zero_when_tests_fail(self):

try:
self.assertEqual(None, run.main(
argv=["progName", "subunit.tests.test_run.TestSubunitTestRunner.FailingTest"],
stdout=stream))
self.assertEqual(
None,
run.main(argv=["progName", "subunit.tests.test_run.TestSubunitTestRunner.FailingTest"], stdout=stream),
)
except SystemExit:
self.fail("SystemExit raised")
self.assertThat(bytestream.getvalue(), StartsWith(_b('\xb3')))
self.assertThat(bytestream.getvalue(), StartsWith(b"\xb3"))

@@ -112,5 +119,8 @@ class ExitingTest(TestCase):

stream = io.TextIOWrapper(bytestream, encoding="utf8")
exc = self.assertRaises(SystemExit, run.main,
argv=["progName", "subunit.tests.test_run.TestSubunitTestRunner.ExitingTest"],
stdout=stream)
exc = self.assertRaises(
SystemExit,
run.main,
argv=["progName", "subunit.tests.test_run.TestSubunitTestRunner.ExitingTest"],
stdout=stream,
)
self.assertEqual(0, exc.args[0])
+98
-135
python/subunit/tests/test_subunit_filter.py

@@ -26,3 +26,3 @@ #

from testtools import TestCase
from testtools.compat import _b
from testtools.testresult.doubles import ExtendedTestResult, StreamResult

@@ -43,3 +43,3 @@

# also has the benefit of detecting any interface skew issues.
example_subunit_stream = _b("""\
example_subunit_stream = b"""\
tags: global

@@ -59,3 +59,3 @@ test passed

xfail todo
""")
"""

@@ -80,42 +80,30 @@ def run_tests(self, result_filter, input_stream=None):

# skips are seen as success by default python TestResult.
self.assertEqual(['error'],
[error[0].id() for error in filtered_result.errors])
self.assertEqual(['failed'],
[failure[0].id() for failure in
filtered_result.failures])
self.assertEqual(["error"], [error[0].id() for error in filtered_result.errors])
self.assertEqual(["failed"], [failure[0].id() for failure in filtered_result.failures])
self.assertEqual(4, filtered_result.testsRun)
def test_tag_filter(self):
tag_filter = make_tag_filter(['global'], ['local'])
tag_filter = make_tag_filter(["global"], ["local"])
result = ExtendedTestResult()
result_filter = TestResultFilter(
result, filter_success=False, filter_predicate=tag_filter)
result_filter = TestResultFilter(result, filter_success=False, filter_predicate=tag_filter)
self.run_tests(result_filter)
tests_included = [
event[1] for event in result._events if event[0] == 'startTest']
tests_expected = list(map(
subunit.RemotedTestCase,
['passed', 'error', 'skipped', 'todo']))
tests_included = [event[1] for event in result._events if event[0] == "startTest"]
tests_expected = list(map(subunit.RemotedTestCase, ["passed", "error", "skipped", "todo"]))
self.assertEqual(tests_expected, tests_included)
def test_tags_tracked_correctly(self):
tag_filter = make_tag_filter(['a'], [])
tag_filter = make_tag_filter(["a"], [])
result = ExtendedTestResult()
result_filter = TestResultFilter(
result, filter_success=False, filter_predicate=tag_filter)
input_stream = _b(
"test: foo\n"
"tags: a\n"
"successful: foo\n"
"test: bar\n"
"successful: bar\n")
result_filter = TestResultFilter(result, filter_success=False, filter_predicate=tag_filter)
input_stream = b"test: foo\ntags: a\nsuccessful: foo\ntest: bar\nsuccessful: bar\n"
self.run_tests(result_filter, input_stream)
foo = subunit.RemotedTestCase('foo')
foo = subunit.RemotedTestCase("foo")
self.assertEqual(
[('startTest', foo),
('tags', {'a'}, set()),
('addSuccess', foo),
('stopTest', foo),
],
result._events
[
("startTest", foo),
("tags", {"a"}, set()),
("addSuccess", foo),
("stopTest", foo),
],
result._events,
)

@@ -129,5 +117,3 @@

self.assertEqual([], filtered_result.errors)
self.assertEqual(['failed'],
[failure[0].id() for failure in
filtered_result.failures])
self.assertEqual(["failed"], [failure[0].id() for failure in filtered_result.failures])
self.assertEqual(3, filtered_result.testsRun)

@@ -137,7 +123,5 @@

filtered_result = unittest.TestResult()
result_filter = TestResultFilter(filtered_result,
fixup_expected_failures={"failed"})
result_filter = TestResultFilter(filtered_result, fixup_expected_failures={"failed"})
self.run_tests(result_filter)
self.assertEqual(['failed', 'todo'],
[failure[0].id() for failure in filtered_result.expectedFailures])
self.assertEqual(["failed", "todo"], [failure[0].id() for failure in filtered_result.expectedFailures])
self.assertEqual([], filtered_result.failures)

@@ -148,7 +132,5 @@ self.assertEqual(4, filtered_result.testsRun)

filtered_result = unittest.TestResult()
result_filter = TestResultFilter(filtered_result,
fixup_expected_failures={"error"})
result_filter = TestResultFilter(filtered_result, fixup_expected_failures={"error"})
self.run_tests(result_filter)
self.assertEqual(['error', 'todo'],
[failure[0].id() for failure in filtered_result.expectedFailures])
self.assertEqual(["error", "todo"], [failure[0].id() for failure in filtered_result.expectedFailures])
self.assertEqual([], filtered_result.errors)

@@ -159,7 +141,5 @@ self.assertEqual(4, filtered_result.testsRun)

filtered_result = unittest.TestResult()
result_filter = TestResultFilter(filtered_result, filter_success=False,
fixup_expected_failures={"passed"})
result_filter = TestResultFilter(filtered_result, filter_success=False, fixup_expected_failures={"passed"})
self.run_tests(result_filter)
self.assertEqual(['passed'],
[passed.id() for passed in filtered_result.unexpectedSuccesses])
self.assertEqual(["passed"], [passed.id() for passed in filtered_result.unexpectedSuccesses])
self.assertEqual(5, filtered_result.testsRun)

@@ -171,7 +151,4 @@

self.run_tests(result_filter)
self.assertEqual(['error'],
[error[0].id() for error in filtered_result.errors])
self.assertEqual([],
[failure[0].id() for failure in
filtered_result.failures])
self.assertEqual(["error"], [error[0].id() for error in filtered_result.errors])
self.assertEqual([], [failure[0].id() for failure in filtered_result.failures])
self.assertEqual(3, filtered_result.testsRun)

@@ -190,10 +167,6 @@

filtered_result = unittest.TestResult()
result_filter = TestResultFilter(filtered_result,
filter_success=False)
result_filter = TestResultFilter(filtered_result, filter_success=False)
self.run_tests(result_filter)
self.assertEqual(['error'],
[error[0].id() for error in filtered_result.errors])
self.assertEqual(['failed'],
[failure[0].id() for failure in
filtered_result.failures])
self.assertEqual(["error"], [error[0].id() for error in filtered_result.errors])
self.assertEqual(["failed"], [failure[0].id() for failure in filtered_result.failures])
self.assertEqual(5, filtered_result.testsRun)

@@ -206,7 +179,7 @@

filtered_result = unittest.TestResult()
def filter_cb(test, outcome, err, details):
return outcome == 'success'
result_filter = TestResultFilter(filtered_result,
filter_predicate=filter_cb,
filter_success=False)
return outcome == "success"
result_filter = TestResultFilter(filtered_result, filter_predicate=filter_cb, filter_success=False)
self.run_tests(result_filter)

@@ -219,7 +192,7 @@ # Only success should pass

filtered_result = unittest.TestResult()
def filter_cb(test, outcome, err, details, tags):
return outcome == 'success'
result_filter = TestResultFilter(filtered_result,
filter_predicate=filter_cb,
filter_success=False)
return outcome == "success"
result_filter = TestResultFilter(filtered_result, filter_predicate=filter_cb, filter_success=False)
self.run_tests(result_filter)

@@ -236,21 +209,21 @@ # Only success should pass

date_c = datetime(year=2000, month=1, day=3, tzinfo=iso8601.UTC)
subunit_stream = _b('\n'.join([
"time: %s",
"test: foo",
"time: %s",
"error: foo",
"time: %s",
""]) % (date_a, date_b, date_c))
subunit_stream = (
"\n".join(["time: %s", "test: foo", "time: %s", "error: foo", "time: %s", ""]) % (date_a, date_b, date_c)
).encode()
result = ExtendedTestResult()
result_filter = TestResultFilter(result)
self.run_tests(result_filter, subunit_stream)
foo = subunit.RemotedTestCase('foo')
foo = subunit.RemotedTestCase("foo")
self.maxDiff = None
self.assertEqual(
[('time', date_a),
('time', date_b),
('startTest', foo),
('addError', foo, {}),
('stopTest', foo),
('time', date_c)], result._events)
[
("time", date_a),
("time", date_b),
("startTest", foo),
("addError", foo, {}),
("stopTest", foo),
("time", date_c),
],
result._events,
)

@@ -263,9 +236,5 @@ def test_time_passes_through_filtered_tests(self):

date_c = datetime(year=2000, month=1, day=3, tzinfo=iso8601.UTC)
subunit_stream = _b('\n'.join([
"time: %s",
"test: foo",
"time: %s",
"success: foo",
"time: %s",
""]) % (date_a, date_b, date_c))
subunit_stream = (
"\n".join(["time: %s", "test: foo", "time: %s", "success: foo", "time: %s", ""]) % (date_a, date_b, date_c)
).encode()
result = ExtendedTestResult()

@@ -276,24 +245,27 @@ result_filter = TestResultFilter(result)

result_filter.stopTestRun()
subunit.RemotedTestCase('foo')
subunit.RemotedTestCase("foo")
self.maxDiff = None
self.assertEqual(
[('startTestRun',),
('time', date_a),
('time', date_c),
('stopTestRun',),], result._events)
[
("startTestRun",),
("time", date_a),
("time", date_c),
("stopTestRun",),
],
result._events,
)
def test_skip_preserved(self):
subunit_stream = _b('\n'.join([
"test: foo",
"skip: foo",
""]))
subunit_stream = "\n".join(["test: foo", "skip: foo", ""]).encode()
result = ExtendedTestResult()
result_filter = TestResultFilter(result)
self.run_tests(result_filter, subunit_stream)
foo = subunit.RemotedTestCase('foo')
foo = subunit.RemotedTestCase("foo")
self.assertEqual(
[('startTest', foo),
('addSkip', foo, {}),
('stopTest', foo), ],
result._events
[
("startTest", foo),
("addSkip", foo, {}),
("stopTest", foo),
],
result._events,
)

@@ -304,14 +276,10 @@

return name + " - renamed"
result = ExtendedTestResult()
result_filter = TestResultFilter(
result, filter_success=False, rename=rename)
input_stream = _b(
"test: foo\n"
"successful: foo\n")
result_filter = TestResultFilter(result, filter_success=False, rename=rename)
input_stream = b"test: foo\nsuccessful: foo\n"
self.run_tests(result_filter, input_stream)
self.assertEqual(
[('startTest', 'foo - renamed'),
('addSuccess', 'foo - renamed'),
('stopTest', 'foo - renamed')],
[(ev[0], ev[1].id()) for ev in result._events]
[("startTest", "foo - renamed"), ("addSuccess", "foo - renamed"), ("stopTest", "foo - renamed")],
[(ev[0], ev[1].id()) for ev in result._events],
)

@@ -321,8 +289,5 @@

class TestFilterCommand(TestCase):
def run_command(self, args, stream):
command = [sys.executable, '-m', 'subunit.filter_scripts.subunit_filter'] + list(args)
ps = subprocess.Popen(
command, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
command = [sys.executable, "-m", "subunit.filter_scripts.subunit_filter"] + list(args)
ps = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = ps.communicate(stream)

@@ -342,6 +307,9 @@ if ps.returncode != 0:

{event[1] for event in events._events}
self.assertEqual([
('status', 'foo', 'inprogress'),
('status', 'foo', 'skip'),
], [event[:3] for event in events._events])
self.assertEqual(
[
("status", "foo", "inprogress"),
("status", "foo", "skip"),
],
[event[:3] for event in events._events],
)

@@ -351,28 +319,23 @@ def test_tags(self):

stream = StreamResultToBytes(byte_stream)
stream.status(
test_id="foo", test_status="inprogress", test_tags={"a"})
stream.status(
test_id="foo", test_status="success", test_tags={"a"})
stream.status(test_id="foo", test_status="inprogress", test_tags={"a"})
stream.status(test_id="foo", test_status="success", test_tags={"a"})
stream.status(test_id="bar", test_status="inprogress")
stream.status(test_id="bar", test_status="inprogress")
stream.status(
test_id="baz", test_status="inprogress", test_tags={"a"})
stream.status(
test_id="baz", test_status="success", test_tags={"a"})
output = self.run_command(
['-s', '--with-tag', 'a'], byte_stream.getvalue())
stream.status(test_id="baz", test_status="inprogress", test_tags={"a"})
stream.status(test_id="baz", test_status="success", test_tags={"a"})
output = self.run_command(["-s", "--with-tag", "a"], byte_stream.getvalue())
events = StreamResult()
ByteStreamToStreamResult(BytesIO(output)).run(events)
ids = {event[1] for event in events._events}
self.assertEqual({'foo', 'baz'}, ids)
self.assertEqual({"foo", "baz"}, ids)
def test_no_passthrough(self):
output = self.run_command(['--no-passthrough'], b'hi thar')
self.assertEqual(b'', output)
output = self.run_command(["--no-passthrough"], b"hi thar")
self.assertEqual(b"", output)
def test_passthrough(self):
output = self.run_command([], b'hi thar')
output = self.run_command([], b"hi thar")
byte_stream = BytesIO()
stream = StreamResultToBytes(byte_stream)
stream.status(file_name="stdout", file_bytes=b'hi thar')
stream.status(file_name="stdout", file_bytes=b"hi thar")
self.assertEqual(byte_stream.getvalue(), output)
+9
-6
python/subunit/tests/test_subunit_stats.py

@@ -22,3 +22,2 @@ #

from testtools.compat import _b

@@ -45,3 +44,4 @@ import subunit

def setUpUsedStream(self):
self.input_stream.write(_b("""tags: global
self.input_stream.write(
b"""tags: global
test passed

@@ -58,6 +58,7 @@ success passed

xfail todo
"""))
"""
)
self.input_stream.seek(0)
self.test.run(self.result)
def test_stats_smoke_everything(self):

@@ -73,3 +74,4 @@ # Statistics are calculated usefully.

def test_stat_formatting(self):
expected = ("""
expected = (
"""
Total tests: 5

@@ -80,5 +82,6 @@ Passed tests: 2

Seen tags: global, local
""")[1:]
"""
)[1:]
self.setUpUsedStream()
self.result.formatStats()
self.assertEqual(expected, self.output.getvalue())
+28
-35
python/subunit/tests/test_subunit_tags.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -30,3 +30,2 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

class TestSubUnitTags(testtools.TestCase):
def setUp(self):

@@ -47,27 +46,26 @@ super().setUp()

reference = [
b'\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86\xb3)'
b'\x83\x1b\x04test\x03\x03bar\x04quux\x03fooqn\xab)',
b'\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86\xb3)'
b'\x83\x1b\x04test\x03\x04quux\x03foo\x03bar\xaf\xbd\x9d\xd6',
b'\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86\xb3)'
b'\x83\x1b\x04test\x03\x04quux\x03bar\x03foo\x03\x04b\r',
b'\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86\xb3)'
b'\x83\x1b\x04test\x03\x03bar\x03foo\x04quux\xd2\x18\x1bC',
b'\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec\xb3)'
b'\x83\x1b\x04test\x03\x03foo\x04quux\x03bar\x08\xc2X\x83',
b'\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec\xb3)'
b'\x83\x1b\x04test\x03\x03bar\x03foo\x04quux\xd2\x18\x1bC',
b'\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec\xb3)'
b'\x83\x1b\x04test\x03\x03foo\x03bar\x04quux:\x05e\x80',
b'\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec\xb3)'
b'\x83\x1b\x04test\x03\x04quux\x03foo\x03bar\xaf\xbd\x9d\xd6',
]
b"\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86"
b"\xb3)\x83\x1b\x04test\x03\x03bar\x04quux\x03fooqn\xab)",
b"\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86"
b"\xb3)\x83\x1b\x04test\x03\x04quux\x03foo\x03bar\xaf\xbd\x9d\xd6",
b"\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86"
b"\xb3)\x83\x1b\x04test\x03\x04quux\x03bar\x03foo\x03\x04b\r",
b"\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86"
b"\xb3)\x83\x1b\x04test\x03\x03bar\x03foo\x04quux\xd2\x18\x1bC",
b"\xb3)\x82\x17\x04test\x02\x04quux\x03foo\x05\x97n\x86"
b"\xb3)\x83\x1b\x04test\x03\x03foo\x04quux\x03bar\x08\xc2X\x83",
b"\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec"
b"\xb3)\x83\x1b\x04test\x03\x03foo\x04quux\x03bar\x08\xc2X\x83",
b"\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec"
b"\xb3)\x83\x1b\x04test\x03\x03bar\x03foo\x04quux\xd2\x18\x1bC",
b"\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec"
b"\xb3)\x83\x1b\x04test\x03\x03foo\x03bar\x04quux:\x05e\x80",
b"\xb3)\x82\x17\x04test\x02\x03foo\x04quux\xa6\xe1\xde\xec"
b"\xb3)\x83\x1b\x04test\x03\x04quux\x03foo\x03bar\xaf\xbd\x9d\xd6",
]
stream = subunit.StreamResultToBytes(self.original)
stream.status(
test_id='test', test_status='inprogress', test_tags={'foo'})
stream.status(
test_id='test', test_status='success', test_tags={'foo', 'bar'})
stream.status(test_id="test", test_status="inprogress", test_tags={"foo"})
stream.status(test_id="test", test_status="success", test_tags={"foo", "bar"})
self.original.seek(0)
self.assertEqual(
0, subunit.tag_stream(self.original, self.filtered, ["quux"]))
self.assertEqual(0, subunit.tag_stream(self.original, self.filtered, ["quux"]))
self.assertThat(reference, Contains(bytes(self.filtered.getvalue())))

@@ -78,14 +76,9 @@

stream = subunit.StreamResultToBytes(reference)
stream.status(
test_id='test', test_status='inprogress', test_tags={'foo'})
stream.status(
test_id='test', test_status='success', test_tags={'foo'})
stream.status(test_id="test", test_status="inprogress", test_tags={"foo"})
stream.status(test_id="test", test_status="success", test_tags={"foo"})
stream = subunit.StreamResultToBytes(self.original)
stream.status(
test_id='test', test_status='inprogress', test_tags={'foo'})
stream.status(
test_id='test', test_status='success', test_tags={'foo', 'bar'})
stream.status(test_id="test", test_status="inprogress", test_tags={"foo"})
stream.status(test_id="test", test_status="success", test_tags={"foo", "bar"})
self.original.seek(0)
self.assertEqual(
0, subunit.tag_stream(self.original, self.filtered, ["-bar"]))
self.assertEqual(0, subunit.tag_stream(self.original, self.filtered, ["-bar"]))
self.assertEqual(reference.getvalue(), self.filtered.getvalue())
+332
-148
python/subunit/tests/test_tap2subunit.py

@@ -9,3 +9,3 @@ #

# compliance with one of these two licences.
#
#
# Unless required by applicable law or agreed to in writing, software

@@ -23,3 +23,3 @@ # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT

from testtools import TestCase
from testtools.compat import _u
from testtools.testresult.doubles import StreamResult

@@ -29,3 +29,3 @@

UTF8_TEXT = 'text/plain; charset=UTF8'
UTF8_TEXT = "text/plain; charset=UTF8"

@@ -52,9 +52,23 @@

# results in a single skipped test.
self.tap.write(_u("1..0 # Skipped: entire file skipped\n"))
self.tap.write("1..0 # Skipped: entire file skipped\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'file skip', 'skip', None, True,
'tap comment', b'Skipped: entire file skipped', True, None, None,
None)])
self.check_events(
[
(
"status",
"file skip",
"skip",
None,
True,
"tap comment",
b"Skipped: entire file skipped",
True,
None,
None,
None,
)
]
)

@@ -67,8 +81,7 @@ def test_ok_test_pass(self):

# stream).
self.tap.write(_u("ok\n"))
self.tap.write("ok\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'success', None, False, None,
None, True, None, None, None)])
self.check_events([("status", "test 1", "success", None, False, None, None, True, None, None, None)])

@@ -79,8 +92,7 @@ def test_ok_test_number_pass(self):

# results in a passed test with name 'test 1'
self.tap.write(_u("ok 1\n"))
self.tap.write("ok 1\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'success', None, False, None,
None, True, None, None, None)])
self.check_events([("status", "test 1", "success", None, False, None, None, True, None, None, None)])

@@ -91,8 +103,9 @@ def test_ok_test_number_description_pass(self):

# results in a passed test with name 'test 1 - There is a description'
self.tap.write(_u("ok 1 - There is a description\n"))
self.tap.write("ok 1 - There is a description\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1 - There is a description',
'success', None, False, None, None, True, None, None, None)])
self.check_events(
[("status", "test 1 - There is a description", "success", None, False, None, None, True, None, None, None)]
)

@@ -103,8 +116,9 @@ def test_ok_test_description_pass(self):

# results in a passed test with name 'test 1 There is a description'
self.tap.write(_u("ok There is a description\n"))
self.tap.write("ok There is a description\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1 There is a description',
'success', None, False, None, None, True, None, None, None)])
self.check_events(
[("status", "test 1 There is a description", "success", None, False, None, None, True, None, None, None)]
)

@@ -115,17 +129,30 @@ def test_ok_SKIP_skip(self):

# results in a skkip test with name 'test 1'
self.tap.write(_u("ok # SKIP\n"))
self.tap.write("ok # SKIP\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'skip', None, False, None,
None, True, None, None, None)])
self.check_events([("status", "test 1", "skip", None, False, None, None, True, None, None, None)])
def test_ok_skip_number_comment_lowercase(self):
self.tap.write(_u("ok 1 # skip no samba environment available, skipping compilation\n"))
self.tap.write("ok 1 # skip no samba environment available, skipping compilation\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'skip', None, False, 'tap comment',
b'no samba environment available, skipping compilation', True,
'text/plain; charset=UTF8', None, None)])
self.check_events(
[
(
"status",
"test 1",
"skip",
None,
False,
"tap comment",
b"no samba environment available, skipping compilation",
True,
"text/plain; charset=UTF8",
None,
None,
)
]
)

@@ -137,9 +164,23 @@ def test_ok_number_description_SKIP_skip_comment(self):

# Not done yet
self.tap.write(_u("ok 1 foo # SKIP Not done yet\n"))
self.tap.write("ok 1 foo # SKIP Not done yet\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1 foo', 'skip', None, False,
'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8',
None, None)])
self.check_events(
[
(
"status",
"test 1 foo",
"skip",
None,
False,
"tap comment",
b"Not done yet",
True,
"text/plain; charset=UTF8",
None,
None,
)
]
)

@@ -150,9 +191,23 @@ def test_ok_SKIP_skip_comment(self):

# results in a skip test with name 'test 1' and a log of Not done yet
self.tap.write(_u("ok # SKIP Not done yet\n"))
self.tap.write("ok # SKIP Not done yet\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'skip', None, False,
'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8',
None, None)])
self.check_events(
[
(
"status",
"test 1",
"skip",
None,
False,
"tap comment",
b"Not done yet",
True,
"text/plain; charset=UTF8",
None,
None,
)
]
)

@@ -163,8 +218,7 @@ def test_ok_TODO_xfail(self):

# results in a xfail test with name 'test 1'
self.tap.write(_u("ok # TODO\n"))
self.tap.write("ok # TODO\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'xfail', None, False, None,
None, True, None, None, None)])
self.check_events([("status", "test 1", "xfail", None, False, None, None, True, None, None, None)])

@@ -175,9 +229,23 @@ def test_ok_TODO_xfail_comment(self):

# results in a xfail test with name 'test 1' and a log of Not done yet
self.tap.write(_u("ok # TODO Not done yet\n"))
self.tap.write("ok # TODO Not done yet\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([('status', 'test 1', 'xfail', None, False,
'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8',
None, None)])
self.check_events(
[
(
"status",
"test 1",
"xfail",
None,
False,
"tap comment",
b"Not done yet",
True,
"text/plain; charset=UTF8",
None,
None,
)
]
)

@@ -188,12 +256,13 @@ def test_bail_out_errors(self):

# is treated as an error
self.tap.write(_u("ok 1 foo\n"))
self.tap.write(_u("Bail out! Lifejacket engaged\n"))
self.tap.write("ok 1 foo\n")
self.tap.write("Bail out! Lifejacket engaged\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 foo', 'success', None, False, None, None, True,
None, None, None),
('status', 'Bail out! Lifejacket engaged', 'fail', None, False,
None, None, True, None, None, None)])
self.check_events(
[
("status", "test 1 foo", "success", None, False, None, None, True, None, None, None),
("status", "Bail out! Lifejacket engaged", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -206,16 +275,27 @@ def test_missing_test_at_end_with_plan_adds_error(self):

# results in three tests, with the third being created
self.tap.write(_u('1..3\n'))
self.tap.write(_u('ok first test\n'))
self.tap.write(_u('not ok second test\n'))
self.tap.write("1..3\n")
self.tap.write("ok first test\n")
self.tap.write("not ok second test\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 first test', 'success', None, False, None,
None, True, None, None, None),
('status', 'test 2 second test', 'fail', None, False, None, None,
True, None, None, None),
('status', 'test 3', 'fail', None, False, 'tap meta',
b'test missing from TAP output', True, 'text/plain; charset=UTF8',
None, None)])
self.check_events(
[
("status", "test 1 first test", "success", None, False, None, None, True, None, None, None),
("status", "test 2 second test", "fail", None, False, None, None, True, None, None, None),
(
"status",
"test 3",
"fail",
None,
False,
"tap meta",
b"test missing from TAP output",
True,
"text/plain; charset=UTF8",
None,
None,
),
]
)

@@ -228,16 +308,27 @@ def test_missing_test_with_plan_adds_error(self):

# results in three tests, with the second being created
self.tap.write(_u('1..3\n'))
self.tap.write(_u('ok first test\n'))
self.tap.write(_u('not ok 3 third test\n'))
self.tap.write("1..3\n")
self.tap.write("ok first test\n")
self.tap.write("not ok 3 third test\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 first test', 'success', None, False, None, None,
True, None, None, None),
('status', 'test 2', 'fail', None, False, 'tap meta',
b'test missing from TAP output', True, 'text/plain; charset=UTF8',
None, None),
('status', 'test 3 third test', 'fail', None, False, None, None,
True, None, None, None)])
self.check_events(
[
("status", "test 1 first test", "success", None, False, None, None, True, None, None, None),
(
"status",
"test 2",
"fail",
None,
False,
"tap meta",
b"test missing from TAP output",
True,
"text/plain; charset=UTF8",
None,
None,
),
("status", "test 3 third test", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -249,15 +340,26 @@ def test_missing_test_no_plan_adds_error(self):

# results in three tests, with the second being created
self.tap.write(_u('ok first test\n'))
self.tap.write(_u('not ok 3 third test\n'))
self.tap.write("ok first test\n")
self.tap.write("not ok 3 third test\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 first test', 'success', None, False, None, None,
True, None, None, None),
('status', 'test 2', 'fail', None, False, 'tap meta',
b'test missing from TAP output', True, 'text/plain; charset=UTF8',
None, None),
('status', 'test 3 third test', 'fail', None, False, None, None,
True, None, None, None)])
self.check_events(
[
("status", "test 1 first test", "success", None, False, None, None, True, None, None, None),
(
"status",
"test 2",
"fail",
None,
False,
"tap meta",
b"test missing from TAP output",
True,
"text/plain; charset=UTF8",
None,
None,
),
("status", "test 3 third test", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -272,19 +374,30 @@ def test_four_tests_in_a_row_trailing_plan(self):

# results in four tests numbered and named
self.tap.write(_u('ok 1 - first test in a script with trailing plan\n'))
self.tap.write(_u('not ok 2 - second\n'))
self.tap.write(_u('ok 3 - third\n'))
self.tap.write(_u('not ok 4 - fourth\n'))
self.tap.write(_u('1..4\n'))
self.tap.write("ok 1 - first test in a script with trailing plan\n")
self.tap.write("not ok 2 - second\n")
self.tap.write("ok 3 - third\n")
self.tap.write("not ok 4 - fourth\n")
self.tap.write("1..4\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 - first test in a script with trailing plan',
'success', None, False, None, None, True, None, None, None),
('status', 'test 2 - second', 'fail', None, False, None, None,
True, None, None, None),
('status', 'test 3 - third', 'success', None, False, None, None,
True, None, None, None),
('status', 'test 4 - fourth', 'fail', None, False, None, None,
True, None, None, None)])
self.check_events(
[
(
"status",
"test 1 - first test in a script with trailing plan",
"success",
None,
False,
None,
None,
True,
None,
None,
None,
),
("status", "test 2 - second", "fail", None, False, None, None, True, None, None, None),
("status", "test 3 - third", "success", None, False, None, None, True, None, None, None),
("status", "test 4 - fourth", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -299,19 +412,30 @@ def test_four_tests_in_a_row_with_plan(self):

# results in four tests numbered and named
self.tap.write(_u('1..4\n'))
self.tap.write(_u('ok 1 - first test in a script with a plan\n'))
self.tap.write(_u('not ok 2 - second\n'))
self.tap.write(_u('ok 3 - third\n'))
self.tap.write(_u('not ok 4 - fourth\n'))
self.tap.write("1..4\n")
self.tap.write("ok 1 - first test in a script with a plan\n")
self.tap.write("not ok 2 - second\n")
self.tap.write("ok 3 - third\n")
self.tap.write("not ok 4 - fourth\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 - first test in a script with a plan',
'success', None, False, None, None, True, None, None, None),
('status', 'test 2 - second', 'fail', None, False, None, None,
True, None, None, None),
('status', 'test 3 - third', 'success', None, False, None, None,
True, None, None, None),
('status', 'test 4 - fourth', 'fail', None, False, None, None,
True, None, None, None)])
self.check_events(
[
(
"status",
"test 1 - first test in a script with a plan",
"success",
None,
False,
None,
None,
True,
None,
None,
None,
),
("status", "test 2 - second", "fail", None, False, None, None, True, None, None, None),
("status", "test 3 - third", "success", None, False, None, None, True, None, None, None),
("status", "test 4 - fourth", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -325,18 +449,29 @@ def test_four_tests_in_a_row_no_plan(self):

# results in four tests numbered and named
self.tap.write(_u('ok 1 - first test in a script with no plan at all\n'))
self.tap.write(_u('not ok 2 - second\n'))
self.tap.write(_u('ok 3 - third\n'))
self.tap.write(_u('not ok 4 - fourth\n'))
self.tap.write("ok 1 - first test in a script with no plan at all\n")
self.tap.write("not ok 2 - second\n")
self.tap.write("ok 3 - third\n")
self.tap.write("not ok 4 - fourth\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1 - first test in a script with no plan at all',
'success', None, False, None, None, True, None, None, None),
('status', 'test 2 - second', 'fail', None, False, None, None,
True, None, None, None),
('status', 'test 3 - third', 'success', None, False, None, None,
True, None, None, None),
('status', 'test 4 - fourth', 'fail', None, False, None, None,
True, None, None, None)])
self.check_events(
[
(
"status",
"test 1 - first test in a script with no plan at all",
"success",
None,
False,
None,
None,
True,
None,
None,
None,
),
("status", "test 2 - second", "fail", None, False, None, None, True, None, None, None),
("status", "test 3 - third", "success", None, False, None, None, True, None, None, None),
("status", "test 4 - fourth", "fail", None, False, None, None, True, None, None, None),
]
)

@@ -348,4 +483,4 @@ def test_todo_and_skip(self):

# results in two tests, numbered and commented.
self.tap.write(_u("not ok 1 - a fail but # TODO but is TODO\n"))
self.tap.write(_u("not ok 2 - another fail # SKIP instead\n"))
self.tap.write("not ok 1 - a fail but # TODO but is TODO\n")
self.tap.write("not ok 2 - another fail # SKIP instead\n")
self.tap.seek(0)

@@ -357,9 +492,32 @@ result = subunit.TAP2SubUnit(self.tap, self.subunit)

subunit.ByteStreamToStreamResult(self.subunit).run(events)
self.check_events([
('status', 'test 1 - a fail but', 'xfail', None, False,
'tap comment', b'but is TODO', True, 'text/plain; charset=UTF8',
None, None),
('status', 'test 2 - another fail', 'skip', None, False,
'tap comment', b'instead', True, 'text/plain; charset=UTF8',
None, None)])
self.check_events(
[
(
"status",
"test 1 - a fail but",
"xfail",
None,
False,
"tap comment",
b"but is TODO",
True,
"text/plain; charset=UTF8",
None,
None,
),
(
"status",
"test 2 - another fail",
"skip",
None,
False,
"tap comment",
b"instead",
True,
"text/plain; charset=UTF8",
None,
None,
),
]
)

@@ -369,18 +527,31 @@ def test_leading_comments_add_to_next_test_log(self):

# # comment
# ok
# ok
# ok
# results in a single test with the comment included
# in the first test and not the second.
self.tap.write(_u("# comment\n"))
self.tap.write(_u("ok\n"))
self.tap.write(_u("ok\n"))
self.tap.write("# comment\n")
self.tap.write("ok\n")
self.tap.write("ok\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1', 'success', None, False, 'tap comment',
b'# comment', True, 'text/plain; charset=UTF8', None, None),
('status', 'test 2', 'success', None, False, None, None, True,
None, None, None)])
self.check_events(
[
(
"status",
"test 1",
"success",
None,
False,
"tap comment",
b"# comment",
True,
"text/plain; charset=UTF8",
None,
None,
),
("status", "test 2", "success", None, False, None, None, True, None, None, None),
]
)
def test_trailing_comments_are_included_in_last_test_log(self):

@@ -393,13 +564,26 @@ # A file

# attached to its log.
self.tap.write(_u("ok\n"))
self.tap.write(_u("ok\n"))
self.tap.write(_u("# comment\n"))
self.tap.write("ok\n")
self.tap.write("ok\n")
self.tap.write("# comment\n")
self.tap.seek(0)
result = subunit.TAP2SubUnit(self.tap, self.subunit)
self.assertEqual(0, result)
self.check_events([
('status', 'test 1', 'success', None, False, None, None, True,
None, None, None),
('status', 'test 2', 'success', None, False, 'tap comment',
b'# comment', True, 'text/plain; charset=UTF8', None, None)])
self.check_events(
[
("status", "test 1", "success", None, False, None, None, True, None, None, None),
(
"status",
"test 2",
"success",
None,
False,
"tap comment",
b"# comment",
True,
"text/plain; charset=UTF8",
None,
None,
),
]
)

@@ -406,0 +590,0 @@ def check_events(self, events):

+639
-725
python/subunit/tests/test_test_protocol.py

@@ -25,16 +25,6 @@ #

from testtools import PlaceHolder, TestCase, TestResult, skipIf
from testtools.compat import _b, _u
from testtools.content import Content, TracebackContent, text_content
from testtools.content_type import ContentType
try:
from testtools.testresult.doubles import (ExtendedTestResult,
Python26TestResult,
Python27TestResult)
except ImportError:
from testtools.tests.helpers import (
Python26TestResult,
Python27TestResult,
ExtendedTestResult,
)
from testtools.testresult.doubles import ExtendedTestResult

@@ -46,6 +36,8 @@ from testtools.matchers import Contains, Equals, MatchesAny

import subunit
from subunit.tests import (_remote_exception_repr,
_remote_exception_repr_chunked,
_remote_exception_str,
_remote_exception_str_chunked)
from subunit.tests import (
_remote_exception_repr,
_remote_exception_repr_chunked,
_remote_exception_str,
_remote_exception_str_chunked,
)

@@ -63,3 +55,3 @@ tb_prelude = "Traceback (most recent call last):\n"

self.addCleanup(os.remove, file_path)
fake_file = os.fdopen(fd, 'r')
fake_file = os.fdopen(fd, "r")
self.assertEqual(fake_file.buffer, subunit._unwrap_text(fake_file))

@@ -70,3 +62,3 @@

self.addCleanup(os.remove, file_path)
fake_file = os.fdopen(fd, 'w')
fake_file = os.fdopen(fd, "w")
self.assertEqual(fake_file.buffer, subunit._unwrap_text(fake_file))

@@ -77,3 +69,3 @@

self.addCleanup(os.remove, file_path)
fake_file = io.FileIO(file_path, 'r')
fake_file = io.FileIO(file_path, "r")
self.assertEqual(fake_file, subunit._unwrap_text(fake_file))

@@ -84,3 +76,3 @@

self.addCleanup(os.remove, file_path)
fake_file = io.FileIO(file_path, 'w')
fake_file = io.FileIO(file_path, "w")
self.assertEqual(fake_file, subunit._unwrap_text(fake_file))

@@ -94,11 +86,16 @@

class TestTestImports(unittest.TestCase):
def test_imports(self):
from subunit import (DiscardStream, ExecTestCase, IsolatedTestCase, # noqa: F401
ProtocolTestCase, RemotedTestCase, RemoteError, # noqa: F401
TestProtocolClient, TestProtocolServer) # noqa: F401
from subunit import ( # noqa: F401
DiscardStream,
ExecTestCase,
IsolatedTestCase,
ProtocolTestCase,
RemotedTestCase,
RemoteError,
TestProtocolClient,
TestProtocolServer,
) # noqa: F401
class TestDiscardStream(unittest.TestCase):
def test_write(self):

@@ -109,3 +106,2 @@ subunit.DiscardStream().write("content")

class TestProtocolServerForward(unittest.TestCase):
def test_story(self):

@@ -115,4 +111,3 @@ client = unittest.TestResult()

protocol = subunit.TestProtocolServer(client, forward_stream=out)
pipe = BytesIO(_b("test old mcdonald\n"
"success old mcdonald\n"))
pipe = BytesIO(b"test old mcdonald\nsuccess old mcdonald\n")
protocol.readFrom(pipe)

@@ -125,12 +120,10 @@ self.assertEqual(client.testsRun, 1)

out = BytesIO()
protocol = subunit.TestProtocolServer(client,
stream=subunit.DiscardStream(), forward_stream=out)
pipe = BytesIO(_b("success old mcdonald\n"))
protocol = subunit.TestProtocolServer(client, stream=subunit.DiscardStream(), forward_stream=out)
pipe = BytesIO(b"success old mcdonald\n")
protocol.readFrom(pipe)
self.assertEqual(client.testsRun, 0)
self.assertEqual(_b(""), out.getvalue())
self.assertEqual(b"", out.getvalue())
class TestTestProtocolServerPipe(unittest.TestCase):
def test_story(self):

@@ -140,10 +133,10 @@ client = unittest.TestResult()

traceback = "foo.c:53:ERROR invalid state\n"
pipe = BytesIO(_b("test old mcdonald\n"
"success old mcdonald\n"
"test bing crosby\n"
"failure bing crosby [\n"
+ traceback +
"]\n"
"test an error\n"
"error an error\n"))
pipe = BytesIO(
b"test old mcdonald\n"
b"success old mcdonald\n"
b"test bing crosby\n"
b"failure bing crosby [\n" + traceback.encode() + b"]\n"
b"test an error\n"
b"error an error\n"
)
protocol.readFrom(pipe)

@@ -153,8 +146,8 @@ bing = subunit.RemotedTestCase("bing crosby")

self.assertEqual(
client.errors, [(an_error, _remote_exception_repr + '\n')],
client.errors,
[(an_error, _remote_exception_repr + "\n")],
)
self.assertEqual(
client.failures,
[(bing, _remote_exception_repr + ": "
+ details_to_str({'traceback': text_content(traceback)}) + "\n")],
[(bing, _remote_exception_repr + ": " + details_to_str({"traceback": text_content(traceback)}) + "\n")],
)

@@ -168,5 +161,4 @@ self.assertEqual(client.testsRun, 3)

class TestTestProtocolServerStartTest(unittest.TestCase):
def setUp(self):
self.client = Python26TestResult()
self.client = ExtendedTestResult()
self.stream = BytesIO()

@@ -176,18 +168,15 @@ self.protocol = subunit.TestProtocolServer(self.client, self.stream)

def test_start_test(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.assertEqual(self.client._events,
[('startTest', subunit.RemotedTestCase("old mcdonald"))])
self.protocol.lineReceived(b"test old mcdonald\n")
self.assertEqual(self.client._events, [("startTest", subunit.RemotedTestCase("old mcdonald"))])
def test_start_testing(self):
self.protocol.lineReceived(_b("testing old mcdonald\n"))
self.assertEqual(self.client._events,
[('startTest', subunit.RemotedTestCase("old mcdonald"))])
self.protocol.lineReceived(b"testing old mcdonald\n")
self.assertEqual(self.client._events, [("startTest", subunit.RemotedTestCase("old mcdonald"))])
def test_start_test_colon(self):
self.protocol.lineReceived(_b("test: old mcdonald\n"))
self.assertEqual(self.client._events,
[('startTest', subunit.RemotedTestCase("old mcdonald"))])
self.protocol.lineReceived(b"test: old mcdonald\n")
self.assertEqual(self.client._events, [("startTest", subunit.RemotedTestCase("old mcdonald"))])
def test_indented_test_colon_ignored(self):
ignored_line = _b(" test: old mcdonald\n")
ignored_line = b" test: old mcdonald\n"
self.protocol.lineReceived(ignored_line)

@@ -198,9 +187,7 @@ self.assertEqual([], self.client._events)

def test_start_testing_colon(self):
self.protocol.lineReceived(_b("testing: old mcdonald\n"))
self.assertEqual(self.client._events,
[('startTest', subunit.RemotedTestCase("old mcdonald"))])
self.protocol.lineReceived(b"testing: old mcdonald\n")
self.assertEqual(self.client._events, [("startTest", subunit.RemotedTestCase("old mcdonald"))])
class TestTestProtocolServerPassThrough(unittest.TestCase):
def setUp(self):

@@ -213,20 +200,15 @@ self.stdout = BytesIO()

def keywords_before_test(self):
self.protocol.lineReceived(_b("failure a\n"))
self.protocol.lineReceived(_b("failure: a\n"))
self.protocol.lineReceived(_b("error a\n"))
self.protocol.lineReceived(_b("error: a\n"))
self.protocol.lineReceived(_b("success a\n"))
self.protocol.lineReceived(_b("success: a\n"))
self.protocol.lineReceived(_b("successful a\n"))
self.protocol.lineReceived(_b("successful: a\n"))
self.protocol.lineReceived(_b("]\n"))
self.assertEqual(self.stdout.getvalue(), _b("failure a\n"
"failure: a\n"
"error a\n"
"error: a\n"
"success a\n"
"success: a\n"
"successful a\n"
"successful: a\n"
"]\n"))
self.protocol.lineReceived(b"failure a\n")
self.protocol.lineReceived(b"failure: a\n")
self.protocol.lineReceived(b"error a\n")
self.protocol.lineReceived(b"error: a\n")
self.protocol.lineReceived(b"success a\n")
self.protocol.lineReceived(b"success: a\n")
self.protocol.lineReceived(b"successful a\n")
self.protocol.lineReceived(b"successful: a\n")
self.protocol.lineReceived(b"]\n")
self.assertEqual(
self.stdout.getvalue(),
b"failure a\nfailure: a\nerror a\nerror: a\nsuccess a\nsuccess: a\nsuccessful a\nsuccessful: a\n]\n",
)

@@ -238,59 +220,74 @@ def test_keywords_before_test(self):

def test_keywords_after_error(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("error old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"error old mcdonald\n")
self.keywords_before_test()
self.assertEqual([
('startTest', self.test),
('addError', self.test, {}),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, {}),
("stopTest", self.test),
],
self.client._events,
)
def test_keywords_after_failure(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("failure old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"failure old mcdonald\n")
self.keywords_before_test()
self.assertEqual(self.client._events, [
('startTest', self.test),
('addFailure', self.test, {}),
('stopTest', self.test),
])
self.assertEqual(
self.client._events,
[
("startTest", self.test),
("addFailure", self.test, {}),
("stopTest", self.test),
],
)
def test_keywords_after_success(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("success old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"success old mcdonald\n")
self.keywords_before_test()
self.assertEqual([
('startTest', self.test),
('addSuccess', self.test),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addSuccess", self.test),
("stopTest", self.test),
],
self.client._events,
)
def test_keywords_after_test(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("failure a\n"))
self.protocol.lineReceived(_b("failure: a\n"))
self.protocol.lineReceived(_b("error a\n"))
self.protocol.lineReceived(_b("error: a\n"))
self.protocol.lineReceived(_b("success a\n"))
self.protocol.lineReceived(_b("success: a\n"))
self.protocol.lineReceived(_b("successful a\n"))
self.protocol.lineReceived(_b("successful: a\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(_b("failure old mcdonald\n"))
self.assertEqual(self.stdout.getvalue(), _b("test old mcdonald\n"
"failure a\n"
"failure: a\n"
"error a\n"
"error: a\n"
"success a\n"
"success: a\n"
"successful a\n"
"successful: a\n"
"]\n"))
self.assertEqual(self.client._events, [
('startTest', self.test),
('addFailure', self.test, {}),
('stopTest', self.test),
])
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"failure a\n")
self.protocol.lineReceived(b"failure: a\n")
self.protocol.lineReceived(b"error a\n")
self.protocol.lineReceived(b"error: a\n")
self.protocol.lineReceived(b"success a\n")
self.protocol.lineReceived(b"success: a\n")
self.protocol.lineReceived(b"successful a\n")
self.protocol.lineReceived(b"successful: a\n")
self.protocol.lineReceived(b"]\n")
self.protocol.lineReceived(b"failure old mcdonald\n")
self.assertEqual(
self.stdout.getvalue(),
b"test old mcdonald\n"
b"failure a\n"
b"failure: a\n"
b"error a\n"
b"error: a\n"
b"success a\n"
b"success: a\n"
b"successful a\n"
b"successful: a\n"
b"]\n",
)
self.assertEqual(
self.client._events,
[
("startTest", self.test),
("addFailure", self.test, {}),
("stopTest", self.test),
],
)

@@ -300,35 +297,40 @@ def test_keywords_during_failure(self):

# appropriately.
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("failure: old mcdonald [\n"))
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("failure a\n"))
self.protocol.lineReceived(_b("failure: a\n"))
self.protocol.lineReceived(_b("error a\n"))
self.protocol.lineReceived(_b("error: a\n"))
self.protocol.lineReceived(_b("success a\n"))
self.protocol.lineReceived(_b("success: a\n"))
self.protocol.lineReceived(_b("successful a\n"))
self.protocol.lineReceived(_b("successful: a\n"))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.assertEqual(self.stdout.getvalue(), _b(""))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"failure: old mcdonald [\n")
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"failure a\n")
self.protocol.lineReceived(b"failure: a\n")
self.protocol.lineReceived(b"error a\n")
self.protocol.lineReceived(b"error: a\n")
self.protocol.lineReceived(b"success a\n")
self.protocol.lineReceived(b"success: a\n")
self.protocol.lineReceived(b"successful a\n")
self.protocol.lineReceived(b"successful: a\n")
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.assertEqual(self.stdout.getvalue(), b"")
details = {}
details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}),
lambda:[_b(
"test old mcdonald\n"
"failure a\n"
"failure: a\n"
"error a\n"
"error: a\n"
"success a\n"
"success: a\n"
"successful a\n"
"successful: a\n"
"]\n")])
self.assertEqual(self.client._events, [
('startTest', self.test),
('addFailure', self.test, details),
('stopTest', self.test),
])
details["traceback"] = Content(
ContentType("text", "x-traceback", {"charset": "utf8"}),
lambda: [
b"test old mcdonald\n"
b"failure a\n"
b"failure: a\n"
b"error a\n"
b"error: a\n"
b"success a\n"
b"success: a\n"
b"successful a\n"
b"successful: a\n"
b"]\n"
],
)
self.assertEqual(
self.client._events,
[
("startTest", self.test),
("addFailure", self.test, details),
("stopTest", self.test),
],
)

@@ -339,3 +341,3 @@ def test_stdout_passthrough(self):

"""
bytes = _b("randombytes\n")
bytes = b"randombytes\n"
self.protocol.lineReceived(bytes)

@@ -346,5 +348,4 @@ self.assertEqual(self.stdout.getvalue(), bytes)

class TestTestProtocolServerLostConnection(unittest.TestCase):
def setUp(self):
self.client = Python26TestResult()
self.client = ExtendedTestResult()
self.protocol = subunit.TestProtocolServer(self.client)

@@ -358,34 +359,40 @@ self.test = subunit.RemotedTestCase("old mcdonald")

def test_lost_connection_after_start(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lostConnection()
failure = subunit.RemoteError(
_u("lost connection during test 'old mcdonald'"))
self.assertEqual([
('startTest', self.test),
('addError', self.test, failure),
('stopTest', self.test),
], self.client._events)
failure = subunit.RemoteError("lost connection during test 'old mcdonald'")
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, failure),
("stopTest", self.test),
],
self.client._events,
)
def test_lost_connected_after_error(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("error old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"error old mcdonald\n")
self.protocol.lostConnection()
self.assertEqual([
('startTest', self.test),
('addError', self.test, subunit.RemoteError(_u(""))),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, {}),
("stopTest", self.test),
],
self.client._events,
)
def do_connection_lost(self, outcome, opening):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("{} old mcdonald {}".format(outcome, opening)))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived("{} old mcdonald {}".format(outcome, opening).encode())
self.protocol.lostConnection()
failure = subunit.RemoteError(
_u("lost connection during %s report of test 'old mcdonald'") %
outcome)
self.assertEqual([
('startTest', self.test),
('addError', self.test, failure),
('stopTest', self.test),
], self.client._events)
failure = subunit.RemoteError("lost connection during %s report of test 'old mcdonald'" % outcome)
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, failure),
("stopTest", self.test),
],
self.client._events,
)

@@ -399,10 +406,13 @@ def test_lost_connection_during_error(self):

def test_lost_connected_after_failure(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("failure old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"failure old mcdonald\n")
self.protocol.lostConnection()
self.assertEqual([
('startTest', self.test),
('addFailure', self.test, subunit.RemoteError(_u(""))),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addFailure", self.test, {}),
("stopTest", self.test),
],
self.client._events,
)

@@ -416,10 +426,13 @@ def test_lost_connection_during_failure(self):

def test_lost_connection_after_success(self):
self.protocol.lineReceived(_b("test old mcdonald\n"))
self.protocol.lineReceived(_b("success old mcdonald\n"))
self.protocol.lineReceived(b"test old mcdonald\n")
self.protocol.lineReceived(b"success old mcdonald\n")
self.protocol.lostConnection()
self.assertEqual([
('startTest', self.test),
('addSuccess', self.test),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addSuccess", self.test),
("stopTest", self.test),
],
self.client._events,
)

@@ -452,37 +465,34 @@ def test_lost_connection_during_success(self):

class TestInTestMultipart(unittest.TestCase):
def setUp(self):
self.client = ExtendedTestResult()
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.test = subunit.RemotedTestCase(_u("mcdonalds farm"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = subunit.RemotedTestCase("mcdonalds farm")
def test__outcome_sets_details_parser(self):
self.protocol._reading_success_details.details_parser = None
self.protocol._state._outcome(0, _b("mcdonalds farm [ multipart\n"),
None, self.protocol._reading_success_details)
self.protocol._state._outcome(0, b"mcdonalds farm [ multipart\n", None, self.protocol._reading_success_details)
parser = self.protocol._reading_success_details.details_parser
self.assertNotEqual(None, parser)
self.assertIsInstance(
parser,
subunit.details.MultipartDetailsParser
)
self.assertIsInstance(parser, subunit.details.MultipartDetailsParser)
class TestTestProtocolServerAddError(unittest.TestCase):
def setUp(self):
self.client = ExtendedTestResult()
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = subunit.RemotedTestCase("mcdonalds farm")
def simple_error_keyword(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
details = {}
self.assertEqual([
('startTest', self.test),
('addError', self.test, details),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, details),
("stopTest", self.test),
],
self.client._events,
)

@@ -496,25 +506,29 @@ def test_simple_error(self):

def test_error_empty_message(self):
self.protocol.lineReceived(_b("error mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(b"error mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}), lambda:[_b("")])
self.assertEqual([
('startTest', self.test),
('addError', self.test, details),
('stopTest', self.test),
], self.client._events)
details["traceback"] = Content(ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b""])
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, details),
("stopTest", self.test),
],
self.client._events,
)
def error_quoted_bracket(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(("%s mcdonalds farm [\n" % keyword).encode())
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}), lambda:[_b("]\n")])
self.assertEqual([
('startTest', self.test),
('addError', self.test, details),
('stopTest', self.test),
], self.client._events)
details["traceback"] = Content(ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b"]\n"])
self.assertEqual(
[
("startTest", self.test),
("addError", self.test, details),
("stopTest", self.test),
],
self.client._events,
)

@@ -529,18 +543,20 @@ def test_error_quoted_bracket(self):

class TestTestProtocolServerAddFailure(unittest.TestCase):
def setUp(self):
self.client = ExtendedTestResult()
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = subunit.RemotedTestCase("mcdonalds farm")
def assertFailure(self, details):
self.assertEqual([
('startTest', self.test),
('addFailure', self.test, details),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addFailure", self.test, details),
("stopTest", self.test),
],
self.client._events,
)
def simple_failure_keyword(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
details = {}

@@ -556,16 +572,14 @@ self.assertFailure(details)

def test_failure_empty_message(self):
self.protocol.lineReceived(_b("failure mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(b"failure mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}), lambda:[_b("")])
details["traceback"] = Content(ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b""])
self.assertFailure(details)
def failure_quoted_bracket(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(("%s mcdonalds farm [\n" % keyword).encode())
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}), lambda:[_b("]\n")])
details["traceback"] = Content(ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b"]\n"])
self.assertFailure(details)

@@ -581,114 +595,52 @@

class TestTestProtocolServerAddxFail(unittest.TestCase):
"""Tests for the xfail keyword.
"""Tests for the xfail keyword."""
In Python this can thunk through to Success due to stdlib limitations (see
README).
"""
def capture_expected_failure(self, test, err):
self._events.append((test, err))
def setup_python26(self):
"""Setup a test object ready to be xfailed and thunk to success."""
self.client = Python26TestResult()
self.setup_protocol()
def setup_python27(self):
"""Setup a test object ready to be xfailed."""
self.client = Python27TestResult()
self.setup_protocol()
def setup_python_ex(self):
def setUp(self):
"""Setup a test object ready to be xfailed with details."""
self.client = ExtendedTestResult()
self.setup_protocol()
def setup_protocol(self):
"""Setup the protocol based on self.client."""
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = self.client._events[-1][-1]
def simple_xfail_keyword(self, keyword, as_success):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.check_success_or_xfail(as_success)
def simple_xfail_keyword(self, keyword):
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
self.check_xfail()
def check_success_or_xfail(self, as_success, error_message=None):
if as_success:
self.assertEqual([
('startTest', self.test),
('addSuccess', self.test),
('stopTest', self.test),
], self.client._events)
else:
details = {}
if error_message is not None:
details['traceback'] = Content(
ContentType("text", "x-traceback", {'charset': 'utf8'}),
lambda:[_b(error_message)])
if isinstance(self.client, ExtendedTestResult):
value = details
else:
if error_message is not None:
value = subunit.RemoteError(details_to_str(details))
else:
value = subunit.RemoteError()
self.assertEqual([
('startTest', self.test),
('addExpectedFailure', self.test, value),
('stopTest', self.test),
], self.client._events)
def check_xfail(self, error_message=None):
details = {}
if error_message is not None:
details["traceback"] = Content(
ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [error_message.encode()]
)
self.assertEqual(
[
("startTest", self.test),
("addExpectedFailure", self.test, details),
("stopTest", self.test),
],
self.client._events,
)
def test_simple_xfail(self):
self.setup_python26()
self.simple_xfail_keyword("xfail", True)
self.setup_python27()
self.simple_xfail_keyword("xfail", False)
self.setup_python_ex()
self.simple_xfail_keyword("xfail", False)
self.simple_xfail_keyword("xfail")
def test_simple_xfail_colon(self):
self.setup_python26()
self.simple_xfail_keyword("xfail:", True)
self.setup_python27()
self.simple_xfail_keyword("xfail:", False)
self.setup_python_ex()
self.simple_xfail_keyword("xfail:", False)
self.simple_xfail_keyword("xfail:")
def test_xfail_empty_message(self):
self.setup_python26()
self.empty_message(True)
self.setup_python27()
self.empty_message(False)
self.setup_python_ex()
self.empty_message(False, error_message="")
self.protocol.lineReceived(b"xfail mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
self.check_xfail(error_message="")
def empty_message(self, as_success, error_message="\n"):
self.protocol.lineReceived(_b("xfail mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.check_success_or_xfail(as_success, error_message)
def xfail_quoted_bracket(self, keyword, as_success):
# This tests it is accepted, but cannot test it is used today, because
# of not having a way to expose it in Python so far.
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.check_success_or_xfail(as_success, "]\n")
def test_xfail_quoted_bracket(self):
self.setup_python26()
self.xfail_quoted_bracket("xfail", True)
self.setup_python27()
self.xfail_quoted_bracket("xfail", False)
self.setup_python_ex()
self.xfail_quoted_bracket("xfail", False)
self.protocol.lineReceived(b"xfail mcdonalds farm [\n")
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.check_xfail("]\n")
def test_xfail_colon_quoted_bracket(self):
self.setup_python26()
self.xfail_quoted_bracket("xfail:", True)
self.setup_python27()
self.xfail_quoted_bracket("xfail:", False)
self.setup_python_ex()
self.xfail_quoted_bracket("xfail:", False)
self.protocol.lineReceived(b"xfail: mcdonalds farm [\n")
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.check_xfail("]\n")

@@ -699,112 +651,55 @@

def capture_expected_failure(self, test, err):
self._events.append((test, err))
def setup_python26(self):
"""Setup a test object ready to be xfailed and thunk to success."""
self.client = Python26TestResult()
self.setup_protocol()
def setup_python27(self):
"""Setup a test object ready to be xfailed."""
self.client = Python27TestResult()
self.setup_protocol()
def setup_python_ex(self):
"""Setup a test object ready to be xfailed with details."""
def setUp(self):
"""Setup a test object ready for uxsuccess with details."""
super().setUp()
self.client = ExtendedTestResult()
self.setup_protocol()
def setup_protocol(self):
"""Setup the protocol based on self.client."""
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = self.client._events[-1][-1]
def simple_uxsuccess_keyword(self, keyword, as_fail):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.check_fail_or_uxsuccess(as_fail)
def simple_uxsuccess_keyword(self, keyword):
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
self.check_uxsuccess()
def check_fail_or_uxsuccess(self, as_fail, error_message=None):
details = {}
def check_uxsuccess(self, error_message=None):
if error_message is not None:
details['traceback'] = Content(
ContentType("text", "x-traceback", {'charset': 'utf8'}),
lambda:[_b(error_message)])
if isinstance(self.client, ExtendedTestResult):
value = details
details = {}
details["traceback"] = Content(
ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [error_message.encode()]
)
expected_events = [
("startTest", self.test),
("addUnexpectedSuccess", self.test, details),
("stopTest", self.test),
]
else:
value = None
if as_fail:
self.client._events[1] = self.client._events[1][:2]
# The value is generated within the extended to original decorator:
# todo use the testtools matcher to check on this.
self.assertEqual([
('startTest', self.test),
('addFailure', self.test),
('stopTest', self.test),
], self.client._events)
elif value:
self.assertEqual([
('startTest', self.test),
('addUnexpectedSuccess', self.test, value),
('stopTest', self.test),
], self.client._events)
else:
self.assertEqual([
('startTest', self.test),
('addUnexpectedSuccess', self.test),
('stopTest', self.test),
], self.client._events)
expected_events = [
("startTest", self.test),
("addUnexpectedSuccess", self.test),
("stopTest", self.test),
]
self.assertEqual(expected_events, self.client._events)
def test_simple_uxsuccess(self):
self.setup_python26()
self.simple_uxsuccess_keyword("uxsuccess", True)
self.setup_python27()
self.simple_uxsuccess_keyword("uxsuccess", False)
self.setup_python_ex()
self.simple_uxsuccess_keyword("uxsuccess", False)
self.simple_uxsuccess_keyword("uxsuccess")
def test_simple_uxsuccess_colon(self):
self.setup_python26()
self.simple_uxsuccess_keyword("uxsuccess:", True)
self.setup_python27()
self.simple_uxsuccess_keyword("uxsuccess:", False)
self.setup_python_ex()
self.simple_uxsuccess_keyword("uxsuccess:", False)
self.simple_uxsuccess_keyword("uxsuccess:")
def test_uxsuccess_empty_message(self):
self.setup_python26()
self.empty_message(True)
self.setup_python27()
self.empty_message(False)
self.setup_python_ex()
self.empty_message(False, error_message="")
self.protocol.lineReceived(b"uxsuccess mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
self.check_uxsuccess(error_message="")
def empty_message(self, as_fail, error_message="\n"):
self.protocol.lineReceived(_b("uxsuccess mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.check_fail_or_uxsuccess(as_fail, error_message)
def uxsuccess_quoted_bracket(self, keyword, as_fail):
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.check_fail_or_uxsuccess(as_fail, "]\n")
def test_uxsuccess_quoted_bracket(self):
self.setup_python26()
self.uxsuccess_quoted_bracket("uxsuccess", True)
self.setup_python27()
self.uxsuccess_quoted_bracket("uxsuccess", False)
self.setup_python_ex()
self.uxsuccess_quoted_bracket("uxsuccess", False)
self.protocol.lineReceived(b"uxsuccess mcdonalds farm [\n")
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.check_uxsuccess("]\n")
def test_uxsuccess_colon_quoted_bracket(self):
self.setup_python26()
self.uxsuccess_quoted_bracket("uxsuccess:", True)
self.setup_python27()
self.uxsuccess_quoted_bracket("uxsuccess:", False)
self.setup_python_ex()
self.uxsuccess_quoted_bracket("uxsuccess:", False)
self.protocol.lineReceived(b"uxsuccess: mcdonalds farm [\n")
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.check_uxsuccess("]\n")

@@ -823,3 +718,3 @@

self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = self.client._events[-1][-1]

@@ -830,12 +725,14 @@

if reason is not None:
details['reason'] = Content(
ContentType("text", "plain"), lambda:[reason])
self.assertEqual([
('startTest', self.test),
('addSkip', self.test, details),
('stopTest', self.test),
], self.client._events)
details["reason"] = Content(ContentType("text", "plain"), lambda: [reason])
self.assertEqual(
[
("startTest", self.test),
("addSkip", self.test, details),
("stopTest", self.test),
],
self.client._events,
)
def simple_skip_keyword(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
self.assertSkip(None)

@@ -850,5 +747,5 @@

def test_skip_empty_message(self):
self.protocol.lineReceived(_b("skip mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.assertSkip(_b(""))
self.protocol.lineReceived(b"skip mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
self.assertSkip(b"")

@@ -858,6 +755,6 @@ def skip_quoted_bracket(self, keyword):

# of not having a way to expose it in Python so far.
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.assertSkip(_b("]\n"))
self.protocol.lineReceived(("%s mcdonalds farm [\n" % keyword).encode())
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
self.assertSkip(b"]\n")

@@ -872,16 +769,18 @@ def test_skip_quoted_bracket(self):

class TestTestProtocolServerAddSuccess(unittest.TestCase):
def setUp(self):
self.client = ExtendedTestResult()
self.protocol = subunit.TestProtocolServer(self.client)
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
self.test = subunit.RemotedTestCase("mcdonalds farm")
def simple_success_keyword(self, keyword):
self.protocol.lineReceived(_b("%s mcdonalds farm\n" % keyword))
self.assertEqual([
('startTest', self.test),
('addSuccess', self.test),
('stopTest', self.test),
], self.client._events)
self.protocol.lineReceived(("%s mcdonalds farm\n" % keyword).encode())
self.assertEqual(
[
("startTest", self.test),
("addSuccess", self.test),
("stopTest", self.test),
],
self.client._events,
)

@@ -895,14 +794,16 @@ def test_simple_success(self):

def assertSuccess(self, details):
self.assertEqual([
('startTest', self.test),
('addSuccess', self.test, details),
('stopTest', self.test),
], self.client._events)
self.assertEqual(
[
("startTest", self.test),
("addSuccess", self.test, details),
("stopTest", self.test),
],
self.client._events,
)
def test_success_empty_message(self):
self.protocol.lineReceived(_b("success mcdonalds farm [\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(b"success mcdonalds farm [\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['message'] = Content(ContentType("text", "plain"),
lambda:[_b("")])
details["message"] = Content(ContentType("text", "plain"), lambda: [b""])
self.assertSuccess(details)

@@ -913,8 +814,7 @@

# of not having a way to expose it in Python so far.
self.protocol.lineReceived(_b("%s mcdonalds farm [\n" % keyword))
self.protocol.lineReceived(_b(" ]\n"))
self.protocol.lineReceived(_b("]\n"))
self.protocol.lineReceived(("%s mcdonalds farm [\n" % keyword).encode())
self.protocol.lineReceived(b" ]\n")
self.protocol.lineReceived(b"]\n")
details = {}
details['message'] = Content(ContentType("text", "plain"),
lambda:[_b("]\n")])
details["message"] = Content(ContentType("text", "plain"), lambda: [b"]\n"])
self.assertSuccess(details)

@@ -933,10 +833,9 @@

def test_progress_accepted_stdlib(self):
self.result = Python26TestResult()
self.result = ExtendedTestResult()
self.stream = BytesIO()
self.protocol = subunit.TestProtocolServer(self.result,
stream=self.stream)
self.protocol.lineReceived(_b("progress: 23"))
self.protocol.lineReceived(_b("progress: -2"))
self.protocol.lineReceived(_b("progress: +4"))
self.assertEqual(_b(""), self.stream.getvalue())
self.protocol = subunit.TestProtocolServer(self.result, stream=self.stream)
self.protocol.lineReceived(b"progress: 23")
self.protocol.lineReceived(b"progress: -2")
self.protocol.lineReceived(b"progress: +4")
self.assertEqual(b"", self.stream.getvalue())

@@ -947,17 +846,19 @@ def test_progress_accepted_extended(self):

self.stream = BytesIO()
self.protocol = subunit.TestProtocolServer(self.result,
stream=self.stream)
self.protocol.lineReceived(_b("progress: 23"))
self.protocol.lineReceived(_b("progress: push"))
self.protocol.lineReceived(_b("progress: -2"))
self.protocol.lineReceived(_b("progress: pop"))
self.protocol.lineReceived(_b("progress: +4"))
self.assertEqual(_b(""), self.stream.getvalue())
self.assertEqual([
('progress', 23, subunit.PROGRESS_SET),
('progress', None, subunit.PROGRESS_PUSH),
('progress', -2, subunit.PROGRESS_CUR),
('progress', None, subunit.PROGRESS_POP),
('progress', 4, subunit.PROGRESS_CUR),
], self.result._events)
self.protocol = subunit.TestProtocolServer(self.result, stream=self.stream)
self.protocol.lineReceived(b"progress: 23")
self.protocol.lineReceived(b"progress: push")
self.protocol.lineReceived(b"progress: -2")
self.protocol.lineReceived(b"progress: pop")
self.protocol.lineReceived(b"progress: +4")
self.assertEqual(b"", self.stream.getvalue())
self.assertEqual(
[
("progress", 23, subunit.PROGRESS_SET),
("progress", None, subunit.PROGRESS_PUSH),
("progress", -2, subunit.PROGRESS_CUR),
("progress", None, subunit.PROGRESS_POP),
("progress", 4, subunit.PROGRESS_CUR),
],
self.result._events,
)

@@ -973,30 +874,36 @@

def test_initial_tags(self):
self.protocol.lineReceived(_b("tags: foo bar:baz quux\n"))
self.assertEqual([
('tags', {"foo", "bar:baz", "quux"}, set()),
], self.client._events)
self.protocol.lineReceived(b"tags: foo bar:baz quux\n")
self.assertEqual(
[
("tags", {"foo", "bar:baz", "quux"}, set()),
],
self.client._events,
)
def test_minus_removes_tags(self):
self.protocol.lineReceived(_b("tags: -bar quux\n"))
self.assertEqual([
('tags', {"quux"}, {"bar"}),
], self.client._events)
self.protocol.lineReceived(b"tags: -bar quux\n")
self.assertEqual(
[
("tags", {"quux"}, {"bar"}),
],
self.client._events,
)
def test_tags_do_not_get_set_on_test(self):
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
test = self.client._events[0][-1]
self.assertEqual(None, getattr(test, 'tags', None))
self.assertEqual(None, getattr(test, "tags", None))
def test_tags_do_not_get_set_on_global_tags(self):
self.protocol.lineReceived(_b("tags: foo bar\n"))
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"tags: foo bar\n")
self.protocol.lineReceived(b"test mcdonalds farm\n")
test = self.client._events[-1][-1]
self.assertEqual(None, getattr(test, 'tags', None))
self.assertEqual(None, getattr(test, "tags", None))
def test_tags_get_set_on_test_tags(self):
self.protocol.lineReceived(_b("test mcdonalds farm\n"))
self.protocol.lineReceived(b"test mcdonalds farm\n")
test = self.client._events[-1][-1]
self.protocol.lineReceived(_b("tags: foo bar\n"))
self.protocol.lineReceived(_b("success mcdonalds farm\n"))
self.assertEqual(None, getattr(test, 'tags', None))
self.protocol.lineReceived(b"tags: foo bar\n")
self.protocol.lineReceived(b"success mcdonalds farm\n")
self.assertEqual(None, getattr(test, "tags", None))

@@ -1008,8 +915,7 @@

def test_time_accepted_stdlib(self):
self.result = Python26TestResult()
self.result = ExtendedTestResult()
self.stream = BytesIO()
self.protocol = subunit.TestProtocolServer(self.result,
stream=self.stream)
self.protocol.lineReceived(_b("time: 2001-12-12 12:59:59Z\n"))
self.assertEqual(_b(""), self.stream.getvalue())
self.protocol = subunit.TestProtocolServer(self.result, stream=self.stream)
self.protocol.lineReceived(b"time: 2001-12-12 12:59:59Z\n")
self.assertEqual(b"", self.stream.getvalue())

@@ -1019,14 +925,9 @@ def test_time_accepted_extended(self):

self.stream = BytesIO()
self.protocol = subunit.TestProtocolServer(self.result,
stream=self.stream)
self.protocol.lineReceived(_b("time: 2001-12-12 12:59:59Z\n"))
self.assertEqual(_b(""), self.stream.getvalue())
self.assertEqual([
('time', datetime.datetime(2001, 12, 12, 12, 59, 59, 0,
iso8601.UTC))
], self.result._events)
self.protocol = subunit.TestProtocolServer(self.result, stream=self.stream)
self.protocol.lineReceived(b"time: 2001-12-12 12:59:59Z\n")
self.assertEqual(b"", self.stream.getvalue())
self.assertEqual([("time", datetime.datetime(2001, 12, 12, 12, 59, 59, 0, iso8601.UTC))], self.result._events)
class TestRemotedTestCase(unittest.TestCase):
def test_simple(self):

@@ -1036,14 +937,9 @@ test = subunit.RemotedTestCase("A test description")

self.assertRaises(NotImplementedError, test.tearDown)
self.assertEqual("A test description",
test.shortDescription())
self.assertEqual("A test description",
test.id())
self.assertEqual("A test description", test.shortDescription())
self.assertEqual("A test description", test.id())
self.assertEqual("A test description (subunit.RemotedTestCase)", "%s" % test)
self.assertEqual("<subunit.RemotedTestCase description="
"'A test description'>", "%r" % test)
self.assertEqual("<subunit.RemotedTestCase description='A test description'>", "%r" % test)
result = unittest.TestResult()
test.run(result)
self.assertEqual([(test, _remote_exception_repr + ': ' +
"Cannot run RemotedTestCases.\n\n")],
result.errors)
self.assertEqual([(test, _remote_exception_repr + ": " + "Cannot run RemotedTestCases.\n\n")], result.errors)
self.assertEqual(1, result.testsRun)

@@ -1058,7 +954,6 @@ another_test = subunit.RemotedTestCase("A test description")

class TestRemoteError(unittest.TestCase):
def test_eq(self):
error = subunit.RemoteError(_u("Something went wrong"))
another_error = subunit.RemoteError(_u("Something went wrong"))
different_error = subunit.RemoteError(_u("boo!"))
error = subunit.RemoteError("Something went wrong")
another_error = subunit.RemoteError("Something went wrong")
different_error = subunit.RemoteError("boo!")
self.assertEqual(error, another_error)

@@ -1069,9 +964,7 @@ self.assertNotEqual(error, different_error)

def test_empty_constructor(self):
self.assertEqual(subunit.RemoteError(), subunit.RemoteError(_u("")))
self.assertEqual(subunit.RemoteError(), subunit.RemoteError(""))
class TestExecTestCase(unittest.TestCase):
class SampleExecTestCase(subunit.ExecTestCase):
def test_sample_method(self):

@@ -1088,4 +981,3 @@ """sample-script.py"""

test = self.SampleExecTestCase("test_sample_method")
self.assertEqual(test.script,
subunit.join_dir(__file__, 'sample-script.py'))
self.assertEqual(test.script, subunit.join_dir(__file__, "sample-script.py"))

@@ -1105,17 +997,21 @@ def test_args(self):

bing_details = {}
bing_details['traceback'] = Content(ContentType("text", "x-traceback",
{'charset': 'utf8'}), lambda:[_b("foo.c:53:ERROR invalid state\n")])
bing_details["traceback"] = Content(
ContentType("text", "x-traceback", {"charset": "utf8"}), lambda: [b"foo.c:53:ERROR invalid state\n"]
)
an_error = subunit.RemotedTestCase("an error")
error_details = {}
self.assertEqual([
('startTest', mcdonald),
('addSuccess', mcdonald),
('stopTest', mcdonald),
('startTest', bing),
('addFailure', bing, bing_details),
('stopTest', bing),
('startTest', an_error),
('addError', an_error, error_details),
('stopTest', an_error),
], result._events)
self.assertEqual(
[
("startTest", mcdonald),
("addSuccess", mcdonald),
("stopTest", mcdonald),
("startTest", bing),
("addFailure", bing, bing_details),
("stopTest", bing),
("startTest", an_error),
("addError", an_error, error_details),
("stopTest", an_error),
],
result._events,
)

@@ -1127,8 +1023,8 @@ def test_debug(self):

def test_count_test_cases(self):
"""TODO run the child process and count responses to determine the count."""
"""Run the child process and count responses to determine the count."""
def test_join_dir(self):
sibling = subunit.join_dir(__file__, 'foo')
sibling = subunit.join_dir(__file__, "foo")
filedir = os.path.abspath(os.path.dirname(__file__))
expected = os.path.join(filedir, 'foo')
expected = os.path.join(filedir, "foo")
self.assertEqual(sibling, expected)

@@ -1138,3 +1034,2 @@

class DoExecTestCase(subunit.ExecTestCase):
def test_working_script(self):

@@ -1145,5 +1040,3 @@ """sample-two-script.py"""

class TestIsolatedTestCase(TestCase):
class SampleIsolatedTestCase(subunit.IsolatedTestCase):
SETUP = False

@@ -1162,3 +1055,2 @@ TEARDOWN = False

def test_construct(self):

@@ -1179,10 +1071,8 @@ self.SampleIsolatedTestCase("test_sets_global_state")

pass
#test = self.SampleExecTestCase("test_sample_method")
#test.debug()
# test = self.SampleExecTestCase("test_sample_method")
# test.debug()
class TestIsolatedTestSuite(TestCase):
class SampleTestToIsolate(unittest.TestCase):
SETUP = False

@@ -1201,3 +1091,2 @@ TEARDOWN = False

def test_construct(self):

@@ -1228,3 +1117,2 @@ subunit.IsolatedTestSuite()

class TestTestProtocolClient(TestCase):
def setUp(self):

@@ -1234,9 +1122,7 @@ super().setUp()

self.protocol = subunit.TestProtocolClient(self.io)
self.unicode_test = PlaceHolder(_u('\u2603'))
self.unicode_test = PlaceHolder("\u2603")
self.test = TestTestProtocolClient("test_start_test")
self.sample_details = {'something':Content(
ContentType('text', 'plain'), lambda:[_b('serialised\nform')])}
self.sample_details = {"something": Content(ContentType("text", "plain"), lambda: [b"serialised\nform"])}
self.sample_tb_details = dict(self.sample_details)
self.sample_tb_details['traceback'] = TracebackContent(
subunit.RemoteError(_u("boo qux")), self.test)
self.sample_tb_details["traceback"] = TracebackContent(subunit.RemoteError("boo qux"), self.test)

@@ -1246,3 +1132,3 @@ def test_start_test(self):

self.protocol.startTest(self.test)
self.assertEqual(self.io.getvalue(), _b("test: %s\n" % self.test.id()))
self.assertEqual(self.io.getvalue(), ("test: %s\n" % self.test.id()).encode())

@@ -1252,3 +1138,3 @@ def test_start_test_unicode_id(self):

self.protocol.startTest(self.unicode_test)
expected = _b("test: ") + _u('\u2603').encode('utf8') + _b("\n")
expected = b"test: " + "\u2603".encode("utf8") + b"\n"
self.assertEqual(expected, self.io.getvalue())

@@ -1259,3 +1145,3 @@

self.protocol.stopTest(self.test)
self.assertEqual(self.io.getvalue(), _b(""))
self.assertEqual(self.io.getvalue(), b"")

@@ -1265,4 +1151,3 @@ def test_add_success(self):

self.protocol.addSuccess(self.test)
self.assertEqual(
self.io.getvalue(), _b("successful: %s\n" % self.test.id()))
self.assertEqual(self.io.getvalue(), ("successful: %s\n" % self.test.id()).encode())

@@ -1272,3 +1157,3 @@ def test_add_outcome_unicode_id(self):

self.protocol.addSuccess(self.unicode_test)
expected = _b("successful: ") + _u('\u2603').encode('utf8') + _b("\n")
expected = b"successful: " + "\u2603".encode("utf8") + b"\n"
self.assertEqual(expected, self.io.getvalue())

@@ -1280,175 +1165,203 @@

self.assertEqual(
self.io.getvalue(), _b("successful: %s [ multipart\n"
self.io.getvalue(),
(
"successful: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n]\n" % self.test.id()))
"F\r\nserialised\nform0\r\n]\n" % self.test.id()
).encode(),
)
def test_add_failure(self):
"""Test addFailure on a TestProtocolClient."""
self.protocol.addFailure(
self.test, subunit.RemoteError(_u("boo qux")))
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
'failure: %s [\n' +
_remote_exception_str + ': boo qux\n' +
']\n') % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
'failure: %s [\n' +
_remote_exception_repr + ': boo qux\n' +
']\n') % self.test.id()))))
self.protocol.addFailure(self.test, subunit.RemoteError("boo qux"))
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals((("failure: %s [\n" + _remote_exception_str + ": boo qux\n" + "]\n") % self.test.id()).encode()),
# testtools >= 2.5.0
Equals(
(("failure: %s [\n" + _remote_exception_repr + ": boo qux\n" + "]\n") % self.test.id()).encode()
),
),
)
def test_add_failure_details(self):
"""Test addFailure on a TestProtocolClient with details."""
self.protocol.addFailure(
self.test, details=self.sample_tb_details)
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
"failure: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked +
"]\n") % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
"failure: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked +
"]\n") % self.test.id()))))
self.protocol.addFailure(self.test, details=self.sample_tb_details)
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals(
(
(
"failure: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked + "]\n"
)
% self.test.id()
).encode()
),
# testtools >= 2.5.0
Equals(
(
(
"failure: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked + "]\n"
)
% self.test.id()
).encode()
),
),
)
def test_add_error(self):
"""Test stopTest on a TestProtocolClient."""
self.protocol.addError(
self.test, subunit.RemoteError(_u("phwoar crikey")))
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
'error: %s [\n' +
_remote_exception_str + ": phwoar crikey\n"
"]\n") % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
'error: %s [\n' +
_remote_exception_repr + ": phwoar crikey\n"
"]\n") % self.test.id()))))
self.protocol.addError(self.test, subunit.RemoteError("phwoar crikey"))
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals((("error: %s [\n" + _remote_exception_str + ": phwoar crikey\n]\n") % self.test.id()).encode()),
# testtools >= 2.5.0
Equals((("error: %s [\n" + _remote_exception_repr + ": phwoar crikey\n]\n") % self.test.id()).encode()),
),
)
def test_add_error_details(self):
"""Test stopTest on a TestProtocolClient with details."""
self.protocol.addError(
self.test, details=self.sample_tb_details)
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
"error: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked +
"]\n") % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
"error: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked +
"]\n") % self.test.id()))))
self.protocol.addError(self.test, details=self.sample_tb_details)
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals(
(
(
"error: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked + "]\n"
)
% self.test.id()
).encode()
),
# testtools >= 2.5.0
Equals(
(
(
"error: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked + "]\n"
)
% self.test.id()
).encode()
),
),
)
def test_add_expected_failure(self):
"""Test addExpectedFailure on a TestProtocolClient."""
self.protocol.addExpectedFailure(
self.test, subunit.RemoteError(_u("phwoar crikey")))
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
'xfail: %s [\n' +
_remote_exception_str + ": phwoar crikey\n"
"]\n") % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
'xfail: %s [\n' +
_remote_exception_repr + ": phwoar crikey\n"
"]\n") % self.test.id()))))
self.protocol.addExpectedFailure(self.test, subunit.RemoteError("phwoar crikey"))
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals((("xfail: %s [\n" + _remote_exception_str + ": phwoar crikey\n]\n") % self.test.id()).encode()),
# testtools >= 2.5.0
Equals((("xfail: %s [\n" + _remote_exception_repr + ": phwoar crikey\n]\n") % self.test.id()).encode()),
),
)
def test_add_expected_failure_details(self):
"""Test addExpectedFailure on a TestProtocolClient with details."""
self.protocol.addExpectedFailure(
self.test, details=self.sample_tb_details)
self.assertThat(self.io.getvalue(), MatchesAny(
# testtools < 2.5.0
Equals(_b((
"xfail: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked +
"]\n") % self.test.id())),
# testtools >= 2.5.0
Equals(_b((
"xfail: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked +
"]\n") % self.test.id()))))
self.protocol.addExpectedFailure(self.test, details=self.sample_tb_details)
self.assertThat(
self.io.getvalue(),
MatchesAny(
# testtools < 2.5.0
Equals(
(
(
"xfail: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_str_chunked + "]\n"
)
% self.test.id()
).encode()
),
# testtools >= 2.5.0
Equals(
(
(
"xfail: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n"
"Content-Type: text/x-traceback;charset=utf8,language=python\n"
"traceback\n" + _remote_exception_repr_chunked + "]\n"
)
% self.test.id()
).encode()
),
),
)
def test_add_skip(self):
"""Test addSkip on a TestProtocolClient."""
self.protocol.addSkip(
self.test, "Has it really?")
self.assertEqual(
self.io.getvalue(),
_b('skip: %s [\nHas it really?\n]\n' % self.test.id()))
self.protocol.addSkip(self.test, "Has it really?")
self.assertEqual(self.io.getvalue(), ("skip: %s [\nHas it really?\n]\n" % self.test.id()).encode())
def test_add_skip_details(self):
"""Test addSkip on a TestProtocolClient with details."""
details = {'reason':Content(
ContentType('text', 'plain'), lambda:[_b('Has it really?')])}
details = {"reason": Content(ContentType("text", "plain"), lambda: [b"Has it really?"])}
self.protocol.addSkip(self.test, details=details)
self.assertEqual(
self.io.getvalue(),
_b("skip: %s [ multipart\n"
"Content-Type: text/plain\n"
"reason\n"
"E\r\nHas it really?0\r\n"
"]\n" % self.test.id()))
(
"skip: %s [ multipart\nContent-Type: text/plain\nreason\nE\r\nHas it really?0\r\n]\n" % self.test.id()
).encode(),
)
def test_progress_set(self):
self.protocol.progress(23, subunit.PROGRESS_SET)
self.assertEqual(self.io.getvalue(), _b('progress: 23\n'))
self.assertEqual(self.io.getvalue(), b"progress: 23\n")
def test_progress_neg_cur(self):
self.protocol.progress(-23, subunit.PROGRESS_CUR)
self.assertEqual(self.io.getvalue(), _b('progress: -23\n'))
self.assertEqual(self.io.getvalue(), b"progress: -23\n")
def test_progress_pos_cur(self):
self.protocol.progress(23, subunit.PROGRESS_CUR)
self.assertEqual(self.io.getvalue(), _b('progress: +23\n'))
self.assertEqual(self.io.getvalue(), b"progress: +23\n")
def test_progress_pop(self):
self.protocol.progress(1234, subunit.PROGRESS_POP)
self.assertEqual(self.io.getvalue(), _b('progress: pop\n'))
self.assertEqual(self.io.getvalue(), b"progress: pop\n")
def test_progress_push(self):
self.protocol.progress(1234, subunit.PROGRESS_PUSH)
self.assertEqual(self.io.getvalue(), _b('progress: push\n'))
self.assertEqual(self.io.getvalue(), b"progress: push\n")
def test_time(self):
# Calling time() outputs a time signal immediately.
self.protocol.time(
datetime.datetime(2009,10,11,12,13,14,15, iso8601.UTC))
self.assertEqual(
_b("time: 2009-10-11 12:13:14.000015Z\n"),
self.io.getvalue())
self.protocol.time(datetime.datetime(2009, 10, 11, 12, 13, 14, 15, iso8601.UTC))
self.assertEqual(b"time: 2009-10-11 12:13:14.000015Z\n", self.io.getvalue())

@@ -1458,4 +1371,3 @@ def test_add_unexpected_success(self):

self.protocol.addUnexpectedSuccess(self.test)
self.assertEqual(
self.io.getvalue(), _b("uxsuccess: %s\n" % self.test.id()))
self.assertEqual(self.io.getvalue(), ("uxsuccess: %s\n" % self.test.id()).encode())

@@ -1466,23 +1378,25 @@ def test_add_unexpected_success_details(self):

self.assertEqual(
self.io.getvalue(), _b("uxsuccess: %s [ multipart\n"
self.io.getvalue(),
(
"uxsuccess: %s [ multipart\n"
"Content-Type: text/plain\n"
"something\n"
"F\r\nserialised\nform0\r\n]\n" % self.test.id()))
"F\r\nserialised\nform0\r\n]\n" % self.test.id()
).encode(),
)
def test_tags_empty(self):
self.protocol.tags(set(), set())
self.assertEqual(_b(""), self.io.getvalue())
self.assertEqual(b"", self.io.getvalue())
def test_tags_add(self):
self.protocol.tags({'foo'}, set())
self.assertEqual(_b("tags: foo\n"), self.io.getvalue())
self.protocol.tags({"foo"}, set())
self.assertEqual(b"tags: foo\n", self.io.getvalue())
def test_tags_both(self):
self.protocol.tags({'quux'}, {'bar'})
self.assertThat(
[b"tags: quux -bar\n", b"tags: -bar quux\n"],
Contains(self.io.getvalue()))
self.protocol.tags({"quux"}, {"bar"})
self.assertThat([b"tags: quux -bar\n", b"tags: -bar quux\n"], Contains(self.io.getvalue()))
def test_tags_gone(self):
self.protocol.tags(set(), {'bar'})
self.assertEqual(_b("tags: -bar\n"), self.io.getvalue())
self.protocol.tags(set(), {"bar"})
self.assertEqual(b"tags: -bar\n", self.io.getvalue())
+298
-198
python/subunit/tests/test_test_protocol2.py

@@ -22,2 +22,3 @@ #

from hypothesis import given
# To debug hypothesis

@@ -38,19 +39,19 @@ # from hypothesis import Settings, Verbosity

CONSTANT_ENUM = b'\xb3)\x01\x0c\x03foo\x08U_\x1b'
CONSTANT_INPROGRESS = b'\xb3)\x02\x0c\x03foo\x8e\xc1-\xb5'
CONSTANT_SUCCESS = b'\xb3)\x03\x0c\x03fooE\x9d\xfe\x10'
CONSTANT_UXSUCCESS = b'\xb3)\x04\x0c\x03fooX\x98\xce\xa8'
CONSTANT_SKIP = b'\xb3)\x05\x0c\x03foo\x93\xc4\x1d\r'
CONSTANT_FAIL = b'\xb3)\x06\x0c\x03foo\x15Po\xa3'
CONSTANT_XFAIL = b'\xb3)\x07\x0c\x03foo\xde\x0c\xbc\x06'
CONSTANT_EOF = b'\xb3!\x10\x08S\x15\x88\xdc'
CONSTANT_FILE_CONTENT = b'\xb3!@\x13\x06barney\x03wooA5\xe3\x8c'
CONSTANT_MIME = b'\xb3! #\x1aapplication/foo; charset=1x3Q\x15'
CONSTANT_TIMESTAMP = b'\xb3+\x03\x13<\x17T\xcf\x80\xaf\xc8\x03barI\x96>-'
CONSTANT_ROUTE_CODE = b'\xb3-\x03\x13\x03bar\x06source\x9cY9\x19'
CONSTANT_RUNNABLE = b'\xb3(\x03\x0c\x03foo\xe3\xea\xf5\xa4'
CONSTANT_ENUM = b"\xb3)\x01\x0c\x03foo\x08U_\x1b"
CONSTANT_INPROGRESS = b"\xb3)\x02\x0c\x03foo\x8e\xc1-\xb5"
CONSTANT_SUCCESS = b"\xb3)\x03\x0c\x03fooE\x9d\xfe\x10"
CONSTANT_UXSUCCESS = b"\xb3)\x04\x0c\x03fooX\x98\xce\xa8"
CONSTANT_SKIP = b"\xb3)\x05\x0c\x03foo\x93\xc4\x1d\r"
CONSTANT_FAIL = b"\xb3)\x06\x0c\x03foo\x15Po\xa3"
CONSTANT_XFAIL = b"\xb3)\x07\x0c\x03foo\xde\x0c\xbc\x06"
CONSTANT_EOF = b"\xb3!\x10\x08S\x15\x88\xdc"
CONSTANT_FILE_CONTENT = b"\xb3!@\x13\x06barney\x03wooA5\xe3\x8c"
CONSTANT_MIME = b"\xb3! #\x1aapplication/foo; charset=1x3Q\x15"
CONSTANT_TIMESTAMP = b"\xb3+\x03\x13<\x17T\xcf\x80\xaf\xc8\x03barI\x96>-"
CONSTANT_ROUTE_CODE = b"\xb3-\x03\x13\x03bar\x06source\x9cY9\x19"
CONSTANT_RUNNABLE = b"\xb3(\x03\x0c\x03foo\xe3\xea\xf5\xa4"
CONSTANT_TAGS = [
b'\xb3)\x80\x15\x03bar\x02\x03foo\x03barTHn\xb4',
b'\xb3)\x80\x15\x03bar\x02\x03bar\x03foo\xf8\xf1\x91o',
]
b"\xb3)\x80\x15\x03bar\x02\x03foo\x03barTHn\xb4",
b"\xb3)\x80\x15\x03bar\x02\x03bar\x03foo\xf8\xf1\x91o",
]

@@ -66,3 +67,2 @@

class TestStreamResultToBytes(TestCase):
def _make_result(self):

@@ -78,24 +78,24 @@ output = BytesIO()

result._write_number(0, packet)
self.assertEqual([b'\x00'], packet)
self.assertEqual([b"\x00"], packet)
del packet[:]
result._write_number(63, packet)
self.assertEqual([b'\x3f'], packet)
self.assertEqual([b"\x3f"], packet)
del packet[:]
result._write_number(64, packet)
self.assertEqual([b'\x40\x40'], packet)
self.assertEqual([b"\x40\x40"], packet)
del packet[:]
result._write_number(16383, packet)
self.assertEqual([b'\x7f\xff'], packet)
self.assertEqual([b"\x7f\xff"], packet)
del packet[:]
result._write_number(16384, packet)
self.assertEqual([b'\x80\x40', b'\x00'], packet)
self.assertEqual([b"\x80\x40", b"\x00"], packet)
del packet[:]
result._write_number(4194303, packet)
self.assertEqual([b'\xbf\xff', b'\xff'], packet)
self.assertEqual([b"\xbf\xff", b"\xff"], packet)
del packet[:]
result._write_number(4194304, packet)
self.assertEqual([b'\xc0\x40\x00\x00'], packet)
self.assertEqual([b"\xc0\x40\x00\x00"], packet)
del packet[:]
result._write_number(1073741823, packet)
self.assertEqual([b'\xff\xff\xff\xff'], packet)
self.assertEqual([b"\xff\xff\xff\xff"], packet)
del packet[:]

@@ -114,43 +114,42 @@ self.assertRaises(Exception, result._write_number, 1073741824, packet)

# 1 byte short:
result.status(file_name="", file_bytes=b'\xff'*0)
result.status(file_name="", file_bytes=b"\xff" * 0)
self.assertThat(output.getvalue(), HasLength(10))
self.assertEqual(b'\x0a', output.getvalue()[3:4])
self.assertEqual(b"\x0a", output.getvalue()[3:4])
output.seek(0)
output.truncate()
# 1 byte long:
result.status(file_name="", file_bytes=b'\xff'*53)
result.status(file_name="", file_bytes=b"\xff" * 53)
self.assertThat(output.getvalue(), HasLength(63))
self.assertEqual(b'\x3f', output.getvalue()[3:4])
self.assertEqual(b"\x3f", output.getvalue()[3:4])
output.seek(0)
output.truncate()
# 2 bytes short
result.status(file_name="", file_bytes=b'\xff'*54)
result.status(file_name="", file_bytes=b"\xff" * 54)
self.assertThat(output.getvalue(), HasLength(65))
self.assertEqual(b'\x40\x41', output.getvalue()[3:5])
self.assertEqual(b"\x40\x41", output.getvalue()[3:5])
output.seek(0)
output.truncate()
# 2 bytes long
result.status(file_name="", file_bytes=b'\xff'*16371)
result.status(file_name="", file_bytes=b"\xff" * 16371)
self.assertThat(output.getvalue(), HasLength(16383))
self.assertEqual(b'\x7f\xff', output.getvalue()[3:5])
self.assertEqual(b"\x7f\xff", output.getvalue()[3:5])
output.seek(0)
output.truncate()
# 3 bytes short
result.status(file_name="", file_bytes=b'\xff'*16372)
result.status(file_name="", file_bytes=b"\xff" * 16372)
self.assertThat(output.getvalue(), HasLength(16385))
self.assertEqual(b'\x80\x40\x01', output.getvalue()[3:6])
self.assertEqual(b"\x80\x40\x01", output.getvalue()[3:6])
output.seek(0)
output.truncate()
# 3 bytes long
result.status(file_name="", file_bytes=b'\xff'*4194289)
result.status(file_name="", file_bytes=b"\xff" * 4194289)
self.assertThat(output.getvalue(), HasLength(4194303))
self.assertEqual(b'\xbf\xff\xff', output.getvalue()[3:6])
self.assertEqual(b"\xbf\xff\xff", output.getvalue()[3:6])
output.seek(0)
output.truncate()
self.assertRaises(Exception, result.status, file_name="",
file_bytes=b'\xff'*4194290)
self.assertRaises(Exception, result.status, file_name="", file_bytes=b"\xff" * 4194290)
def test_trivial_enumeration(self):
result, output = self._make_result()
result.status("foo", 'exists')
result.status("foo", "exists")
self.assertEqual(CONSTANT_ENUM, output.getvalue())

@@ -160,3 +159,3 @@

result, output = self._make_result()
result.status("foo", 'inprogress')
result.status("foo", "inprogress")
self.assertEqual(CONSTANT_INPROGRESS, output.getvalue())

@@ -166,3 +165,3 @@

result, output = self._make_result()
result.status("foo", 'success')
result.status("foo", "success")
self.assertEqual(CONSTANT_SUCCESS, output.getvalue())

@@ -172,3 +171,3 @@

result, output = self._make_result()
result.status("foo", 'uxsuccess')
result.status("foo", "uxsuccess")
self.assertEqual(CONSTANT_UXSUCCESS, output.getvalue())

@@ -178,3 +177,3 @@

result, output = self._make_result()
result.status("foo", 'skip')
result.status("foo", "skip")
self.assertEqual(CONSTANT_SKIP, output.getvalue())

@@ -184,3 +183,3 @@

result, output = self._make_result()
result.status("foo", 'fail')
result.status("foo", "fail")
self.assertEqual(CONSTANT_FAIL, output.getvalue())

@@ -190,3 +189,3 @@

result, output = self._make_result()
result.status("foo", 'xfail')
result.status("foo", "xfail")
self.assertEqual(CONSTANT_XFAIL, output.getvalue())

@@ -196,4 +195,4 @@

result, output = self._make_result()
self.assertRaises(Exception, result.status, "foo", 'boo')
self.assertEqual(b'', output.getvalue())
self.assertRaises(Exception, result.status, "foo", "boo")
self.assertEqual(b"", output.getvalue())

@@ -217,4 +216,3 @@ def test_eof(self):

result, output = self._make_result()
result.status(test_id="bar", test_status='success',
route_code="source")
result.status(test_id="bar", test_status="success", route_code="source")
self.assertEqual(CONSTANT_ROUTE_CODE, output.getvalue())

@@ -224,3 +222,3 @@

result, output = self._make_result()
result.status("foo", 'success', runnable=False)
result.status("foo", "success", runnable=False)
self.assertEqual(CONSTANT_RUNNABLE, output.getvalue())

@@ -230,10 +228,9 @@

result, output = self._make_result()
result.status(test_id="bar", test_tags={'foo', 'bar'})
result.status(test_id="bar", test_tags={"foo", "bar"})
self.assertThat(CONSTANT_TAGS, Contains(output.getvalue()))
def test_timestamp(self):
timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45,
iso8601.UTC)
timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45, iso8601.UTC)
result, output = self._make_result()
result.status(test_id="bar", test_status='success', timestamp=timestamp)
result.status(test_id="bar", test_status="success", timestamp=timestamp)
self.assertEqual(CONSTANT_TIMESTAMP, output.getvalue())

@@ -243,33 +240,35 @@

class TestByteStreamToStreamResult(TestCase):
def test_non_subunit_encapsulated(self):
source = BytesIO(b"foo\nbar\n")
result = StreamResult()
subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout").run(result)
self.assertEqual([
('status', None, None, None, True, 'stdout', b'f', False, None, None, None),
('status', None, None, None, True, 'stdout', b'o', False, None, None, None),
('status', None, None, None, True, 'stdout', b'o', False, None, None, None),
('status', None, None, None, True, 'stdout', b'\n', False, None, None, None),
('status', None, None, None, True, 'stdout', b'b', False, None, None, None),
('status', None, None, None, True, 'stdout', b'a', False, None, None, None),
('status', None, None, None, True, 'stdout', b'r', False, None, None, None),
('status', None, None, None, True, 'stdout', b'\n', False, None, None, None),
], result._events)
self.assertEqual(b'', source.read())
subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout").run(result)
self.assertEqual(
[
("status", None, None, None, True, "stdout", b"f", False, None, None, None),
("status", None, None, None, True, "stdout", b"o", False, None, None, None),
("status", None, None, None, True, "stdout", b"o", False, None, None, None),
("status", None, None, None, True, "stdout", b"\n", False, None, None, None),
("status", None, None, None, True, "stdout", b"b", False, None, None, None),
("status", None, None, None, True, "stdout", b"a", False, None, None, None),
("status", None, None, None, True, "stdout", b"r", False, None, None, None),
("status", None, None, None, True, "stdout", b"\n", False, None, None, None),
],
result._events,
)
self.assertEqual(b"", source.read())
def test_signature_middle_utf8_char(self):
utf8_bytes = b'\xe3\xb3\x8a'
utf8_bytes = b"\xe3\xb3\x8a"
source = BytesIO(utf8_bytes)
# Should be treated as one character (it is u'\u3cca') and wrapped
result = StreamResult()
subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout").run(
result)
self.assertEqual([
('status', None, None, None, True, 'stdout', b'\xe3', False, None, None, None),
('status', None, None, None, True, 'stdout', b'\xb3', False, None, None, None),
('status', None, None, None, True, 'stdout', b'\x8a', False, None, None, None),
], result._events)
subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout").run(result)
self.assertEqual(
[
("status", None, None, None, True, "stdout", b"\xe3", False, None, None, None),
("status", None, None, None, True, "stdout", b"\xb3", False, None, None, None),
("status", None, None, None, True, "stdout", b"\x8a", False, None, None, None),
],
result._events,
)

@@ -281,4 +280,4 @@ def test_non_subunit_disabled_raises(self):

e = self.assertRaises(Exception, case.run, result)
self.assertEqual(b'f', e.args[1])
self.assertEqual(b'oo\nbar\n', source.read())
self.assertEqual(b"f", e.args[1])
self.assertEqual(b"oo\nbar\n", source.read())
self.assertEqual([], result._events)

@@ -289,8 +288,10 @@

result = StreamResult()
subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout").run(result)
self.assertEqual(b'', source.read())
self.assertEqual([
('status', 'foo', 'exists', None, True, None, None, False, None, None, None),
], result._events)
subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout").run(result)
self.assertEqual(b"", source.read())
self.assertEqual(
[
("status", "foo", "exists", None, True, None, None, False, None, None, None),
],
result._events,
)

@@ -300,27 +301,29 @@ def test_multiple_events(self):

result = StreamResult()
subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout").run(result)
self.assertEqual(b'', source.read())
self.assertEqual([
('status', 'foo', 'exists', None, True, None, None, False, None, None, None),
('status', 'foo', 'exists', None, True, None, None, False, None, None, None),
], result._events)
subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout").run(result)
self.assertEqual(b"", source.read())
self.assertEqual(
[
("status", "foo", "exists", None, True, None, None, False, None, None, None),
("status", "foo", "exists", None, True, None, None, False, None, None, None),
],
result._events,
)
def test_inprogress(self):
self.check_event(CONSTANT_INPROGRESS, 'inprogress')
self.check_event(CONSTANT_INPROGRESS, "inprogress")
def test_success(self):
self.check_event(CONSTANT_SUCCESS, 'success')
self.check_event(CONSTANT_SUCCESS, "success")
def test_uxsuccess(self):
self.check_event(CONSTANT_UXSUCCESS, 'uxsuccess')
self.check_event(CONSTANT_UXSUCCESS, "uxsuccess")
def test_skip(self):
self.check_event(CONSTANT_SKIP, 'skip')
self.check_event(CONSTANT_SKIP, "skip")
def test_fail(self):
self.check_event(CONSTANT_FAIL, 'fail')
self.check_event(CONSTANT_FAIL, "fail")
def test_xfail(self):
self.check_event(CONSTANT_XFAIL, 'xfail')
self.check_event(CONSTANT_XFAIL, "xfail")

@@ -330,7 +333,6 @@ def check_events(self, source_bytes, events):

result = StreamResult()
subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout").run(result)
self.assertEqual(b'', source.read())
subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout").run(result)
self.assertEqual(b"", source.read())
self.assertEqual(events, result._events)
#- any file attachments should be bytes equivalent [as users assume that].
# - any file attachments should be bytes equivalent [as users assume that].
for event in result._events:

@@ -340,16 +342,56 @@ if event[5] is not None:

def check_event(self, source_bytes, test_status=None, test_id="foo",
route_code=None, timestamp=None, tags=None, mime_type=None,
file_name=None, file_bytes=None, eof=False, runnable=True):
event = self._event(test_id=test_id, test_status=test_status,
tags=tags, runnable=runnable, file_name=file_name,
file_bytes=file_bytes, eof=eof, mime_type=mime_type,
route_code=route_code, timestamp=timestamp)
def check_event(
self,
source_bytes,
test_status=None,
test_id="foo",
route_code=None,
timestamp=None,
tags=None,
mime_type=None,
file_name=None,
file_bytes=None,
eof=False,
runnable=True,
):
event = self._event(
test_id=test_id,
test_status=test_status,
tags=tags,
runnable=runnable,
file_name=file_name,
file_bytes=file_bytes,
eof=eof,
mime_type=mime_type,
route_code=route_code,
timestamp=timestamp,
)
self.check_events(source_bytes, [event])
def _event(self, test_status=None, test_id=None, route_code=None,
timestamp=None, tags=None, mime_type=None, file_name=None,
file_bytes=None, eof=False, runnable=True):
return ('status', test_id, test_status, tags, runnable, file_name,
file_bytes, eof, mime_type, route_code, timestamp)
def _event(
self,
test_status=None,
test_id=None,
route_code=None,
timestamp=None,
tags=None,
mime_type=None,
file_name=None,
file_bytes=None,
eof=False,
runnable=True,
):
return (
"status",
test_id,
test_status,
tags,
runnable,
file_name,
file_bytes,
eof,
mime_type,
route_code,
timestamp,
)

@@ -360,101 +402,158 @@ def test_eof(self):

def test_file_content(self):
self.check_event(CONSTANT_FILE_CONTENT,
test_id=None, file_name="barney", file_bytes=b"woo")
self.check_event(CONSTANT_FILE_CONTENT, test_id=None, file_name="barney", file_bytes=b"woo")
def test_file_content_length_into_checksum(self):
# A bad file content length which creeps into the checksum.
bad_file_length_content = b'\xb3!@\x13\x06barney\x04woo\xdc\xe2\xdb\x35'
self.check_events(bad_file_length_content, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=bad_file_length_content,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b"File content extends past end of packet: claimed 4 bytes, 3 available",
mime_type="text/plain;charset=utf8"),
])
bad_file_length_content = b"\xb3!@\x13\x06barney\x04woo\xdc\xe2\xdb\x35"
self.check_events(
bad_file_length_content,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=bad_file_length_content,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"File content extends past end of packet: claimed 4 bytes, 3 available",
mime_type="text/plain;charset=utf8",
),
],
)
def test_packet_length_4_word_varint(self):
packet_data = b'\xb3!@\xc0\x00\x11'
self.check_events(packet_data, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=packet_data,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b"3 byte maximum given but 4 byte value found.",
mime_type="text/plain;charset=utf8"),
])
packet_data = b"\xb3!@\xc0\x00\x11"
self.check_events(
packet_data,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=packet_data,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"3 byte maximum given but 4 byte value found.",
mime_type="text/plain;charset=utf8",
),
],
)
def test_mime(self):
self.check_event(CONSTANT_MIME,
test_id=None, mime_type='application/foo; charset=1')
self.check_event(CONSTANT_MIME, test_id=None, mime_type="application/foo; charset=1")
def test_route_code(self):
self.check_event(CONSTANT_ROUTE_CODE,
'success', route_code="source", test_id="bar")
self.check_event(CONSTANT_ROUTE_CODE, "success", route_code="source", test_id="bar")
def test_runnable(self):
self.check_event(CONSTANT_RUNNABLE,
test_status='success', runnable=False)
self.check_event(CONSTANT_RUNNABLE, test_status="success", runnable=False)
def test_tags(self):
self.check_event(CONSTANT_TAGS[0],
None, tags={'foo', 'bar'}, test_id="bar")
self.check_event(CONSTANT_TAGS[0], None, tags={"foo", "bar"}, test_id="bar")
def test_timestamp(self):
timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45,
iso8601.UTC)
self.check_event(CONSTANT_TIMESTAMP,
'success', test_id='bar', timestamp=timestamp)
timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45, iso8601.UTC)
self.check_event(CONSTANT_TIMESTAMP, "success", test_id="bar", timestamp=timestamp)
def test_bad_crc_errors_via_status(self):
file_bytes = CONSTANT_MIME[:-1] + b'\x00'
self.check_events( file_bytes, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=file_bytes,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b'Bad checksum - calculated (0x78335115), '
b'stored (0x78335100)',
mime_type="text/plain;charset=utf8"),
])
file_bytes = CONSTANT_MIME[:-1] + b"\x00"
self.check_events(
file_bytes,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=file_bytes,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"Bad checksum - calculated (0x78335115), stored (0x78335100)",
mime_type="text/plain;charset=utf8",
),
],
)
def test_not_utf8_in_string(self):
file_bytes = CONSTANT_ROUTE_CODE[:5] + b'\xb4' + CONSTANT_ROUTE_CODE[6:-4] + b'\xce\x56\xc6\x17'
self.check_events(file_bytes, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=file_bytes,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b'UTF8 string at offset 2 is not UTF8',
mime_type="text/plain;charset=utf8"),
])
file_bytes = CONSTANT_ROUTE_CODE[:5] + b"\xb4" + CONSTANT_ROUTE_CODE[6:-4] + b"\xce\x56\xc6\x17"
self.check_events(
file_bytes,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=file_bytes,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"UTF8 string at offset 2 is not UTF8",
mime_type="text/plain;charset=utf8",
),
],
)
def test_NULL_in_string(self):
file_bytes = CONSTANT_ROUTE_CODE[:6] + b'\x00' + CONSTANT_ROUTE_CODE[7:-4] + b'\xd7\x41\xac\xfe'
self.check_events(file_bytes, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=file_bytes,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b'UTF8 string at offset 2 contains NUL byte',
mime_type="text/plain;charset=utf8"),
])
file_bytes = CONSTANT_ROUTE_CODE[:6] + b"\x00" + CONSTANT_ROUTE_CODE[7:-4] + b"\xd7\x41\xac\xfe"
self.check_events(
file_bytes,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=file_bytes,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"UTF8 string at offset 2 contains NUL byte",
mime_type="text/plain;charset=utf8",
),
],
)
def test_bad_utf8_stringlength(self):
file_bytes = CONSTANT_ROUTE_CODE[:4] + b'\x3f' + CONSTANT_ROUTE_CODE[5:-4] + b'\xbe\x29\xe0\xc2'
self.check_events(file_bytes, [
self._event(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=file_bytes,
mime_type="application/octet-stream"),
self._event(test_id="subunit.parser", test_status="fail", eof=True,
file_name="Parser Error",
file_bytes=b'UTF8 string at offset 2 extends past end of '
b'packet: claimed 63 bytes, 10 available',
mime_type="text/plain;charset=utf8"),
])
file_bytes = CONSTANT_ROUTE_CODE[:4] + b"\x3f" + CONSTANT_ROUTE_CODE[5:-4] + b"\xbe\x29\xe0\xc2"
self.check_events(
file_bytes,
[
self._event(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=file_bytes,
mime_type="application/octet-stream",
),
self._event(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=b"UTF8 string at offset 2 extends past end of packet: claimed 63 bytes, 10 available",
mime_type="text/plain;charset=utf8",
),
],
)

@@ -464,8 +563,10 @@ def test_route_code_and_file_content(self):

subunit.StreamResultToBytes(content).status(
route_code='0', mime_type='text/plain', file_name='bar',
file_bytes=b'foo')
self.check_event(content.getvalue(), test_id=None, file_name='bar',
route_code='0', mime_type='text/plain', file_bytes=b'foo')
route_code="0", mime_type="text/plain", file_name="bar", file_bytes=b"foo"
)
self.check_event(
content.getvalue(), test_id=None, file_name="bar", route_code="0", mime_type="text/plain", file_bytes=b"foo"
)
if st is not None:
@given(st.binary())

@@ -475,5 +576,4 @@ def test_hypothesis_decoding(self, code_bytes):

result = StreamResult()
stream = subunit.ByteStreamToStreamResult(
source, non_subunit_name="stdout")
stream = subunit.ByteStreamToStreamResult(source, non_subunit_name="stdout")
stream.run(result)
self.assertEqual(b'', source.read())
self.assertEqual(b"", source.read())
+130
-139
python/subunit/tests/test_test_results.py

@@ -34,3 +34,2 @@ #

class LoggingDecorator(subunit.test_results.HookedTestResultDecorator):
def __init__(self, decorated):

@@ -57,3 +56,2 @@ self._calls = 0

class TimeCapturingResult(unittest.TestResult):
def __init__(self):

@@ -67,5 +65,15 @@ super().__init__()

def progress(self, offset, whence):
pass
def addDuration(self, test, duration):
"""Called to add a test duration.
:param test: The test that completed.
:param duration: The duration of the test as a float in seconds.
"""
pass
class TestHookedTestResultDecorator(unittest.TestCase):
def setUp(self):

@@ -152,5 +160,7 @@ # An end to the chain

def test_addDuration(self):
self.result.addDuration(self, 1.5)
class TestAutoTimingTestResultDecorator(unittest.TestCase):
def setUp(self):

@@ -160,4 +170,3 @@ # And end to the chain which captures time events.

# The result object under test.
self.result = subunit.test_results.AutoTimingTestResultDecorator(
terminal)
self.result = subunit.test_results.AutoTimingTestResultDecorator(terminal)
self.decorated = terminal

@@ -182,3 +191,3 @@

# automatically adding one when other methods are called.
time = datetime.datetime(2009,10,11,12,13,14,15, iso8601.UTC)
time = datetime.datetime(2009, 10, 11, 12, 13, 14, 15, iso8601.UTC)
self.result.time(time)

@@ -191,3 +200,3 @@ self.result.startTest(self)

def test_calling_time_None_enables_automatic_time(self):
time = datetime.datetime(2009,10,11,12,13,14,15, iso8601.UTC)
time = datetime.datetime(2009, 10, 11, 12, 13, 14, 15, iso8601.UTC)
self.result.time(time)

@@ -212,14 +221,14 @@ self.assertEqual(1, len(self.decorated._calls))

class TestTagCollapsingDecorator(TestCase):
def test_tags_collapsed_outside_of_tests(self):
result = ExtendedTestResult()
tag_collapser = subunit.test_results.TagCollapsingDecorator(result)
tag_collapser.tags({'a'}, set())
tag_collapser.tags({'b'}, set())
tag_collapser.tags({"a"}, set())
tag_collapser.tags({"b"}, set())
tag_collapser.startTest(self)
self.assertEqual(
[('tags', {'a', 'b'}, set()),
('startTest', self),
],
result._events
[
("tags", {"a", "b"}, set()),
("startTest", self),
],
result._events,
)

@@ -231,4 +240,4 @@

tag_collapser.startTestRun()
tag_collapser.tags({'a'}, set())
tag_collapser.tags({'b'}, set())
tag_collapser.tags({"a"}, set())
tag_collapser.tags({"b"}, set())
tag_collapser.startTest(self)

@@ -239,14 +248,15 @@ tag_collapser.addSuccess(self)

self.assertEqual(
[('startTestRun',),
('tags', {'a', 'b'}, set()),
('startTest', self),
('addSuccess', self),
('stopTest', self),
('stopTestRun',),
],
result._events
[
("startTestRun",),
("tags", {"a", "b"}, set()),
("startTest", self),
("addSuccess", self),
("stopTest", self),
("stopTestRun",),
],
result._events,
)
def test_tags_forwarded_after_tests(self):
test = subunit.RemotedTestCase('foo')
test = subunit.RemotedTestCase("foo")
result = ExtendedTestResult()

@@ -258,13 +268,15 @@ tag_collapser = subunit.test_results.TagCollapsingDecorator(result)

tag_collapser.stopTest(test)
tag_collapser.tags({'a'}, {'b'})
tag_collapser.tags({"a"}, {"b"})
tag_collapser.stopTestRun()
self.assertEqual(
[('startTestRun',),
('startTest', test),
('addSuccess', test),
('stopTest', test),
('tags', {'a'}, {'b'}),
('stopTestRun',),
],
result._events)
[
("startTestRun",),
("startTest", test),
("addSuccess", test),
("stopTest", test),
("tags", {"a"}, {"b"}),
("stopTestRun",),
],
result._events,
)

@@ -274,14 +286,9 @@ def test_tags_collapsed_inside_of_tests(self):

tag_collapser = subunit.test_results.TagCollapsingDecorator(result)
test = subunit.RemotedTestCase('foo')
test = subunit.RemotedTestCase("foo")
tag_collapser.startTest(test)
tag_collapser.tags({'a'}, set())
tag_collapser.tags({'b'}, {'a'})
tag_collapser.tags({'c'}, set())
tag_collapser.tags({"a"}, set())
tag_collapser.tags({"b"}, {"a"})
tag_collapser.tags({"c"}, set())
tag_collapser.stopTest(test)
self.assertEqual(
[('startTest', test),
('tags', {'b', 'c'}, {'a'}),
('stopTest', test)],
result._events
)
self.assertEqual([("startTest", test), ("tags", {"b", "c"}, {"a"}), ("stopTest", test)], result._events)

@@ -291,14 +298,9 @@ def test_tags_collapsed_inside_of_tests_different_ordering(self):

tag_collapser = subunit.test_results.TagCollapsingDecorator(result)
test = subunit.RemotedTestCase('foo')
test = subunit.RemotedTestCase("foo")
tag_collapser.startTest(test)
tag_collapser.tags(set(), {'a'})
tag_collapser.tags({'a', 'b'}, set())
tag_collapser.tags({'c'}, set())
tag_collapser.tags(set(), {"a"})
tag_collapser.tags({"a", "b"}, set())
tag_collapser.tags({"c"}, set())
tag_collapser.stopTest(test)
self.assertEqual(
[('startTest', test),
('tags', {'a', 'b', 'c'}, set()),
('stopTest', test)],
result._events
)
self.assertEqual([("startTest", test), ("tags", {"a", "b", "c"}, set()), ("stopTest", test)], result._events)

@@ -312,13 +314,9 @@ def test_tags_sent_before_result(self):

tag_collapser = subunit.test_results.TagCollapsingDecorator(result)
test = subunit.RemotedTestCase('foo')
test = subunit.RemotedTestCase("foo")
tag_collapser.startTest(test)
tag_collapser.tags({'a'}, set())
tag_collapser.tags({"a"}, set())
tag_collapser.addSuccess(test)
tag_collapser.stopTest(test)
self.assertEqual(
[('startTest', test),
('tags', {'a'}, set()),
('addSuccess', test),
('stopTest', test)],
result._events
[("startTest", test), ("tags", {"a"}, set()), ("addSuccess", test), ("stopTest", test)], result._events
)

@@ -328,7 +326,5 @@

class TestTimeCollapsingDecorator(TestCase):
def make_time(self):
# Heh heh.
return datetime.datetime(
2000, 1, self.getUniqueInteger(), tzinfo=iso8601.UTC)
return datetime.datetime(2000, 1, self.getUniqueInteger(), tzinfo=iso8601.UTC)

@@ -341,3 +337,3 @@ def test_initial_time_forwarded(self):

tag_collapser.time(a_time)
self.assertEqual([('time', a_time)], result._events)
self.assertEqual([("time", a_time)], result._events)

@@ -352,7 +348,4 @@ def test_time_collapsed_to_first_and_last(self):

tag_collapser.time(a_time)
tag_collapser.startTest(subunit.RemotedTestCase('foo'))
self.assertEqual(
[('time', times[0]), ('time', times[-1])],
result._events[:-1]
)
tag_collapser.startTest(subunit.RemotedTestCase("foo"))
self.assertEqual([("time", times[0]), ("time", times[-1])], result._events[:-1])

@@ -366,4 +359,4 @@ def test_only_one_time_sent(self):

tag_collapser.time(a_time)
tag_collapser.startTest(subunit.RemotedTestCase('foo'))
self.assertEqual([('time', a_time)], result._events[:-1])
tag_collapser.startTest(subunit.RemotedTestCase("foo"))
self.assertEqual([("time", a_time)], result._events[:-1])

@@ -378,4 +371,4 @@ def test_duplicate_times_not_sent(self):

tag_collapser.time(a_time)
tag_collapser.startTest(subunit.RemotedTestCase('foo'))
self.assertEqual([('time', a_time)], result._events[:-1])
tag_collapser.startTest(subunit.RemotedTestCase("foo"))
self.assertEqual([("time", a_time)], result._events[:-1])

@@ -387,17 +380,10 @@ def test_no_times_inserted(self):

tag_collapser.time(a_time)
foo = subunit.RemotedTestCase('foo')
foo = subunit.RemotedTestCase("foo")
tag_collapser.startTest(foo)
tag_collapser.addSuccess(foo)
tag_collapser.stopTest(foo)
self.assertEqual(
[('time', a_time),
('startTest', foo),
('addSuccess', foo),
('stopTest', foo)],
result._events
)
self.assertEqual([("time", a_time), ("startTest", foo), ("addSuccess", foo), ("stopTest", foo)], result._events)
class TestByTestResultTests(testtools.TestCase):
def setUp(self):

@@ -411,8 +397,8 @@ super().setUp()

defaults = {
'test': self,
'tags': set(),
'details': None,
'start_time': 0,
'stop_time': 1,
}
"test": self,
"tags": set(),
"details": None,
"start_time": 0,
"stop_time": 1,
}
defaults.update(kwargs)

@@ -433,19 +419,19 @@ self.assertEqual([defaults], self.log)

self.result.stopTest(self)
self.assertCalled(status='success')
self.assertCalled(status="success")
def test_add_success_details(self):
self.result.startTest(self)
details = {'foo': 'bar'}
details = {"foo": "bar"}
self.result.addSuccess(self, details=details)
self.result.stopTest(self)
self.assertCalled(status='success', details=details)
self.assertCalled(status="success", details=details)
def test_tags(self):
if not getattr(self.result, 'tags', None):
if not getattr(self.result, "tags", None):
self.skipTest("No tags in testtools")
self.result.tags(['foo'], [])
self.result.tags(["foo"], [])
self.result.startTest(self)
self.result.addSuccess(self)
self.result.stopTest(self)
self.assertCalled(status='success', tags={'foo'})
self.assertCalled(status="success", tags={"foo"})

@@ -455,3 +441,3 @@ def test_add_error(self):

try:
1/0
1 / 0
except ZeroDivisionError:

@@ -461,5 +447,3 @@ error = sys.exc_info()

self.result.stopTest(self)
self.assertCalled(
status='error',
details={'traceback': TracebackContent(error, self)})
self.assertCalled(status="error", details={"traceback": TracebackContent(error, self)})

@@ -471,3 +455,3 @@ def test_add_error_details(self):

self.result.stopTest(self)
self.assertCalled(status='error', details=details)
self.assertCalled(status="error", details=details)

@@ -482,5 +466,3 @@ def test_add_failure(self):

self.result.stopTest(self)
self.assertCalled(
status='failure',
details={'traceback': TracebackContent(failure, self)})
self.assertCalled(status="failure", details={"traceback": TracebackContent(failure, self)})

@@ -492,3 +474,3 @@ def test_add_failure_details(self):

self.result.stopTest(self)
self.assertCalled(status='failure', details=details)
self.assertCalled(status="failure", details=details)

@@ -498,3 +480,3 @@ def test_add_xfail(self):

try:
1/0
1 / 0
except ZeroDivisionError:

@@ -504,5 +486,3 @@ error = sys.exc_info()

self.result.stopTest(self)
self.assertCalled(
status='xfail',
details={'traceback': TracebackContent(error, self)})
self.assertCalled(status="xfail", details={"traceback": TracebackContent(error, self)})

@@ -514,10 +494,10 @@ def test_add_xfail_details(self):

self.result.stopTest(self)
self.assertCalled(status='xfail', details=details)
self.assertCalled(status="xfail", details=details)
def test_add_unexpected_success(self):
self.result.startTest(self)
details = {'foo': 'bar'}
details = {"foo": "bar"}
self.result.addUnexpectedSuccess(self, details=details)
self.result.stopTest(self)
self.assertCalled(status='success', details=details)
self.assertCalled(status="success", details=details)

@@ -529,15 +509,14 @@ def test_add_skip_reason(self):

self.result.stopTest(self)
self.assertCalled(
status='skip', details={'reason': text_content(reason)})
self.assertCalled(status="skip", details={"reason": text_content(reason)})
def test_add_skip_details(self):
self.result.startTest(self)
details = {'foo': 'bar'}
details = {"foo": "bar"}
self.result.addSkip(self, details=details)
self.result.stopTest(self)
self.assertCalled(status='skip', details=details)
self.assertCalled(status="skip", details=details)
def test_twice(self):
self.result.startTest(self)
self.result.addSuccess(self, details={'foo': 'bar'})
self.result.addSuccess(self, details={"foo": "bar"})
self.result.stopTest(self)

@@ -548,20 +527,18 @@ self.result.startTest(self)

self.assertEqual(
[{'test': self,
'status': 'success',
'start_time': 0,
'stop_time': 1,
'tags': set(),
'details': {'foo': 'bar'}},
{'test': self,
'status': 'success',
'start_time': 2,
'stop_time': 3,
'tags': set(),
'details': None},
],
self.log)
[
{
"test": self,
"status": "success",
"start_time": 0,
"stop_time": 1,
"tags": set(),
"details": {"foo": "bar"},
},
{"test": self, "status": "success", "start_time": 2, "stop_time": 3, "tags": set(), "details": None},
],
self.log,
)
class TestCsvResult(testtools.TestCase):
def parse_stream(self, stream):

@@ -582,6 +559,8 @@ stream.seek(0)

self.assertEqual(
[['test', 'status', 'start_time', 'stop_time'],
[self.id(), 'success', '0', '1'],
],
self.parse_stream(stream))
[
["test", "status", "start_time", "stop_time"],
[self.id(), "success", "0", "1"],
],
self.parse_stream(stream),
)

@@ -593,5 +572,3 @@ def test_just_header_when_no_tests(self):

result.stopTestRun()
self.assertEqual(
[['test', 'status', 'start_time', 'stop_time']],
self.parse_stream(stream))
self.assertEqual([["test", "status", "start_time", "stop_time"]], self.parse_stream(stream))

@@ -602,1 +579,15 @@ def test_no_output_before_events(self):

self.assertEqual([], self.parse_stream(stream))
class TestTestIdPrintingResult(testtools.TestCase):
def setUp(self):
super().setUp()
self.stream = StringIO()
self.result = subunit.test_results.TestIdPrintingResult(self.stream)
def test_addDuration(self):
# addDuration should not raise an exception
test = subunit.RemotedTestCase("foo")
self.result.addDuration(test, 2.5)
# TestIdPrintingResult doesn't output anything for addDuration
self.assertEqual("", self.stream.getvalue())
+138
-105
python/subunit/v2.py

@@ -17,3 +17,2 @@ #

import builtins
import codecs

@@ -32,12 +31,12 @@ import datetime

__all__ = [
'ByteStreamToStreamResult',
'StreamResultToBytes',
]
"ByteStreamToStreamResult",
"StreamResultToBytes",
]
SIGNATURE = b'\xb3'
FMT_8 = '>B'
FMT_16 = '>H'
FMT_24 = '>HB'
FMT_32 = '>I'
FMT_TIMESTAMP = '>II'
SIGNATURE = b"\xb3"
FMT_8 = ">B"
FMT_16 = ">H"
FMT_24 = ">HB"
FMT_32 = ">I"
FMT_TIMESTAMP = ">II"
FLAG_TEST_ID = 0x0800

@@ -52,3 +51,3 @@ FLAG_ROUTE_CODE = 0x0400

EPOCH = datetime.datetime.fromtimestamp(0, tz=iso8601.UTC)
NUL_ELEMENT = b'\0'[0]
NUL_ELEMENT = b"\0"[0]
# Contains True for types for which 'nul in thing' falsely returns false.

@@ -65,3 +64,3 @@ _nul_test_broken = {}

"""
data = b''
data = b""
remaining = size

@@ -71,4 +70,3 @@ while remaining:

if len(read) == 0:
raise ParseError('Short read - got %d bytes, wanted %d bytes' % (
len(data), size))
raise ParseError("Short read - got %d bytes, wanted %d bytes" % (len(data), size))
data += read

@@ -91,12 +89,12 @@ remaining -= len(read)

None: 0,
'exists': 0x1,
'inprogress': 0x2,
'success': 0x3,
'uxsuccess': 0x4,
'skip': 0x5,
'fail': 0x6,
'xfail': 0x7,
}
"exists": 0x1,
"inprogress": 0x2,
"success": 0x3,
"uxsuccess": 0x4,
"skip": 0x5,
"fail": 0x6,
"xfail": 0x7,
}
zero_b = b'\0'[0]
zero_b = b"\0"[0]

@@ -119,12 +117,30 @@ def __init__(self, output_stream):

def status(self, test_id=None, test_status=None, test_tags=None,
runnable=True, file_name=None, file_bytes=None, eof=False,
mime_type=None, route_code=None, timestamp=None):
self._write_packet(test_id=test_id, test_status=test_status,
test_tags=test_tags, runnable=runnable, file_name=file_name,
file_bytes=file_bytes, eof=eof, mime_type=mime_type,
route_code=route_code, timestamp=timestamp)
def status(
self,
test_id=None,
test_status=None,
test_tags=None,
runnable=True,
file_name=None,
file_bytes=None,
eof=False,
mime_type=None,
route_code=None,
timestamp=None,
):
self._write_packet(
test_id=test_id,
test_status=test_status,
test_tags=test_tags,
runnable=runnable,
file_name=file_name,
file_bytes=file_bytes,
eof=eof,
mime_type=mime_type,
route_code=route_code,
timestamp=timestamp,
)
def _write_utf8(self, a_string, packet):
utf8 = a_string.encode('utf-8')
utf8 = a_string.encode("utf-8")
self._write_number(len(utf8), packet)

@@ -149,18 +165,27 @@ packet.append(utf8)

value = value | 0x800000
return [struct.pack(FMT_16, value >> 8),
struct.pack(FMT_8, value & 0xff)]
return [struct.pack(FMT_16, value >> 8), struct.pack(FMT_8, value & 0xFF)]
elif value < 1073741824:
value = value | 0xc0000000
value = value | 0xC0000000
return [struct.pack(FMT_32, value)]
else:
raise ValueError('value too large to encode: %r' % (value,))
raise ValueError("value too large to encode: %r" % (value,))
def _write_packet(self, test_id=None, test_status=None, test_tags=None,
runnable=True, file_name=None, file_bytes=None, eof=False,
mime_type=None, route_code=None, timestamp=None):
def _write_packet(
self,
test_id=None,
test_status=None,
test_tags=None,
runnable=True,
file_name=None,
file_bytes=None,
eof=False,
mime_type=None,
route_code=None,
timestamp=None,
):
packet = [SIGNATURE]
packet.append(b'FF') # placeholder for flags
packet.append(b"FF") # placeholder for flags
# placeholder for length, but see below as length is variable.
packet.append(b'')
flags = 0x2000 # Version 0x2
packet.append(b"")
flags = 0x2000 # Version 0x2
if timestamp is not None:

@@ -170,3 +195,3 @@ flags = flags | FLAG_TIMESTAMP

nanoseconds = since_epoch.microseconds * 1000
seconds = (since_epoch.seconds + since_epoch.days * 24 * 3600)
seconds = since_epoch.seconds + since_epoch.days * 24 * 3600
packet.append(struct.pack(FMT_32, seconds))

@@ -192,4 +217,4 @@ self._write_number(nanoseconds, packet)

packet.append(file_bytes)
if eof:
flags = flags | FLAG_EOF
if eof:
flags = flags | FLAG_EOF
if route_code is not None:

@@ -223,4 +248,4 @@ flags = flags | FLAG_ROUTE_CODE

# For now, simplest code: join, crc32, join, output
content = b''.join(packet)
data = content + struct.pack(FMT_32, zlib.crc32(content) & 0xffffffff)
content = b"".join(packet)
data = content + struct.pack(FMT_32, zlib.crc32(content) & 0xFFFFFFFF)
# On eventlet 0.17.3, GreenIO.write() can make partial write.

@@ -245,3 +270,3 @@ # Use a loop to ensure that all bytes are written.

Mixed streams that contain non-subunit content is supported when a
non_subunit_name is passed to the contructor. The default is to raise an
non_subunit_name is passed to the constructor. The default is to raise an
error containing the non-subunit byte after it has been read from the

@@ -261,10 +286,10 @@ stream.

0x0: None,
0x1: 'exists',
0x2: 'inprogress',
0x3: 'success',
0x4: 'uxsuccess',
0x5: 'skip',
0x6: 'fail',
0x7: 'xfail',
}
0x1: "exists",
0x2: "inprogress",
0x3: "success",
0x4: "uxsuccess",
0x5: "skip",
0x6: "fail",
0x7: "xfail",
}

@@ -284,3 +309,3 @@ def __init__(self, source, non_subunit_name=None):

self.source = subunit.make_stream_binary(source)
self.codec = codecs.lookup('utf8').incrementaldecoder()
self.codec = codecs.lookup("utf8").incrementaldecoder()

@@ -326,3 +351,3 @@ def run(self, result):

# select.select. fallback to one-byte-at-a-time.
if sys.platform == 'win32':
if sys.platform == "win32":
break

@@ -365,5 +390,3 @@

break
result.status(
file_name=self.non_subunit_name,
file_bytes=b''.join(buffered))
result.status(file_name=self.non_subunit_name, file_bytes=b"".join(buffered))
if mid_character or not len(content) or content[0] != SIGNATURE[0]:

@@ -379,9 +402,17 @@ continue

except ParseError as error:
result.status(test_id="subunit.parser", eof=True,
file_name="Packet data", file_bytes=b''.join(packet),
mime_type="application/octet-stream")
result.status(test_id="subunit.parser", test_status='fail',
eof=True, file_name="Parser Error",
file_bytes=(error.args[0]).encode('utf8'),
mime_type="text/plain;charset=utf8")
result.status(
test_id="subunit.parser",
eof=True,
file_name="Packet data",
file_bytes=b"".join(packet),
mime_type="application/octet-stream",
)
result.status(
test_id="subunit.parser",
test_status="fail",
eof=True,
file_name="Parser Error",
file_bytes=(error.args[0]).encode("utf8"),
mime_type="text/plain;charset=utf8",
)

@@ -393,17 +424,17 @@ def _parse_varint(self, data, pos, max_3_bytes=False):

# error not a normal situation.
data_0 = struct.unpack(FMT_8, data[pos:pos+1])[0]
typeenum = data_0 & 0xc0
value_0 = data_0 & 0x3f
data_0 = struct.unpack(FMT_8, data[pos : pos + 1])[0]
typeenum = data_0 & 0xC0
value_0 = data_0 & 0x3F
if typeenum == 0x00:
return value_0, 1
elif typeenum == 0x40:
data_1 = struct.unpack(FMT_8, data[pos+1:pos+2])[0]
data_1 = struct.unpack(FMT_8, data[pos + 1 : pos + 2])[0]
return (value_0 << 8) | data_1, 2
elif typeenum == 0x80:
data_1 = struct.unpack(FMT_16, data[pos+1:pos+3])[0]
data_1 = struct.unpack(FMT_16, data[pos + 1 : pos + 3])[0]
return (value_0 << 16) | data_1, 3
else:
if max_3_bytes:
raise ParseError('3 byte maximum given but 4 byte value found.')
data_1, data_2 = struct.unpack(FMT_24, data[pos+1:pos+4])
raise ParseError("3 byte maximum given but 4 byte value found.")
data_1, data_2 = struct.unpack(FMT_24, data[pos + 1 : pos + 4])
result = (value_0 << 24) | data_1 << 8 | data_2

@@ -433,3 +464,3 @@ return result, 4

crc = zlib.crc32(packet[-1][:-4], crc) & 0xffffffff
crc = zlib.crc32(packet[-1][:-4], crc) & 0xFFFFFFFF
packet_crc = struct.unpack(FMT_32, packet[-1][-4:])[0]

@@ -439,10 +470,5 @@

# Bad CRC, report it and stop parsing the packet.
raise ParseError(
'Bad checksum - calculated (0x%x), stored (0x%x)' % (
crc, packet_crc))
raise ParseError("Bad checksum - calculated (0x%x), stored (0x%x)" % (crc, packet_crc))
if hasattr(builtins, 'memoryview'):
body = memoryview(packet[-1])
else:
body = packet[-1]
body = memoryview(packet[-1])

@@ -455,7 +481,6 @@ # Discard CRC-32

if flags & FLAG_TIMESTAMP:
seconds = struct.unpack(FMT_32, body[pos:pos+4])[0]
nanoseconds, consumed = self._parse_varint(body, pos+4)
seconds = struct.unpack(FMT_32, body[pos : pos + 4])[0]
nanoseconds, consumed = self._parse_varint(body, pos + 4)
pos = pos + 4 + consumed
timestamp = EPOCH + datetime.timedelta(
seconds=seconds, microseconds=nanoseconds/1000)
timestamp = EPOCH + datetime.timedelta(seconds=seconds, microseconds=nanoseconds / 1000)
else:

@@ -488,7 +513,8 @@ timestamp = None

pos += consumed
file_bytes = body[pos:pos+content_length]
file_bytes = body[pos : pos + content_length]
if len(file_bytes) != content_length:
raise ParseError('File content extends past end of packet: '
'claimed %d bytes, %d available' % (
content_length, len(file_bytes)))
raise ParseError(
"File content extends past end of packet: "
"claimed %d bytes, %d available" % (content_length, len(file_bytes))
)
pos += content_length

@@ -508,6 +534,13 @@ else:

result.status(
test_id=test_id, test_status=test_status,
test_tags=test_tags, runnable=runnable, mime_type=mime_type,
eof=eof, file_name=file_name, file_bytes=file_bytes,
route_code=route_code, timestamp=timestamp)
test_id=test_id,
test_status=test_status,
test_tags=test_tags,
runnable=runnable,
mime_type=mime_type,
eof=eof,
file_name=file_name,
file_bytes=file_bytes,
route_code=route_code,
timestamp=timestamp,
)

@@ -519,19 +552,19 @@ __call__ = run

pos += consumed
utf8_bytes = buf[pos:pos+length]
utf8_bytes = buf[pos : pos + length]
if length != len(utf8_bytes):
raise ParseError(
'UTF8 string at offset %d extends past end of packet: '
'claimed %d bytes, %d available' % (pos - 2, length,
len(utf8_bytes)))
"UTF8 string at offset %d extends past end of packet: "
"claimed %d bytes, %d available" % (pos - 2, length, len(utf8_bytes))
)
if NUL_ELEMENT in utf8_bytes:
raise ParseError('UTF8 string at offset %d contains NUL byte' % (
pos-2,))
raise ParseError("UTF8 string at offset %d contains NUL byte" % (pos - 2,))
try:
utf8, decoded_bytes = utf_8_decode(utf8_bytes)
if decoded_bytes != length:
raise ParseError("Invalid (partially decodable) string at "
"offset %d, %d undecoded bytes" % (
pos-2, length - decoded_bytes))
return utf8, length+pos
raise ParseError(
"Invalid (partially decodable) string at "
"offset %d, %d undecoded bytes" % (pos - 2, length - decoded_bytes)
)
return utf8, length + pos
except UnicodeDecodeError:
raise ParseError('UTF8 string at offset %d is not UTF8' % (pos-2,))
raise ParseError("UTF8 string at offset %d is not UTF8" % (pos - 2,))
+1
-89
setup.py
#!/usr/bin/env python3
import os.path
from setuptools import setup
def _get_version_from_file(filename, start_of_line, split_marker):
"""Extract version from file, giving last matching value or None"""
try:
return [
x for x in open(filename) if x.startswith(start_of_line)
][-1].split(split_marker)[1].strip()
except (IOError, IndexError):
return None
VERSION = (
# Assume we are in a distribution, which has PKG-INFO
_get_version_from_file('PKG-INFO', 'Version:', ':')
# Must be a development checkout, so use the Makefile
or _get_version_from_file('Makefile', 'VERSION', '=')
or "0.0"
)
relpath = os.path.dirname(__file__)
if relpath:
os.chdir(relpath)
setup(
name='python-subunit',
version=VERSION,
description=('Python implementation of subunit test streaming protocol'),
long_description=open('README.rst').read(),
classifiers=[
'Intended Audience :: Developers',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Topic :: Software Development :: Testing',
],
keywords='python test streaming',
author='Robert Collins',
author_email='subunit-dev@lists.launchpad.net',
url='http://launchpad.net/subunit',
license='Apache-2.0 or BSD',
project_urls={
"Bug Tracker": "https://bugs.launchpad.net/subunit",
"Source Code": "https://github.com/testing-cabal/subunit/",
},
packages=['subunit', 'subunit.tests', 'subunit.filter_scripts'],
package_dir={'subunit': 'python/subunit'},
python_requires=">=3.7",
install_requires=[
'iso8601',
'testtools>=0.9.34',
],
entry_points={
'console_scripts': [
'subunit-1to2=subunit.filter_scripts.subunit_1to2:main',
'subunit-2to1=subunit.filter_scripts.subunit_2to1:main',
'subunit-filter=subunit.filter_scripts.subunit_filter:main',
'subunit-ls=subunit.filter_scripts.subunit_ls:main',
'subunit-notify=subunit.filter_scripts.subunit_notify:main',
'subunit-output=subunit.filter_scripts.subunit_output:main',
'subunit-stats=subunit.filter_scripts.subunit_stats:main',
'subunit-tags=subunit.filter_scripts.subunit_tags:main',
'subunit2csv=subunit.filter_scripts.subunit2csv:main',
'subunit2disk=subunit.filter_scripts.subunit2disk:main',
'subunit2gtk=subunit.filter_scripts.subunit2gtk:main',
'subunit2junitxml=subunit.filter_scripts.subunit2junitxml:main',
'subunit2pyunit=subunit.filter_scripts.subunit2pyunit:main',
'tap2subunit=subunit.filter_scripts.tap2subunit:main',
]
},
tests_require=[
'fixtures',
'hypothesis',
'testscenarios',
],
extras_require={
'docs': ['docutils'],
'test': ['fixtures', 'testscenarios', 'hypothesis'],
},
)
setup()
-1
python_subunit.egg-info/dependency_links.txt
-15
python_subunit.egg-info/entry_points.txt
[console_scripts]
subunit-1to2 = subunit.filter_scripts.subunit_1to2:main
subunit-2to1 = subunit.filter_scripts.subunit_2to1:main
subunit-filter = subunit.filter_scripts.subunit_filter:main
subunit-ls = subunit.filter_scripts.subunit_ls:main
subunit-notify = subunit.filter_scripts.subunit_notify:main
subunit-output = subunit.filter_scripts.subunit_output:main
subunit-stats = subunit.filter_scripts.subunit_stats:main
subunit-tags = subunit.filter_scripts.subunit_tags:main
subunit2csv = subunit.filter_scripts.subunit2csv:main
subunit2disk = subunit.filter_scripts.subunit2disk:main
subunit2gtk = subunit.filter_scripts.subunit2gtk:main
subunit2junitxml = subunit.filter_scripts.subunit2junitxml:main
subunit2pyunit = subunit.filter_scripts.subunit2pyunit:main
tap2subunit = subunit.filter_scripts.tap2subunit:main
-517
python_subunit.egg-info/PKG-INFO
Metadata-Version: 2.1
Name: python-subunit
Version: 1.4.4
Summary: Python implementation of subunit test streaming protocol
Home-page: http://launchpad.net/subunit
Author: Robert Collins
Author-email: subunit-dev@lists.launchpad.net
License: Apache-2.0 or BSD
Project-URL: Bug Tracker, https://bugs.launchpad.net/subunit
Project-URL: Source Code, https://github.com/testing-cabal/subunit/
Keywords: python test streaming
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.7
Provides-Extra: docs
Provides-Extra: test
License-File: COPYING
subunit: A streaming protocol for test results
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
See the COPYING file for full details on the licensing of Subunit.
Subunit
-------
Subunit is a streaming protocol for test results.
There are two major revisions of the protocol. Version 1 was trivially human
readable but had significant defects as far as highly parallel testing was
concerned - it had no room for doing discovery and execution in parallel,
required substantial buffering when multiplexing and was fragile - a corrupt
byte could cause an entire stream to be misparsed. Version 1.1 added
encapsulation of binary streams which mitigated some of the issues but the
core remained.
Version 2 shares many of the good characteristics of Version 1 - it can be
embedded into a regular text stream (e.g. from a build system) and it still
models xUnit style test execution. It also fixes many of the issues with
Version 1 - Version 2 can be multiplexed without excessive buffering (in
time or space), it has a well defined recovery mechanism for dealing with
corrupted streams (e.g. where two processes write to the same stream
concurrently, or where the stream generator suffers a bug).
More details on both protocol version s can be found in the 'Protocol' section
of this document.
Subunit comes with command line filters to process a subunit stream and
language bindings for python, C, C++ and shell. Bindings are easy to write
for other languages.
A number of useful things can be done easily with subunit:
* Test aggregation: Tests run separately can be combined and then
reported/displayed together. For instance, tests from different languages
can be shown as a seamless whole, and tests running on multiple machines
can be aggregated into a single stream through a multiplexer.
* Test archiving: A test run may be recorded and replayed later.
* Test isolation: Tests that may crash or otherwise interact badly with each
other can be run seperately and then aggregated, rather than interfering
with each other or requiring an adhoc test->runner reporting protocol.
* Grid testing: subunit can act as the necessary serialisation and
deserialiation to get test runs on distributed machines to be reported in
real time.
Subunit supplies the following filters:
* tap2subunit - convert perl's TestAnythingProtocol to subunit.
* subunit2csv - convert a subunit stream to csv.
* subunit2disk - export a subunit stream to files on disk.
* subunit2pyunit - convert a subunit stream to pyunit test results.
* subunit2gtk - show a subunit stream in GTK.
* subunit2junitxml - convert a subunit stream to JUnit's XML format.
* subunit-diff - compare two subunit streams.
* subunit-filter - filter out tests from a subunit stream.
* subunit-ls - list info about tests present in a subunit stream.
* subunit-stats - generate a summary of a subunit stream.
* subunit-tags - add or remove tags from a stream.
Integration with other tools
----------------------------
Subunit's language bindings act as integration with various test runners like
'check', 'cppunit', Python's 'unittest'. Beyond that a small amount of glue
(typically a few lines) will allow Subunit to be used in more sophisticated
ways.
Python
======
Subunit has excellent Python support: most of the filters and tools are written
in python and there are facilities for using Subunit to increase test isolation
seamlessly within a test suite.
The most common way is to run an existing python test suite and have it output
subunit via the ``subunit.run`` module::
$ python -m subunit.run mypackage.tests.test_suite
For more information on the Python support Subunit offers , please see
``pydoc subunit``, or the source in ``python/subunit/``
C
=
Subunit has C bindings to emit the protocol. The 'check' C unit testing project
has included subunit support in their project for some years now. See
'c/README' for more details.
C++
===
The C library is includable and usable directly from C++. A TestListener for
CPPUnit is included in the Subunit distribution. See 'c++/README' for details.
shell
=====
There are two sets of shell tools. There are filters, which accept a subunit
stream on stdin and output processed data (or a transformed stream) on stdout.
Then there are unittest facilities similar to those for C : shell bindings
consisting of simple functions to output protocol elements, and a patch for
adding subunit output to the 'ShUnit' shell test runner. See 'shell/README' for
details.
Filter recipes
--------------
To ignore some failing tests whose root cause is already known::
subunit-filter --without 'AttributeError.*flavor'
The xUnit test model
--------------------
Subunit implements a slightly modified xUnit test model. The stock standard
model is that there are tests, which have an id(), can be run, and when run
start, emit an outcome (like success or failure) and then finish.
Subunit extends this with the idea of test enumeration (find out about tests
a runner has without running them), tags (allow users to describe tests in
ways the test framework doesn't apply any semantic value to), file attachments
(allow arbitrary data to make analysing a failure easy) and timestamps.
The protocol
------------
Version 2, or v2 is new and still under development, but is intended to
supercede version 1 in the very near future. Subunit's bundled tools accept
only version 2 and only emit version 2, but the new filters subunit-1to2 and
subunit-2to1 can be used to interoperate with older third party libraries.
Version 2
=========
Version 2 is a binary protocol consisting of independent packets that can be
embedded in the output from tools like make - as long as each packet has no
other bytes mixed in with it (which 'make -j N>1' has a tendency of doing).
Version 2 is currently in draft form, and early adopters should be willing
to either discard stored results (if protocol changes are made), or bulk
convert them back to v1 and then to a newer edition of v2.
The protocol synchronises at the start of the stream, after a packet, or
after any 0x0A byte. That is, a subunit v2 packet starts after a newline or
directly after the end of the prior packet.
Subunit is intended to be transported over a reliable streaming protocol such
as TCP. As such it does not concern itself with out of order delivery of
packets. However, because of the possibility of corruption due to either
bugs in the sender, or due to mixed up data from concurrent writes to the same
fd when being embedded, subunit strives to recover reasonably gracefully from
damaged data.
A key design goal for Subunit version 2 is to allow processing and multiplexing
without forcing buffering for semantic correctness, as buffering tends to hide
hung or otherwise misbehaving tests. That said, limited time based buffering
for network efficiency is a good idea - this is ultimately implementator
choice. Line buffering is also discouraged for subunit streams, as dropping
into a debugger or other tool may require interactive traffic even if line
buffering would not otherwise be a problem.
In version two there are two conceptual events - a test status event and a file
attachment event. Events may have timestamps, and the path of multiplexers that
an event is routed through is recorded to permit sending actions back to the
source (such as new tests to run or stdin for driving debuggers and other
interactive input). Test status events are used to enumerate tests, to report
tests and test helpers as they run. Tests may have tags, used to allow
tunnelling extra meanings through subunit without requiring parsing of
arbitrary file attachments. Things that are not standalone tests get marked
as such by setting the 'Runnable' flag to false. (For instance, individual
assertions in TAP are not runnable tests, only the top level TAP test script
is runnable).
File attachments are used to provide rich detail about the nature of a failure.
File attachments can also be used to encapsulate stdout and stderr both during
and outside tests.
Most numbers are stored in network byte order - Most Significant Byte first
encoded using a variation of http://www.dlugosz.com/ZIP2/VLI.html. The first
byte's top 2 high order bits encode the total number of octets in the number.
This encoding can encode values from 0 to 2**30-1, enough to encode a
nanosecond. Numbers that are not variable length encoded are still stored in
MSB order.
+--------+--------+---------+------------+
| prefix | octets | max | max |
+========+========+=========+============+
| 00 | 1 | 2**6-1 | 63 |
+--------+--------+---------+------------+
| 01 | 2 | 2**14-1 | 16383 |
+--------+--------+---------+------------+
| 10 | 3 | 2**22-1 | 4194303 |
+--------+--------+---------+------------+
| 11 | 4 | 2**30-1 | 1073741823 |
+--------+--------+---------+------------+
All variable length elements of the packet are stored with a length prefix
number allowing them to be skipped over for consumers that don't need to
interpret them.
UTF-8 strings are with no terminating NUL and should not have any embedded NULs
(implementations SHOULD validate any such strings that they process and take
some remedial action (such as discarding the packet as corrupt).
In short the structure of a packet is:
PACKET := SIGNATURE FLAGS PACKET_LENGTH TIMESTAMP? TESTID? TAGS? MIME?
FILECONTENT? ROUTING_CODE? CRC32
In more detail...
Packets are identified by a single byte signature - 0xB3, which is never legal
in a UTF-8 stream as the first byte of a character. 0xB3 starts with the first
bit set and the second not, which is the UTF-8 signature for a continuation
byte. 0xB3 was chosen as 0x73 ('s' in ASCII') with the top two bits replaced by
the 1 and 0 for a continuation byte.
If subunit packets are being embedded in a non-UTF-8 text stream, where 0x73 is
a legal character, consider either recoding the text to UTF-8, or using
subunit's 'file' packets to embed the text stream in subunit, rather than the
other way around.
Following the signature byte comes a 16-bit flags field, which includes a
4-bit version field - if the version is not 0x2 then the packet cannot be
read. It is recommended to signal an error at this point (e.g. by emitting
a synthetic error packet and returning to the top level loop to look for
new packets, or exiting with an error). If recovery is desired, treat the
packet signature as an opaque byte and scan for a new synchronisation point.
NB: Subunit V1 and V2 packets may legitimately included 0xB3 internally,
as they are an 8-bit safe container format, so recovery from this situation
may involve an arbitrary number of false positives until an actual packet
is encountered : and even then it may still be false, failing after passing
the version check due to coincidence.
Flags are stored in network byte order too.
+------------+------------+------------------------+
| High byte | Low byte |
+------------+------------+------------------------+
| 15 14 13 12 11 10 9 8 | 7 6 5 4 3 2 1 0 |
+------------+------------+------------------------+
| VERSION | feature bits |
+------------+-------------------------------------+
Valid version values are:
0x2 - version 2
Feature bits:
+---------+-------------+---------------------------+
| Bit 11 | mask 0x0800 | Test id present. |
+---------+-------------+---------------------------+
| Bit 10 | mask 0x0400 | Routing code present. |
+---------+-------------+---------------------------+
| Bit 9 | mask 0x0200 | Timestamp present. |
+---------+-------------+---------------------------+
| Bit 8 | mask 0x0100 | Test is 'runnable'. |
+---------+-------------+---------------------------+
| Bit 7 | mask 0x0080 | Tags are present. |
+---------+-------------+---------------------------+
| Bit 6 | mask 0x0040 | File content is present. |
+---------+-------------+---------------------------+
| Bit 5 | mask 0x0020 | File MIME type is present.|
+---------+-------------+---------------------------+
| Bit 4 | mask 0x0010 | EOF marker. |
+---------+-------------+---------------------------+
| Bit 3 | mask 0x0008 | Must be zero in version 2.|
+---------+-------------+---------------------------+
Test status gets three bits:
Bit 2 | Bit 1 | Bit 0 - mask 0x0007 - A test status enum lookup:
* 000 - undefined / no test
* 001 - Enumeration / existence
* 002 - In progress
* 003 - Success
* 004 - Unexpected Success
* 005 - Skipped
* 006 - Failed
* 007 - Expected failure
After the flags field is a number field giving the length in bytes for the
entire packet including the signature and the checksum. This length must
be less than 4MiB - 4194303 bytes. The encoding can obviously record a larger
number but one of the goals is to avoid requiring large buffers, or causing
large latency in the packet forward/processing pipeline. Larger file
attachments can be communicated in multiple packets, and the overhead in such a
4MiB packet is approximately 0.2%.
The rest of the packet is a series of optional features as specified by the set
feature bits in the flags field. When absent they are entirely absent.
Forwarding and multiplexing of packets can be done without interpreting the
remainder of the packet until the routing code and checksum (which are both at
the end of the packet). Additionally, routers can often avoid copying or moving
the bulk of the packet, as long as the routing code size increase doesn't force
the length encoding to take up a new byte (which will only happen to packets
less than or equal to 16KiB in length) - large packets are very efficient to
route.
Timestamp when present is a 32 bit unsigned integer for seconds, and a variable
length number for nanoseconds, representing UTC time since Unix Epoch in
seconds and nanoseconds.
Test id when present is a UTF-8 string. The test id should uniquely identify
runnable tests such that they can be selected individually. For tests and other
actions which cannot be individually run (such as test
fixtures/layers/subtests) uniqueness is not required (though being human
meaningful is highly recommended).
Tags when present is a length prefixed vector of UTF-8 strings, one per tag.
There are no restrictions on tag content (other than the restrictions on UTF-8
strings in subunit in general). Tags have no ordering.
When a MIME type is present, it defines the MIME type for the file across all
packets same file (routing code + testid + name uniquely identifies a file,
reset when EOF is flagged). If a file never has a MIME type set, it should be
treated as application/octet-stream.
File content when present is a UTF-8 string for the name followed by the length
in bytes of the content, and then the content octets.
If present routing code is a UTF-8 string. The routing code is used to
determine which test backend a test was running on when doing data analysis,
and to route stdin to the test process if interaction is required.
Multiplexers SHOULD add a routing code if none is present, and prefix any
existing routing code with a routing code ('/' separated) if one is already
present. For example, a multiplexer might label each stream it is multiplexing
with a simple ordinal ('0', '1' etc), and given an incoming packet with route
code '3' from stream '0' would adjust the route code when forwarding the packet
to be '0/3'.
Following the end of the packet is a CRC-32 checksum of the contents of the
packet including the signature.
Example packets
~~~~~~~~~~~~~~~
Trivial test "foo" enumeration packet, with test id, runnable set,
status=enumeration. Spaces below are to visually break up signature / flags /
length / testid / crc32
b3 2901 0c 03666f6f 08555f1b
Version 1 (and 1.1)
===================
Version 1 (and 1.1) are mostly human readable protocols.
Sample subunit wire contents
----------------------------
The following::
test: test foo works
success: test foo works
test: tar a file.
failure: tar a file. [
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
]
a writeln to stdout
When run through subunit2pyunit::
.F
a writeln to stdout
========================
FAILURE: tar a file.
-------------------
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
Subunit v1 protocol description
===============================
This description is being ported to an EBNF style. Currently its only partly in
that style, but should be fairly clear all the same. When in doubt, refer the
source (and ideally help fix up the description!). Generally the protocol is
line orientated and consists of either directives and their parameters, or
when outside a DETAILS region unexpected lines which are not interpreted by
the parser - they should be forwarded unaltered::
test|testing|test:|testing: test LABEL
success|success:|successful|successful: test LABEL
success|success:|successful|successful: test LABEL DETAILS
failure: test LABEL
failure: test LABEL DETAILS
error: test LABEL
error: test LABEL DETAILS
skip[:] test LABEL
skip[:] test LABEL DETAILS
xfail[:] test LABEL
xfail[:] test LABEL DETAILS
uxsuccess[:] test LABEL
uxsuccess[:] test LABEL DETAILS
progress: [+|-]X
progress: push
progress: pop
tags: [-]TAG ...
time: YYYY-MM-DD HH:MM:SSZ
LABEL: UTF8*
NAME: UTF8*
DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' CR UTF8-lines ']' CR
MULTIPART ::= '[ multipart' CR PART* ']' CR
PART ::= PART_TYPE CR NAME CR PART_BYTES CR
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF
unexpected output on stdout -> stdout.
exit w/0 or last test completing -> error
Tags given outside a test are applied to all following tests
Tags given after a test: line and before the result line for the same test
apply only to that test, and inherit the current global tags.
A '-' before a tag is used to remove tags - e.g. to prevent a global tag
applying to a single test, or to cancel a global tag.
The progress directive is used to provide progress information about a stream
so that stream consumer can provide completion estimates, progress bars and so
on. Stream generators that know how many tests will be present in the stream
should output "progress: COUNT". Stream filters that add tests should output
"progress: +COUNT", and those that remove tests should output
"progress: -COUNT". An absolute count should reset the progress indicators in
use - it indicates that two separate streams from different generators have
been trivially concatenated together, and there is no knowledge of how many
more complete streams are incoming. Smart concatenation could scan each stream
for their count and sum them, or alternatively translate absolute counts into
relative counts inline. It is recommended that outputters avoid absolute counts
unless necessary. The push and pop directives are used to provide local regions
for progress reporting. This fits with hierarchically operating test
environments - such as those that organise tests into suites - the top-most
runner can report on the number of suites, and each suite surround its output
with a (push, pop) pair. Interpreters should interpret a pop as also advancing
the progress of the restored level by one step. Encountering progress
directives between the start and end of a test pair indicates that a previous
test was interrupted and did not cleanly terminate: it should be implicitly
closed with an error (the same as when a stream ends with no closing test
directive for the most recently started test).
The time directive acts as a clock event - it sets the time for all future
events. The value should be a valid ISO8601 time.
The skip, xfail and uxsuccess outcomes are not supported by all testing
environments. In Python the testttools (https://launchpad.net/testtools)
library is used to translate these automatically if an older Python version
that does not support them is in use. See the testtools documentation for the
translation policy.
skip is used to indicate a test was discovered but not executed. xfail is used
to indicate a test that errored in some expected fashion (also know as "TODO"
tests in some frameworks). uxsuccess is used to indicate and unexpected success
where a test though to be failing actually passes. It is complementary to
xfail.
Hacking on subunit
------------------
Releases
========
* Update versions in configure.ac and python/subunit/__init__.py.
* Update NEWS.
* Do a make distcheck, which will update Makefile etc.
* Do a PyPI release: PYTHONPATH=python python setup.py sdist bdist_wheel; twine upload -s dist/*
* Upload the regular one to LP.
* Push a tagged commit.
git push -t origin master:master
-10
python_subunit.egg-info/requires.txt
iso8601
testtools>=0.9.34
[docs]
docutils
[test]
fixtures
testscenarios
hypothesis
-54
python_subunit.egg-info/SOURCES.txt
COPYING
MANIFEST.in
NEWS
README.rst
pyproject.toml
setup.py
python/subunit/__init__.py
python/subunit/_output.py
python/subunit/_to_disk.py
python/subunit/chunked.py
python/subunit/details.py
python/subunit/filters.py
python/subunit/progress_model.py
python/subunit/run.py
python/subunit/test_results.py
python/subunit/v2.py
python/subunit/filter_scripts/__init__.py
python/subunit/filter_scripts/subunit2csv.py
python/subunit/filter_scripts/subunit2disk.py
python/subunit/filter_scripts/subunit2gtk.py
python/subunit/filter_scripts/subunit2junitxml.py
python/subunit/filter_scripts/subunit2pyunit.py
python/subunit/filter_scripts/subunit_1to2.py
python/subunit/filter_scripts/subunit_2to1.py
python/subunit/filter_scripts/subunit_filter.py
python/subunit/filter_scripts/subunit_ls.py
python/subunit/filter_scripts/subunit_notify.py
python/subunit/filter_scripts/subunit_output.py
python/subunit/filter_scripts/subunit_stats.py
python/subunit/filter_scripts/subunit_tags.py
python/subunit/filter_scripts/tap2subunit.py
python/subunit/tests/__init__.py
python/subunit/tests/sample-script.py
python/subunit/tests/sample-two-script.py
python/subunit/tests/test_chunked.py
python/subunit/tests/test_details.py
python/subunit/tests/test_filter_to_disk.py
python/subunit/tests/test_filters.py
python/subunit/tests/test_output_filter.py
python/subunit/tests/test_progress_model.py
python/subunit/tests/test_run.py
python/subunit/tests/test_subunit_filter.py
python/subunit/tests/test_subunit_stats.py
python/subunit/tests/test_subunit_tags.py
python/subunit/tests/test_tap2subunit.py
python/subunit/tests/test_test_protocol.py
python/subunit/tests/test_test_protocol2.py
python/subunit/tests/test_test_results.py
python_subunit.egg-info/PKG-INFO
python_subunit.egg-info/SOURCES.txt
python_subunit.egg-info/dependency_links.txt
python_subunit.egg-info/entry_points.txt
python_subunit.egg-info/requires.txt
python_subunit.egg-info/top_level.txt
-1
python_subunit.egg-info/top_level.txt
subunit
-490
README.rst
subunit: A streaming protocol for test results
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
See the COPYING file for full details on the licensing of Subunit.
Subunit
-------
Subunit is a streaming protocol for test results.
There are two major revisions of the protocol. Version 1 was trivially human
readable but had significant defects as far as highly parallel testing was
concerned - it had no room for doing discovery and execution in parallel,
required substantial buffering when multiplexing and was fragile - a corrupt
byte could cause an entire stream to be misparsed. Version 1.1 added
encapsulation of binary streams which mitigated some of the issues but the
core remained.
Version 2 shares many of the good characteristics of Version 1 - it can be
embedded into a regular text stream (e.g. from a build system) and it still
models xUnit style test execution. It also fixes many of the issues with
Version 1 - Version 2 can be multiplexed without excessive buffering (in
time or space), it has a well defined recovery mechanism for dealing with
corrupted streams (e.g. where two processes write to the same stream
concurrently, or where the stream generator suffers a bug).
More details on both protocol version s can be found in the 'Protocol' section
of this document.
Subunit comes with command line filters to process a subunit stream and
language bindings for python, C, C++ and shell. Bindings are easy to write
for other languages.
A number of useful things can be done easily with subunit:
* Test aggregation: Tests run separately can be combined and then
reported/displayed together. For instance, tests from different languages
can be shown as a seamless whole, and tests running on multiple machines
can be aggregated into a single stream through a multiplexer.
* Test archiving: A test run may be recorded and replayed later.
* Test isolation: Tests that may crash or otherwise interact badly with each
other can be run seperately and then aggregated, rather than interfering
with each other or requiring an adhoc test->runner reporting protocol.
* Grid testing: subunit can act as the necessary serialisation and
deserialiation to get test runs on distributed machines to be reported in
real time.
Subunit supplies the following filters:
* tap2subunit - convert perl's TestAnythingProtocol to subunit.
* subunit2csv - convert a subunit stream to csv.
* subunit2disk - export a subunit stream to files on disk.
* subunit2pyunit - convert a subunit stream to pyunit test results.
* subunit2gtk - show a subunit stream in GTK.
* subunit2junitxml - convert a subunit stream to JUnit's XML format.
* subunit-diff - compare two subunit streams.
* subunit-filter - filter out tests from a subunit stream.
* subunit-ls - list info about tests present in a subunit stream.
* subunit-stats - generate a summary of a subunit stream.
* subunit-tags - add or remove tags from a stream.
Integration with other tools
----------------------------
Subunit's language bindings act as integration with various test runners like
'check', 'cppunit', Python's 'unittest'. Beyond that a small amount of glue
(typically a few lines) will allow Subunit to be used in more sophisticated
ways.
Python
======
Subunit has excellent Python support: most of the filters and tools are written
in python and there are facilities for using Subunit to increase test isolation
seamlessly within a test suite.
The most common way is to run an existing python test suite and have it output
subunit via the ``subunit.run`` module::
$ python -m subunit.run mypackage.tests.test_suite
For more information on the Python support Subunit offers , please see
``pydoc subunit``, or the source in ``python/subunit/``
C
=
Subunit has C bindings to emit the protocol. The 'check' C unit testing project
has included subunit support in their project for some years now. See
'c/README' for more details.
C++
===
The C library is includable and usable directly from C++. A TestListener for
CPPUnit is included in the Subunit distribution. See 'c++/README' for details.
shell
=====
There are two sets of shell tools. There are filters, which accept a subunit
stream on stdin and output processed data (or a transformed stream) on stdout.
Then there are unittest facilities similar to those for C : shell bindings
consisting of simple functions to output protocol elements, and a patch for
adding subunit output to the 'ShUnit' shell test runner. See 'shell/README' for
details.
Filter recipes
--------------
To ignore some failing tests whose root cause is already known::
subunit-filter --without 'AttributeError.*flavor'
The xUnit test model
--------------------
Subunit implements a slightly modified xUnit test model. The stock standard
model is that there are tests, which have an id(), can be run, and when run
start, emit an outcome (like success or failure) and then finish.
Subunit extends this with the idea of test enumeration (find out about tests
a runner has without running them), tags (allow users to describe tests in
ways the test framework doesn't apply any semantic value to), file attachments
(allow arbitrary data to make analysing a failure easy) and timestamps.
The protocol
------------
Version 2, or v2 is new and still under development, but is intended to
supercede version 1 in the very near future. Subunit's bundled tools accept
only version 2 and only emit version 2, but the new filters subunit-1to2 and
subunit-2to1 can be used to interoperate with older third party libraries.
Version 2
=========
Version 2 is a binary protocol consisting of independent packets that can be
embedded in the output from tools like make - as long as each packet has no
other bytes mixed in with it (which 'make -j N>1' has a tendency of doing).
Version 2 is currently in draft form, and early adopters should be willing
to either discard stored results (if protocol changes are made), or bulk
convert them back to v1 and then to a newer edition of v2.
The protocol synchronises at the start of the stream, after a packet, or
after any 0x0A byte. That is, a subunit v2 packet starts after a newline or
directly after the end of the prior packet.
Subunit is intended to be transported over a reliable streaming protocol such
as TCP. As such it does not concern itself with out of order delivery of
packets. However, because of the possibility of corruption due to either
bugs in the sender, or due to mixed up data from concurrent writes to the same
fd when being embedded, subunit strives to recover reasonably gracefully from
damaged data.
A key design goal for Subunit version 2 is to allow processing and multiplexing
without forcing buffering for semantic correctness, as buffering tends to hide
hung or otherwise misbehaving tests. That said, limited time based buffering
for network efficiency is a good idea - this is ultimately implementator
choice. Line buffering is also discouraged for subunit streams, as dropping
into a debugger or other tool may require interactive traffic even if line
buffering would not otherwise be a problem.
In version two there are two conceptual events - a test status event and a file
attachment event. Events may have timestamps, and the path of multiplexers that
an event is routed through is recorded to permit sending actions back to the
source (such as new tests to run or stdin for driving debuggers and other
interactive input). Test status events are used to enumerate tests, to report
tests and test helpers as they run. Tests may have tags, used to allow
tunnelling extra meanings through subunit without requiring parsing of
arbitrary file attachments. Things that are not standalone tests get marked
as such by setting the 'Runnable' flag to false. (For instance, individual
assertions in TAP are not runnable tests, only the top level TAP test script
is runnable).
File attachments are used to provide rich detail about the nature of a failure.
File attachments can also be used to encapsulate stdout and stderr both during
and outside tests.
Most numbers are stored in network byte order - Most Significant Byte first
encoded using a variation of http://www.dlugosz.com/ZIP2/VLI.html. The first
byte's top 2 high order bits encode the total number of octets in the number.
This encoding can encode values from 0 to 2**30-1, enough to encode a
nanosecond. Numbers that are not variable length encoded are still stored in
MSB order.
+--------+--------+---------+------------+
| prefix | octets | max | max |
+========+========+=========+============+
| 00 | 1 | 2**6-1 | 63 |
+--------+--------+---------+------------+
| 01 | 2 | 2**14-1 | 16383 |
+--------+--------+---------+------------+
| 10 | 3 | 2**22-1 | 4194303 |
+--------+--------+---------+------------+
| 11 | 4 | 2**30-1 | 1073741823 |
+--------+--------+---------+------------+
All variable length elements of the packet are stored with a length prefix
number allowing them to be skipped over for consumers that don't need to
interpret them.
UTF-8 strings are with no terminating NUL and should not have any embedded NULs
(implementations SHOULD validate any such strings that they process and take
some remedial action (such as discarding the packet as corrupt).
In short the structure of a packet is:
PACKET := SIGNATURE FLAGS PACKET_LENGTH TIMESTAMP? TESTID? TAGS? MIME?
FILECONTENT? ROUTING_CODE? CRC32
In more detail...
Packets are identified by a single byte signature - 0xB3, which is never legal
in a UTF-8 stream as the first byte of a character. 0xB3 starts with the first
bit set and the second not, which is the UTF-8 signature for a continuation
byte. 0xB3 was chosen as 0x73 ('s' in ASCII') with the top two bits replaced by
the 1 and 0 for a continuation byte.
If subunit packets are being embedded in a non-UTF-8 text stream, where 0x73 is
a legal character, consider either recoding the text to UTF-8, or using
subunit's 'file' packets to embed the text stream in subunit, rather than the
other way around.
Following the signature byte comes a 16-bit flags field, which includes a
4-bit version field - if the version is not 0x2 then the packet cannot be
read. It is recommended to signal an error at this point (e.g. by emitting
a synthetic error packet and returning to the top level loop to look for
new packets, or exiting with an error). If recovery is desired, treat the
packet signature as an opaque byte and scan for a new synchronisation point.
NB: Subunit V1 and V2 packets may legitimately included 0xB3 internally,
as they are an 8-bit safe container format, so recovery from this situation
may involve an arbitrary number of false positives until an actual packet
is encountered : and even then it may still be false, failing after passing
the version check due to coincidence.
Flags are stored in network byte order too.
+------------+------------+------------------------+
| High byte | Low byte |
+------------+------------+------------------------+
| 15 14 13 12 11 10 9 8 | 7 6 5 4 3 2 1 0 |
+------------+------------+------------------------+
| VERSION | feature bits |
+------------+-------------------------------------+
Valid version values are:
0x2 - version 2
Feature bits:
+---------+-------------+---------------------------+
| Bit 11 | mask 0x0800 | Test id present. |
+---------+-------------+---------------------------+
| Bit 10 | mask 0x0400 | Routing code present. |
+---------+-------------+---------------------------+
| Bit 9 | mask 0x0200 | Timestamp present. |
+---------+-------------+---------------------------+
| Bit 8 | mask 0x0100 | Test is 'runnable'. |
+---------+-------------+---------------------------+
| Bit 7 | mask 0x0080 | Tags are present. |
+---------+-------------+---------------------------+
| Bit 6 | mask 0x0040 | File content is present. |
+---------+-------------+---------------------------+
| Bit 5 | mask 0x0020 | File MIME type is present.|
+---------+-------------+---------------------------+
| Bit 4 | mask 0x0010 | EOF marker. |
+---------+-------------+---------------------------+
| Bit 3 | mask 0x0008 | Must be zero in version 2.|
+---------+-------------+---------------------------+
Test status gets three bits:
Bit 2 | Bit 1 | Bit 0 - mask 0x0007 - A test status enum lookup:
* 000 - undefined / no test
* 001 - Enumeration / existence
* 002 - In progress
* 003 - Success
* 004 - Unexpected Success
* 005 - Skipped
* 006 - Failed
* 007 - Expected failure
After the flags field is a number field giving the length in bytes for the
entire packet including the signature and the checksum. This length must
be less than 4MiB - 4194303 bytes. The encoding can obviously record a larger
number but one of the goals is to avoid requiring large buffers, or causing
large latency in the packet forward/processing pipeline. Larger file
attachments can be communicated in multiple packets, and the overhead in such a
4MiB packet is approximately 0.2%.
The rest of the packet is a series of optional features as specified by the set
feature bits in the flags field. When absent they are entirely absent.
Forwarding and multiplexing of packets can be done without interpreting the
remainder of the packet until the routing code and checksum (which are both at
the end of the packet). Additionally, routers can often avoid copying or moving
the bulk of the packet, as long as the routing code size increase doesn't force
the length encoding to take up a new byte (which will only happen to packets
less than or equal to 16KiB in length) - large packets are very efficient to
route.
Timestamp when present is a 32 bit unsigned integer for seconds, and a variable
length number for nanoseconds, representing UTC time since Unix Epoch in
seconds and nanoseconds.
Test id when present is a UTF-8 string. The test id should uniquely identify
runnable tests such that they can be selected individually. For tests and other
actions which cannot be individually run (such as test
fixtures/layers/subtests) uniqueness is not required (though being human
meaningful is highly recommended).
Tags when present is a length prefixed vector of UTF-8 strings, one per tag.
There are no restrictions on tag content (other than the restrictions on UTF-8
strings in subunit in general). Tags have no ordering.
When a MIME type is present, it defines the MIME type for the file across all
packets same file (routing code + testid + name uniquely identifies a file,
reset when EOF is flagged). If a file never has a MIME type set, it should be
treated as application/octet-stream.
File content when present is a UTF-8 string for the name followed by the length
in bytes of the content, and then the content octets.
If present routing code is a UTF-8 string. The routing code is used to
determine which test backend a test was running on when doing data analysis,
and to route stdin to the test process if interaction is required.
Multiplexers SHOULD add a routing code if none is present, and prefix any
existing routing code with a routing code ('/' separated) if one is already
present. For example, a multiplexer might label each stream it is multiplexing
with a simple ordinal ('0', '1' etc), and given an incoming packet with route
code '3' from stream '0' would adjust the route code when forwarding the packet
to be '0/3'.
Following the end of the packet is a CRC-32 checksum of the contents of the
packet including the signature.
Example packets
~~~~~~~~~~~~~~~
Trivial test "foo" enumeration packet, with test id, runnable set,
status=enumeration. Spaces below are to visually break up signature / flags /
length / testid / crc32
b3 2901 0c 03666f6f 08555f1b
Version 1 (and 1.1)
===================
Version 1 (and 1.1) are mostly human readable protocols.
Sample subunit wire contents
----------------------------
The following::
test: test foo works
success: test foo works
test: tar a file.
failure: tar a file. [
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
]
a writeln to stdout
When run through subunit2pyunit::
.F
a writeln to stdout
========================
FAILURE: tar a file.
-------------------
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
Subunit v1 protocol description
===============================
This description is being ported to an EBNF style. Currently its only partly in
that style, but should be fairly clear all the same. When in doubt, refer the
source (and ideally help fix up the description!). Generally the protocol is
line orientated and consists of either directives and their parameters, or
when outside a DETAILS region unexpected lines which are not interpreted by
the parser - they should be forwarded unaltered::
test|testing|test:|testing: test LABEL
success|success:|successful|successful: test LABEL
success|success:|successful|successful: test LABEL DETAILS
failure: test LABEL
failure: test LABEL DETAILS
error: test LABEL
error: test LABEL DETAILS
skip[:] test LABEL
skip[:] test LABEL DETAILS
xfail[:] test LABEL
xfail[:] test LABEL DETAILS
uxsuccess[:] test LABEL
uxsuccess[:] test LABEL DETAILS
progress: [+|-]X
progress: push
progress: pop
tags: [-]TAG ...
time: YYYY-MM-DD HH:MM:SSZ
LABEL: UTF8*
NAME: UTF8*
DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' CR UTF8-lines ']' CR
MULTIPART ::= '[ multipart' CR PART* ']' CR
PART ::= PART_TYPE CR NAME CR PART_BYTES CR
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF
unexpected output on stdout -> stdout.
exit w/0 or last test completing -> error
Tags given outside a test are applied to all following tests
Tags given after a test: line and before the result line for the same test
apply only to that test, and inherit the current global tags.
A '-' before a tag is used to remove tags - e.g. to prevent a global tag
applying to a single test, or to cancel a global tag.
The progress directive is used to provide progress information about a stream
so that stream consumer can provide completion estimates, progress bars and so
on. Stream generators that know how many tests will be present in the stream
should output "progress: COUNT". Stream filters that add tests should output
"progress: +COUNT", and those that remove tests should output
"progress: -COUNT". An absolute count should reset the progress indicators in
use - it indicates that two separate streams from different generators have
been trivially concatenated together, and there is no knowledge of how many
more complete streams are incoming. Smart concatenation could scan each stream
for their count and sum them, or alternatively translate absolute counts into
relative counts inline. It is recommended that outputters avoid absolute counts
unless necessary. The push and pop directives are used to provide local regions
for progress reporting. This fits with hierarchically operating test
environments - such as those that organise tests into suites - the top-most
runner can report on the number of suites, and each suite surround its output
with a (push, pop) pair. Interpreters should interpret a pop as also advancing
the progress of the restored level by one step. Encountering progress
directives between the start and end of a test pair indicates that a previous
test was interrupted and did not cleanly terminate: it should be implicitly
closed with an error (the same as when a stream ends with no closing test
directive for the most recently started test).
The time directive acts as a clock event - it sets the time for all future
events. The value should be a valid ISO8601 time.
The skip, xfail and uxsuccess outcomes are not supported by all testing
environments. In Python the testttools (https://launchpad.net/testtools)
library is used to translate these automatically if an older Python version
that does not support them is in use. See the testtools documentation for the
translation policy.
skip is used to indicate a test was discovered but not executed. xfail is used
to indicate a test that errored in some expected fashion (also know as "TODO"
tests in some frameworks). uxsuccess is used to indicate and unexpected success
where a test though to be failing actually passes. It is complementary to
xfail.
Hacking on subunit
------------------
Releases
========
* Update versions in configure.ac and python/subunit/__init__.py.
* Update NEWS.
* Do a make distcheck, which will update Makefile etc.
* Do a PyPI release: PYTHONPATH=python python setup.py sdist bdist_wheel; twine upload -s dist/*
* Upload the regular one to LP.
* Push a tagged commit.
git push -t origin master:master