pytest-server-fixtures

Dependencies

Maintainers

Alerts

File Explorer

Install Socket

Detect and block malicious and high-risk dependencies

Install

pytest-server-fixtures

Extensible server fixures for py.test

1.7.1

PyPI

Maintainers: 3

Readme

Pytest Server Fixtures

This library provides an extensible framework for running up real network servers in your tests, as well as a suite of fixtures for some well-known webservices and databases.

Batteries Included <#batteries-included>__
Installation <#installation>__
Configuration <#configuration>__
Common fixture properties <#common-fixture-properties>__
MongoDB <#mongodb>__
Postgres <#postgres>__
Redis <#redis>__
RethinkDB <#rethinkdb>__
S3 Minio <#s3-minio>__
Apache httpd <#apache-httpd>__
Simple HTTP Server <#simple-http-server>__
Xvfb <#xvfp>__
Jenkins <#jenkins>__
Server Framework <#server-framework>__
Integration Tests <#integration-tests>__

Batteries Included

===================================== ===================== Fixture Extra Dependency Name ===================================== ===================== MongoDB mongodb Postgres postgres Redis redis RethinkDB rethinkdb S3 Minio s3 Apache Httpd
Simple HTTP Server
Jenkins jenkins Xvfb (X-Windows Virtual Frame Buffer) ===================================== =====================

Note: v2 fixtures support launching fixtures locally, in Docker containers or as Kubernetes pods (See Configuration <#configuration>__)

Installation

Installation of this package varies on which parts of it you would like to use. It uses optional dependencies (specified in the table above) to reduce the number of 3rd party packages required. This way if you don’t use MongoDB, you don’t need to install PyMongo.

.. code:: bash

   # Install with support for just mongodb
   pip install pytest-server-fixtures[mongodb]

   # Install with support for mongodb and jenkins
   pip install pytest-server-fixtures[mongodb,jenkins]

   # Install with Docker support
   pip install pytest-server-fixtures[docker]

   # Install with Kubernetes support
   pip install pytest-server-fixtures[kubernetes]

   # Install with only core library and support for httpd and xvfp
   pip install pytest-server-fixtures

Enable the fixture explicitly in your tests or conftest.py (not required when using setuptools entry points):

.. code:: python

   pytest_plugins = ['pytest_server_fixtures.httpd',
                     'pytest_server_fixtures.jenkins',
                     'pytest_server_fixtures.mongo',
                     'pytest_server_fixtures.postgres',
                     'pytest_server_fixtures.redis',
                     'pytest_server_fixtures.rethink',
                     'pytest_server_fixtures.xvfb',
                     ]

Configuration

The fixtures are configured using the following evironment variables:

Common fixture properties

All of these fixtures follow the pattern of spinning up a server on a unique port and then killing the server and cleaning up on fixture teardown.

All test fixtures share the following properties at runtime:

MongoDB

The mongo module contains the following fixtures:

===================== ============================== Fixture Name Description ===================== ============================== mongo_server Function-scoped MongoDB server mongo_server_sess Session-scoped MongoDB server mongo_server_cls Class-scoped MongoDB server ===================== ==============================

All these fixtures have the following properties:

======== =================================================== Property Description ======== =================================================== api pymongo.MongoClient connected to running server ======== ===================================================

Here’s an example on how to run up one of these servers:

.. code:: python

def test_mongo(mongo_server): db = mongo_server.api.mydb collection = db.test_coll test_coll.insert({'foo': 'bar'}) assert test_coll.find_one()['foo'] == 'bar'

Postgres

The postgres module contains the following fixture:

======================== ============================== Fixture Name Description ======================== ============================== postgres_server_sess Session-scoped Postgres server ======================== ==============================

The Postgres server fixture has the following properties:

You may wish to build another fixture on top of the session-scoped fixture; for example:

.. code:: python

def create_full_schema(connection): """Create the database schema""" pass

@pytest.fixture(scope='session') def db_config_sess(postgres_server_sess: PostgresServer) -> PostgresServer: """Returns a DbConfig pointing at a fully-created db schema""" server_cfg = postgres_server_sess.connection_config create_full_schema(postgres_server_sess.connect()) return postgres_server_sess

Redis

The redis module contains the following fixtures:

===================== ============================ Fixture Name Description ===================== ============================ redis_server Function-scoped Redis server redis_server_sess Session-scoped Redis server ===================== ============================

All these fixtures have the following properties:

======== ====================================================== Property Description ======== ====================================================== api redis.Redis client connected to the running server ======== ======================================================

Here’s an example on how to run up one of these servers:

.. code:: python

def test_redis(redis_server): redis_server.api.set('foo': 'bar') assert redis_server.api.get('foo') == 'bar'

S3 Minio

The s3 module contains the following fixtures:

============= ================================================ Fixture Name Description ============= ================================================ s3_server Session-scoped S3 server using the ‘minio’ tool. s3_bucket Function-scoped S3 bucket ============= ================================================

The S3 server has the following properties:

The S3 Bucket has the following properties:

========== ================================== Property Description ========== ================================== name Bucket name, a UUID client Boto3 Resource from the server ========== ==================================

Here’s an example on how to run up one of these servers:

.. code:: python

def test_connection(s3_bucket): bucket = s3_bucket.client.Bucket(s3_bucket.name) assert bucket is not None

RethinkDB

The rethink module contains the following fixtures:

The server fixtures have the following properties

Here’s an example on how to run up one of these servers:

.. code:: python

def test_rethink(rethink_server): conn = rethink_server.conn conn.table_create('my_table').run(conn) inserted = conn.table('my_table').insert({'foo': 'bar'}).run(conn) assert conn.get(inserted.generated_keys[0])['foo'] == 'bar

Creating Tables


You can create tables for every test in your module like so:

.. code:: python

   FIXTURE_TABLES = ['accounts','transactions']

   def test_table_creation(rethink_module_db, rethink_make_tables):
       conn = rethink_module_db
       assert conn.table_list().run(conn) == ['accounts', 'transactions']

Emptying Databases

RehinkDb is annecdotally slower to create tables that it is to empty them (at least at time of writing), so we have a fixture that will empty out tables between tests for us that were created with the rethink_make_tables fixture above:

.. code:: python

FIXTURE_TABLES = ['accounts','transactions']

def test_put_things_in_db(rethink_module_db, rethink_make_tables): conn = rethink_module_db conn.table('accounts').insert({'foo': 'bar'}).run(conn) conn.table('transactions').insert({'baz': 'qux'}).run(conn)

def test_empty_db(rethink_empty_db): conn = rethink_empty_db assert not conn.table('accounts').run(conn) assert not conn.table('transactions').run(conn)

Apache httpd

The httpd module contains the following fixtures:

================ ================================================== Fixture Name Description ================ ================================================== httpd_server Function-scoped httpd server to use as a web proxy ================ ==================================================

The fixture has the following properties at runtime:

================= ================================== Property Description ================= ================================== document_root path.path to the document root log_dir path.path to the log directory ================= ==================================

Here’s an example showing some of the features of the fixture:

.. code:: python

def test_httpd(httpd_server): # Log files can be accessed by the log_dir property assert 'access.log' in [i.basename() for i in httpd_server.log_dir.files()]

   # Files in the document_root are accessable by HTTP
   hello = httpd_server.document_root / 'hello.txt'
   hello.write_text('Hello World!')
   response = httpd_server.get('/hello.txt')
   assert response.status_code == 200
   assert response.text == 'Hello World!'

Proxy Rules

An httpd server on its own isn’t super-useful, so the underlying class for the fixture has options for configuring it as a reverse proxy. Here’s an example where we’ve pulled in a pytest-pyramid fixture and set it up to be proxied from the httpd server:

.. code:: python

import pytest from pytest_server_fixtures.httpd import HTTPDServer

pytest_plugins=['pytest_pyramid']

@pytest.yield_fixture() def proxy_server(pyramid_server):

   # Configure the proxy rules as a dict of source -> dest URLs
   proxy_rules = {'/downstream/' : pyramid_server.url
                 }

   server = HTTPDServer(proxy_rules,
                        # You can also specify any arbitrary text you want to
                        # put in the config file
                        extra_cfg = 'Alias /tmp /var/tmp\n',
                        )
   server.start()
   yield server
   server.teardown()

def test_proxy(proxy_server): # This request will be proxied to the pyramid server response = proxy_server.get('/downstream/accounts') assert response.status_code == 200

Simple HTTP Server

The http module contains the following fixtures:

The fixture has the following properties at runtime:

================= ================================== Property Description ================= ================================== document_root path.path to the document root ================= ==================================

Here’s an example showing some of the features of the fixture:

.. code:: python

def test_simple_server(simple_http_server): # Files in the document_root are accessable by HTTP hello = simple_http_server.document_root / 'hello.txt' hello.write_text('Hello World!') response = simple_http_server.get('/hello.txt') assert response.status_code == 200 assert response.text == 'Hello World!'

Jenkins

The jenkins module contains the following fixtures:

================== ====================================== Fixture Name Description ================== ====================================== jenkins_server Session-scoped Jenkins server instance ================== ======================================

The fixture has the following methods and properties:

+-----------------------------+----------------------------------------+ | Property | Description | +=============================+========================================+ | api | jenkins.Jenkins API client | | | connected to the running server (see | | | h | | | ttps://python-jenkins.readthedocs.org) | +-----------------------------+----------------------------------------+ | load_plugins() | Load plugins into the server from a | | | directory | +-----------------------------+----------------------------------------+

Here’s an example showing how to run up the server:

.. code:: python

PLUGIN_DIR='/path/to/some/plugins'

def test_jenkins(jenkins_server): jenkins_server.load_plugins(PLUGIN_DIR) assert not jenkins_server.api.get_jobs()

Xvfb

The xvfb module contains the following fixtures:

==================== =========================== Fixture Name Description ==================== =========================== xvfb_server Function-scoped Xvfb server xvfb_server_sess Session-scoped Xvfb server ==================== ===========================

The fixture has the following properties:

=========== ============================== Property Description =========== ============================== display X-windows DISPLAY variable =========== ==============================

Here’s an example showing how to run up the server:

.. code:: python

def test_xvfb(xvfb_server): assert xvfb_server.display

Server Framework

All the included fixtures and others in this suite of plugins are built on an extensible TCP server running framework, and as such many of them share various properties and methods.

pytest_shutil.workspace.Workspace | *--base2.TestServerV2 | *--mongo.MongoTestServer *--redis.RedisTestServer *--rethink.RethinkDBServer *--base.TestServer | *--http.HTTPTestServer | *--http.SimpleHTTPTestServer *--httpd.HTTPDServer *--jenkins.JenkinsTestServer *--pytest_pyramid.PyramidTestServer

Class Methods

The best way to understand the framework is look at the code, but here’s a quick summary on the class methods that child classes of base.TestServer can override.

Class Attributes

At a minimum child classes must define run_cmd and check_server_up. There are also some class attributes that can be overridden to modify server behavior:

Constructor Arguments

The base class constructor also accepts these arguments:

+--------------+--------------------------------------------------------------+ | Argument | Description | +==============+==============================================================+ | port | Explicitly set the port number | +--------------+--------------------------------------------------------------+ | hostname | Explicitly set the hostname | +--------------+--------------------------------------------------------------+ | env | Dict of the shell environment passed to the server process | +--------------+--------------------------------------------------------------+ | cwd | Override the current working directory of the server process | +--------------+--------------------------------------------------------------+

Integration Tests

$ vagrant up $ vagrant ssh ... $ . venv/bin/activate $ cd /vagrant $ make develop $ cd pytest-server-fixtures

test serverclass="thread"

$ pytest

test serverclass="docker"

$ SERVER_FIXTURES_SERVER_CLASS=docker pytest

test serverclass="kubernetes"

$ SERVER_FIXTURES_SERVER_CLASS=kubernetes SERVER_FIXTURES_K8S_LOCAL_TEST=True pytest

Changelog

1.7.0


-  All: Support pytest >= 4.0.0
-  All: Support Python 3.7
-  pytest-server-fixtures: if host not defined on your machine, default
   to localhost
-  pytest-server-fixture: Pin to rethinkdb < 2.4.0 due to upstream API
   changes
-  pytest-verbose-parametrize: Add support for revamped marker
   infrastructure
-  pytest-verbose-parametrize: Fix integration tests to support pytest
   >= 4.1.0
-  pytest-virtualenv: Add virtualenv as install requirement. Fixes #122
-  pytest-webdriver: Fix RemovedInPytest4Warning using getfixturevalue
-  circleci: Fix checks by skipping coverall submission for developer
   without push access
-  wheels: Generate universal wheels installable with both python 2.x
   and 3.x
-  dist: Remove support for building and distributing \*.egg files
-  VagrantFile: Install python 3.7 and initialize python 3.7 by default
-  Fix DeprecationWarning warnings using “logger.warning()” function

.. _section-1:

1.6.2 (2019-02-21)

pytest-server-fixtures: suppress stacktrace if kill() is called
pytest-server-fixtures: fix random port logic in TestServerV2

.. _section-2:

1.6.1 (2019-02-12)


-  pytest-server-fixtures: fix exception when attempting to access
   hostname while server is not started

.. _section-3:

1.6.0 (2019-02-12)

pytest-server-fixtures: added previously removed TestServerV2.kill() function
pytest-profiling: pin more-itertools==5.0.0 in integration tests, as that’s a PY3 only release

.. _section-4:

1.5.1 (2019-01-24)


-  pytest-verbose-parametrize: fixed unicode parameters when using
   ``@pytest.mark.parametrize``

.. _section-5:

1.5.0 (2019-01-23)

pytest-server-fixtures: made postgres fixtures and its tests optional, like all other fixtures
pytest-server-fixtures: reverted a fix for pymongo deprecation warning, as this will break compatibility with pymongo 3.6.0
pytest-server-fixtures: dropped RHEL5 support in httpd

.. _section-6:

1.4.1 (2019-01-18)


-  pytest-server-fixtures: server fixture binary path specified in ENV
   now only affect server class ‘thread’

.. _section-7:

1.4.0 (2019-01-15)

Fixing python 3 compatibility in Simple HTTP Server fixture
Fixed broken tests in pytest-profiling
Pinned pytest<4.0.0 until all deprecation warnings are fixed.
pytest-webdriver: replaced deprecated phantomjs with headless Google Chrome.
Add Vagrantfile to project to make test environment portable.
Add .editorconfig file to project.
pytest-server-fixtures: add TestServerV2 with Docker and Kubernetes support.
pytest-server-fixtures: fix for an issue where MinioServer is not cleaned up after use.
pytest-server-fixtures: fix deprecation warnings when calling pymongo.
pytest-server-fixtures: close pymongo client on MongoTestServer teardown.
pytest-server-fixtures: upgrade Mongo, Redis and RethinkDB to TestServerV2.
coveralls: fix broken coveralls

.. _section-8:

1.3.1 (2018-06-28)


-  Use pymongo list_database_names() instead of the deprecated
   database_names(), added pymongo>=3.6.0 dependency

.. _section-9:

1.3.0 (2017-11-17)

Fixed workspace deletion when teardown is None
Fixed squash of root logger in pytest-listener
Added S3 Minio fixture (many thanks to Gavin Bisesi)
Added Postgres fixture (many thanks to Gavin Bisesi)
Use requests for server fixtures http gets as it handles redirects and proxies properly

.. _section-10:

1.2.12 (2017-8-1)


-  Fixed regression on cacheing ephemeral hostname, some clients were
   relying on this. This is now optional.

.. _section-11:

1.2.11 (2017-7-21)

Fix for OSX binding to illegal local IP range (Thanks to Gavin Bisesi)
Setup and Py3k fixes for pytest-profiling (Thanks to xoviat)
We no longer try and bind port 5000 when reserving a local IP host, as someone could have bound it to 0.0.0.0
Fix for #46 sourcing gprof2dot when the local venv has not been activated

.. _section-12:

1.2.10 (2017-2-23)


-  Handle custom Pytest test items in pytest-webdriver

.. _section-13:

1.2.9 (2017-2-23)
~~~~~~~~~~~~~~~~~

-  Add username into mongo server fixture tempdir path to stop
   collisions on shared multiuser filesystems

.. _section-14:

1.2.8 (2017-2-21)
~~~~~~~~~~~~~~~~~

-  Return function results in shutil.run.run_as_main

.. _section-15:

1.2.7 (2017-2-20)
~~~~~~~~~~~~~~~~~

-  More handling for older versions of path.py
-  Allow virtualenv argument passing in pytest-virtualenv

.. _section-16:

1.2.6 (2017-2-16 )

Updated devpi server server setup for devpi-server >= 2.0
Improvements for random port picking
HTTPD server now binds to 0.0.0.0 by default to aid Selenium-style testing
Updated mongodb server args for mongodb >= 3.2
Corrections for mongodb fixture config and improve startup logic
Added module-scoped mongodb fixture
Handling for older versions of path.py
Fix for #40 where tests that chdir break pytest-profiling

.. _section-17:

1.2.5 (2016-12-09)


-  Improvements for server runner host and port generation, now supports
   random local IPs
-  Bugfix for RethinkDB fixture config

.. _section-18:

1.2.4 (2016-11-14)

Bugfix for pymongo extra dependency
Windows compatibility fix for pytest-virtualenv (Thanks to Jean-Christophe Fillion-Robin for PR)
Fix symlink handling for pytest-shutil.cmdline.get_real_python_executable

.. _section-19:

1.2.3 (2016-11-7)


-  Improve resiliency of Mongo fixture startup checks

.. _section-20:

1.2.2 (2016-10-27)

Python 3 compatibility across most of the modules
Fixed deprecated Path.py imports (Thanks to Bryan Moscon)
Fixed deprecated multicall in pytest-profiling (Thanks to Paul van der Linden for PR)
Added devpi-server fixture to create an index per test function
Added missing licence file
Split up httpd server fixture config so child classes can override loaded modules easier
Added ‘preserve_sys_path’ argument to TestServer base class which exports the current python sys.path to subprocesses.
Updated httpd, redis and jenkins runtime args and paths to current Ubuntu spec
Ignore errors when tearing down workspaces to avoid race conditions in ‘shutil.rmtree’ implementation

.. _section-21:

1.2.1 (2016-3-1)


-  Fixed pytest-verbose-parametrize for latest version of py.test

.. _section-22:

1.2.0 (2016-2-19)

New plugin: git repository fixture

.. _section-23:

1.1.1 (2016-2-16)


-  pytest-profiling improvement: escape illegal characters in .prof
   files (Thanks to Aarni Koskela for the PR)

.. _section-24:

1.1.0 (2016-2-15)

New plugin: devpi server fixture
pytest-profiling improvement: overly-long .prof files are saved as the short hash of the test name (Thanks to Vladimir Lagunov for PR)
Changed default behavior of workspace.run() to not use a subshell for security reasons
Corrected virtualenv.run() method to handle arguments the same as the parent method workspace.run()
Removed deprecated ‘–distribute’ from virtualenv args

.. _section-25:

1.0.1 (2015-12-23)


-  Packaging bugfix

.. _section-26:

1.0.0 (2015-12-21)

Initial public release

FAQs

What is pytest-server-fixtures?

Is pytest-server-fixtures well maintained?

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

pytest-server-fixtures

Pytest Server Fixtures

Table of Contents

Batteries Included

Installation

Configuration

Common fixture properties

MongoDB

Postgres

Redis

S3 Minio

RethinkDB

Apache httpd

Proxy Rules

Simple HTTP Server

Jenkins

Xvfb

Server Framework

Class Methods

Class Attributes

Constructor Arguments

Integration Tests

test serverclass="thread"

test serverclass="docker"

test serverclass="kubernetes"

Changelog

Related posts

pytest-server-fixtures

Pytest Server Fixtures

Table of Contents

Batteries Included

Installation

Configuration

Common fixture properties

MongoDB

Postgres

Redis

S3 Minio

RethinkDB

Apache httpd

Proxy Rules

Simple HTTP Server

Jenkins

Xvfb

Server Framework

Class Methods

Class Attributes

Constructor Arguments

Integration Tests

test serverclass="thread"

test serverclass="docker"

test serverclass="kubernetes"

Changelog

Related posts

Python Software Foundation Announces 5-Year Sponsorship Commitment from Fastly

LDAPjs Open Source Project Decommissioned After Maintainer Receives Abusive Email