This repository contains the source code of the Opsani Servo agent.
The servo connects applications to the Opsani cloud optimization engine to identify cost savings and performance enhancements by applying machine learning technology. Servos are lightweight Python applications and are typically deployed as a container under an orchestration layer such as Kubernetes, ECS, or Docker Compose.
Servos are composed of connectors, which provide the core functionality for integrating with metrics, orchestration, and load generation systems/utilities. The ServoX codebase provides core functionality shared by all servos and a rich library supporting the development of connectors.
ServoX is a modern Python application distributed as an installable Python package. Development is done in a Python install managed with Pyenv and a virtual environment managed by Poetry. This is the path of least resistance but any Python package management system should work.
git clone git@github.com:opsani/servoxcd servox && pyenv installcurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | pythonpoetry env use `cat .python-version` poetry installpoetry shellservo initservo --helpAccess to an Opsani optimizer is required to deploy the servo and run the end to end integration tests. If you do not currently have access to an Opsani environment but are otherwise interested in working with the optimizer and Servo, please reach out to us at info@opsani.com and we will get back with you.
❯ servo --help
NOTE: A Dotenv file is recommended during development to keep the CLI options under control. The init command will generate one for you.
❯ servo init
# Generate a config file with all available connectors
❯ servo generate
# Validate a config file
❯ servo validate
# Display config schema
❯ servo schema
# Display all available connectors
❯ servo connectors
# Display instance specific info (requires configuration)
❯ servo show connectors
❯ servo show events
❯ servo show components
❯ servo show metrics
# Check servo readiness
❯ servo run --dry-run
# Describe application state
❯ servo describe
# Capture and display a measurement
❯ servo measure
# Adjust the memory of web-server to 512MB and cpu of gateway to 800 millicores
❯ servo adjust web.mem=512 gateway.cpu=0.8
# Run the servo to start optimizing
❯ servo run
ServoX has been designed to provide a delightful experience for engineers integrating cloud optimization into their systems and workflow. Developer ergonomics and operator efficiency are primary concerns as integrating and orchestrating disparate components can quickly become tiresome and complex. As a library, ServoX aspires to be as "batteries included" as possible and support developers with well designed, implemented, and tested solutions for common concerns. As a tool, ServoX strives to support system operators and devops engineers with strong support for common tasks and a high-velocity workflow.
There are a few key components that form the foundation of the architecture:
Connector base class.
Connectors can process events dispatched from the servo (see Events below),
provide services to the user (see CLI below), and interact with other
connectors.DESCRIBE, MEASURE, ADJUST, and PROMOTE which correspond to declaring
the metrics & components that the connector is interested in, taking
measurements and returning aggregated scalar or time series data points,
making changes to the application under optimization, or promoting an
optimized configuration to the broader system.servo run --check CLI command,
or by themselves via servo run --dry-run The design of the checks subsystem
is covered in depth in the docs.servo schema will emit a complete JSON Schema
document for the assembly while servo schema kubernetes will emit a JSON
Schema specific to the Kubernetes connector.The servo is built around an event driven architecture and utilizes a lightweight eventing system to pass messages between connectors. Eventing is implemented in asynchronous Python on top of asyncio. Events are simple strings identifiers that are bound to a Python inspect.Signature object. The signature is used when event handlers are registered to enforce a contract around the parameter and return types, method arity, etc.
Any connector can define an event provided that no other connector has already
registered an event with the desired name. The Servo class defines several
basic events covering essential functionality such as declaring metrics and
components, capturing measurements, and performing adjustments. These events are
integrated into the CLI and readily accessible. For example, executing servo show metrics dispatches the metrics event to all active connectors and
displays the aggregate results in a table. More substantial commands such as
measure and adjust work in the same manner -- dispatching events and
visualizing the results. Whenever possible functionality is implemented via
eventing.
When the servo is run, it connects to the Opsani optimizer via the API and receives instructions about which actions to take to facilitate the optimization activity. These commands are dispatched to the connectors via events.
The default events are available on the servo.servo.Events enumeration and
include:
| Event | Category | Description |
|---|---|---|
| startup | Lifecycle | Dispatched when the servo is assembled. |
| shutdown | Lifecycle | Dispatched when the servo is being shutdown. |
| metrics | Informational | Dispatched to gather the metrics being measured. |
| components | Informational | Dispatched to gather the components & settings available for adjustment. |
| check | Operational | Asks connectors to check if they are ready to run by validating settings, etc. |
| describe | Operational | Gathers the current state of the application under optimization. |
| measure | Operational | Takes a measurement of target metrics and reports them to the optimizer. |
| adjust | Operational | Applies a change to the components/settings of the application under optimization and reports status to the optimizer. |
Event handlers are easily registered via a set of method decorators available on
the servo.connector module. Handlers are registered with a preposition which
determines if it is invoked before, on, or after the event has been processed.
Handlers are invoked when the servo or another connector dispatches an event.
Event handlers can be implemented either synchronously or asynchronously
depending on if the method is a coroutine declared with the async prefix.
from typing import List
import servo
class SomeConnector(servo.BaseConnector):
@servo.before_event('measure')
def notify_before_measure(self) -> None:
self.logger.info("We are about to measure...")
@servo.on_event('metrics')
def metrics(self) -> List[servo.Metric]:
return [servo.Metric('throughput', servo.Unit.REQUESTS_PER_MINUTE)]
@servo.after_event('adjust')
def analyze_results(self, results: List[servo.EventResult]) -> None:
self.logger.info(f"We got some results: {results}")
Each preposition has different capabilities available to it. Before event
handlers can cancel the execution of the event by raising a EventCancelledError.
On event handlers can return results that are aggregated and available for
processing. After event handlers get access to all of the results returned by
active connectors via the on event handlers.
Events can be created either programmatically via the Connector.create_event()
class method or declaratively via the event() decorator:
import servo
class EventExample(servo.BaseConnector):
@servo.event()
async def trace(self, url: str) -> str:
...
The event decorator uses the parameters and return type of the decorated
method to define the signature requirements for on event handlers registered
against the event. The body of the decorated method must be ..., pass, or an
async generator that yields None exactly once.
The body of the decorated method can be used to define setup and tear-down activities around on event handlers. This allows for common setup and tear-down functionality to be defined by the event creator. This is achieved by implementing the body of the decorated method as an async generator that yields control to the on event handler:
from typing import AsyncIterator
import servo
class SetupAndTearDownExample(servo.BaseConnector):
@servo.event()
async def trace(self, url: str) -> AsyncIterator[str]:
print("Entering event handler...")
yield
print("Exited event handler.")
The event decorator can also be used to create an event and register a handler
for the event at the same time. In this example, the handler=True keyword
argument is provided to created the event and register the decorated method
as an on event handler for the new event.
import servo
class AnotherConnector(servo.BaseConnector):
@servo.event('load_test', handler=True)
async def run_load_test(self, url: str, duration: int = 60) -> str:
return "Do something..."
Once an event is created, the connector can dispatch against it to notify other connectors of changes in state or to request data from them.
Connectors can notify or interact with other connectors by dispatching an event. When the servo is assembled, an event bus is transparently established between the connectors to facilitate event driven interaction.
from typing import List
import servo
class ExampleConnector(servo.BaseConnector):
async def do_something(self) -> None:
# Gather metrics from other connectors
results: List[servo.EventResult] = await self.dispatch_event("metrics")
for result in results:
print(f"Gathered metrics: {result.value}")
Pay attention to the output of servo --help and servo schema to identify
environment variables that can be used for configuration. The servo handles
configuration of deeply nested attributes by building the environment variable
mapping on the fly at assembly time.
For convenience, the servo CLI utility automatically supports .env files for
loading configuration and is already in the .gitignore. Interacting with the
CLI is much cleaner if you drop in a dotenv file to avoid having to deal with
the options to configure the optimizer. The servo init command will help set
this up interactively.
The servo base library provides logging services for all connectors and core
components. By default, the servo CLI utility runs at the INFO log level
which is designed to provide balanced output that informs you of what is
happening operationally without becoming overwhelming and pedantic. During
development, debugging, or troubleshooting it may become desirable to run at an
elevated log level. The log level can be set via a commandline option on the
servo utility or via the SERVO_LOG_LEVEL environment variable. The servo
will emit ANSI colored output to stderr when the terminal is a TTY. Coloring
can be suppressed via the --no-color commandline option or with the
SERVO_NO_COLOR or NO_COLOR environment variables.
-l, --log-level [TRACE|DEBUG|INFO|SUCCESS|WARNING|ERROR|CRITICAL]
Set the log level [env var:
SERVO_LOG_LEVEL; default: INFO]
--no-color Disable colored output [env var:
SERVO_NO_COLOR, NO_COLOR]
By default, log messages are written to stderr and a file sink at the logs/
subdirectory relative to the servo root. Backtraces are preformatted and as much
context as possible is provided when an exception is logged.
The servo.logging module exposes some interesting functionality for operators
and developers alike. Logging is aware of the eventing subsystem and will
automatically attribute log messages to the currently executing connector and
event context. Long running operations can be automatically reported to the
Opsani API by including a progress key with a numeric percentage value ranging
from 0.0 to 100.0. There are several function decorators available that can
provide automatic logging output for entry and exit, execution timing, etc.
Dependent libraries such as backoff have been configured to emit their logs
into the servo logging module. Every component that has logging support is
intercepted and handled by the logging subsystem and conforms to the log levels
outlined above.
Connectors are set up to be auto-discovered using the setuptools entry point functionality available from the Python standard library. When a new connector is installed into the assembly, it will be automatically discovered and become available for interaction.
The specific of how this mechanism works is discussed in detail on the Python Packaging Guide.
The bundled connectors are registered and discovered using this mechanism via
entries in the pyproject.toml file under the
[tool.poetry.plugins."servo.connectors"] stanza.
If you're writing your own connector in an external package, you need to include
[tool.poetry.plugins."servo.connectors"]
"my_connector" = "my_project.foo:MyConnector"
in your pyproject.toml to add your connector as an entry point.
and you must name your connector class MyConnector(servo.BaseConnector): to have it discoverable.
Discovery is performed via the servo.connector:_name_for_connector_class() function. It matches
the My in the connector class name to the top level connector key in the servo.yaml.
So, for example, your servo.yaml could be as follows:
...
my:
foo: bar
baz: bat
...
ServoX is designed to support assemblies that contain an arbitrary number of connectors that may or may not be active and enable the use of multiple instances of a connector with different settings. This introduces a few modes of configuration.
The servo looks to a connectors configuration key that explicitly declares
which connectors are active within the assembly. If a connectors key is not
present in the config file, then all available connectors become optionally
available based on the presence or absence of their default configuration key.
For example, an assembly that includes New Relic, Datadog, and SignalFX
connectors installed as Python packages with the following configuration would
only activate Datadog due to the presence of its configuration stanza:
datadog:
setting_1: some value
setting_2: another value
This mode supports the general case of utilizing a small number of connectors in "off the shelf" configurations.
From time to time, it may become necessary to connect to multiple instances of a
given service -- we have seen this a few times with Prometheus in canary mode
deployments where metrics are scattered across a few instances. In these cases,
it can become necessary to explicitly alias a connector and utilize it under two
or more configurations. In such cases, the connectors key becomes required in
order to disambiguate aliases from configuration errors. In such cases, the
connectors key can be configured as a dictionary where the key identifies the
alias and the value identifies the connector:
connectors:
prom1: prometheus
prom2: prometheus
prom1:
setting_1: some value
setting_2: another value
prom2:
setting_1: some value
setting_2: another value
It is also possible to utilize the connectors key in order to exclude
connectors from the active set. This can be done with the dictionary syntax
referenced above or using an array syntax if aliasing is not being utilized. For
example, given a configuration with New Relic and Prometheus active but some
sort of issue warranting the isolation of Prometheus from the active set, the
config file might be configured like:
connectors:
- new_relic
prometheus:
setting_1: some value
setting_2: another value
new_relic:
setting_1: some value
setting_2: another value
These configurations can be generated for you by providing arguments to servo generate. A space delimited list of connector names will explicitly populate
the connectors key and a syntax of alias:connector will configure aliases:
❯ servo generate foo:vegeta bar:kubernetes
bar:
description: Update the namespace, deployment, etc. to match your Kubernetes cluster
namespace: default
connectors:
bar: kubernetes
foo: vegeta
foo:
description: Update the rate and target/targets to match your load profile
duration: 5m
rate: 50/1s
target: https://example.com/
Generated servo.yaml
ServoX is capable of running multiple servos within the same assembly and servos can be added and removed dynamically at runtime. This is useful for optimizing several applications at one time from a single servo deployment to simplify operations or more interestingly to support the integration and automation of optimization into CI and CD pipelines. For example, it is possible to configure the build system to trigger optimization for apps as they head into staging or upon emergence into production.
Multi-servo execution mode is straightforward. When the servo.yaml config file
is found to contain multiple documents (delimited by ---), a servo instance is
constructed for each entry in the file and added to the assembly. There are
however a few differences in configuration options.
When multi-servo mode is enabled, the --optimizer, --token, --token-file,
--base-url, and --url options are unavailable. The optimizer and
connectivity configuration must be provided via the optimizer stanza within
each configuration document in the config file. The CLI will raise errors if
these options are utilized with a multi-servo configuration because they are
ambiguous. This does not preclude a single servo being promoted into a
multi-servo configuration at runtime -- it is a configuration resolution
concern.
When running multi-servo, logging is changed to provide context about the servo
that is active and generating the output. The servo.current_servo() method
returns the active servo at runtime.
Because ServoX is based on asyncio and functions as an orchestrator, it is
capable of managing a large number of optimizations in parallel (we have tested
into the thousands). Most operations performed are I/O bound and asynchronous
but the specifics of the connectors used in a multi-servo configuration will
have a significant impact on the upper bounds of concurrency.
Basically all that you need to do is use the --- delimiter to create multiple
documents within the servo.yaml file and configure an optimizer within
each one. For example:
---
optimizer:
id: newco.com/awesome-app1
token: 00000000-0000-0000-0000-000000000000
connectors: [vegeta]
vegeta:
duration: 5m
rate: 50/1s
target: GET https://app1.example.com/
---
optimizer:
id: newco.com/awesome-app2
token: 00000000-0000-0000-0000-000000000000
connectors: [vegeta]
vegeta:
duration: 5m
rate: 50/1s
target: GET https://app2.example.com/
Servos can be added and removed from the assembly at runtime via methods on the
servo.Assembly class:
import servo
assembly = servo.current_assembly()
new_servo = servo.Servo()
assembly.add_servo(new_servo)
assembly.remove_servo(new_servo)
Should your connector wish to expose additional commands to the CLI, it can do
so via the ConnectorCLI class. Instances are automatically registred with the
CLI and the Context is configured appropriately when commands are invoked. All
CLI extensions are namespaced as subcommands to keep things tidy and avoid
naming conflicts.
A simple example of a CLI extension that will register servo vegeta attack is:
import servo
import servo.cli
import servo.connectors.vegeta
cli = servo.cli.ConnectorCLI(servo.connectors.vegeta.VegetaConnector, help="Load testing with Vegeta")
@cli.command()
def attack(context: servo.cli.Context):
"""
Run an adhoc load generation
"""
context.connector.measure()
ServoX is implemented in Python and supported by a handful of excellent libraries from the Python Open Source community. Additional dependencies in the form of Python packages or system utilities are imposed by connectors (see below).
Open Source contributions are always appreciated. Should you wish to get involved, drop us a line via GitHub issues or email to coordinate efforts.
It is expected that most Open Source contributions will come in the form of new connectors. Should you wish to develop a connector, reach out to us at Opsani as we have connector developer guides that are in pre-release while ServoX matures.
The core development team typically works in VSCode. Poetry and VSCode have not quite yet become seamlessly integrated. For your convenience, there are a couple of Makefile tasks that can simplify configuration:
make init - Initialize a Poetry environment, configure .vscode/settings.json,
and then run the `servo initialize command.make vscode - Export the Poetry environment variables and then open the
local working copy within VSCode. The built-in terminal and Python extension
should auto-detect the Poetry environment and behave.The project is configured with a pre-commit hook to enforce as much of the coding standards and style guide as possible. To install it into your working copy, run:
poetry run pre-commit install
When developing against dependencies or building out new connectors, it can be useful to utilize a local package so that development can be done in the dependency and within the servox package at the same time.
To do so, utilize Poetry Path based dependencies with the develop = true flag
by adding a path reference to the package into the to tool.poetry.dev-dependencies
stanza of the pyproject.toml file, and then run poetry update [dependency name].
For example, if developing on servox and the statesman state machine library in a working copy in the parent directory, you would add:
[tool.poetry.dev-dependencies]
# ...
statesman = {path = "../statesman", develop = true}
And then run poetry update statesman.
Changes made to the dependency are immediately visible to servox, making it easy to develop across package boundaries efficiently.
The project is structured to support and enforce consistent, idiomatic code. A number of tools have been integrated to provide linting and automatic reformatting when possible/appropriate.
This functionality is exposed via tasks in the Makefile, Git commit hooks, and
GitHub Actions.
During local development, you may wish to invoke these tools as follows:
make lint - Run the linters and output actionable feedback for conformance
with the engineering standards and style of the project.make format - Automatically apply changes to the working copy to conform
with project standards where possible (e.g., reformatting imports).make pre-commit - Run all pre-commit hooks in the manual hook stage. There
are a number of standards that are currently soft enforced in the interest of
pragmatism. Eventually the set of pre-commit and lint checks will form a
perfect union but we are not there yet. Most hooks set in manual mode have to
do with fit and finish concerns such as docstring tone/formatting.make test - Execute the test suite and generate a code coverage report.Linting is automatically performed when a new branch is pushed or a PR is opened and conformance can be remediated during post-implementation code review.
Optimizer API and authentication can configured be inline with your config yaml (servo.yaml)
optimizer:
base_url: some_url
id: org.domain/app
...
These details can also be configured via environment variables as highlighted in the following sections. Note these sections are mutually exclusive: Opsani API refers to Opsani managed endpoints wheras the Appdynamics API refers to upcoming FSO Optimization. Which API type is used is dynamically derived during configuration/env var parsing by the pydantic configuration model (see BaseServoConfiguration.optimizer).
By default, the servo will connect to the primary Opsani API hosted on https://api.opsani.com. This behavior can be overridden via CLI options and environment variables:
| Config Option | Environment Variable | Description | Example |
|---|---|---|---|
optimizer.base-url | OPSANI_BASE_URL | Sets an alternate base URL. The path is computed using the provided optimizer id. Useful for staging deployments and integration tests. | OPSANI_BASE_URL=https://staging-api.opsani.com:3456 servo run |
optimizer.url | OPSANI_URL | Sets an explicit URL target. When given, the base URL is ignored and paths are not computed. Useful in unit tests and workstation development with a partial server stack. | OPSANI_URL=http://localhost:1234 servo run |
optimizer.id | OPSANI_ID | Identifier of the application as provided by the Opsani optimizer provisioned for your application | OPSANI_ID=newco.com/awesome-app1 servo run |
optimizer.token | OPSANI_TOKEN | Authentication token as provided by the Opsani optimizer provisioned for your application | OPSANI_TOKEN=00000000-0000-0000-0000-000000000000 servo run |
| Config Option | Environment Variable | Description | Example |
|---|---|---|---|
optimizer.base-url | APPD_BASE_URL | Sets an alternate base URL. The path is computed using the provided workload and tenant ids. | APPD_BASE_URL=https://optimize-ignite-test.saas.appd-test.com/ servo run |
optimizer.url | APPD_URL | Sets an explicit URL target. When given, the base URL is ignored and paths are not computed. Useful in unit tests and workstation development with a partial server stack. | APPD_URL=http://localhost:1234 servo run |
optimizer.tenant_id | APPD_TENANT_ID | Identifier of the application tenant | APPD_TENANT_ID=00000000-0000-0000-0000-000000000000 servo run |
optimizer.optimizer_id | APPD_OPTIMIZER_ID | Base64 encoded identifier of the workload to be optimized as provided by the provisioned optimizer tenant | APPD_OPTIMIZER_ID=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==== servo run |
optimizer.client_id | APPD_CLIENT_ID | Authentication client ID as provided by the Opsani optimizer provisioned for your application | APPD_CLIENT_ID=<client id text> servo run |
optimizer.client_secret | APPD_CLIENT_SECRET | Authentication client secret as provided by the Opsani optimizer provisioned for your application | APPD_CLIENT_SECRET=<client secret text> servo run |
Dockerfile and docker-compose.yaml configurations are available in the
repository and have been designed to support both development and deployment
workflows. Configuration file mounts and environment variables can be used to
influence the behavior of the servo within the container.
The SERVO_ENV build argument controls the target environment for the built
image. Building with SERVO_ENV=production excludes development packages from
the image to reduce size and build time.
Pre-built Docker images are available on opsani/servox on Docker Hub. The documentation for these images is available within this repository at docs/README-DOCKER_HUB.md.
The latest release version is available under the opsani/servox:latest tag.
The main development branch is published as the opsani/servox:edge tag.
Docker images are built and published to Docker Hub via the
Docker GitHub Actions Workflow. The workflow
builds branches, published releases, and the main integration branch. Pull
Requests are not published to Docker Hub because as a publicly available
repository it could become an attack vector.
Git branches and Docker images have differing naming constraints that impact how
tag names are computed. For example, Docker tags cannot contain slashes, which
is a common practice for namespacing branches and tags in Git. As such, slashes
are converted to hyphens when computing tag names for branches and tags. The
full naming constraints on Docker image tags is covered in the docker tag
documentation.
Pre-built images are built using BuildKit and can be used as the basis for very fast customized builds:
❯ DOCKER_BUILDKIT=1 docker build -t servox --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from opsani/servox:edge .
After changing Python interpreter versions you may find that you are "stuck" in the existing virtual environment rather than your new desired version.
The problem is that Poetry is linked against the previous environment and needs a nudge to select the new interpreter.
The project is bound to a local Python version via the .python-version file.
Tell Poetry to bind against the locally selected environment via:
poetry env use `cat .python-version`
When upgrading between point releases of the Python interpreter, you may need
to tear down and recreate your venv:
poetry env remove `cat .python-version` && poetry install
Tests are implemented using pytest and
live in the tests subdirectory. Tests can be executed directly via the
pytest CLI interface (e.g., pytest tests) or via make test, which
will also compute coverage details.
ServoX is developed with a heavily test-driven workflow and philosophy. The framework strives to provide as much support for robust testing as possible and make things that you would think are very hard to test programmatically very simple. You will want to familiarize yourself with what is available, there are tools that can dramatically accelerate your development.
ServoX divides the test suite into three types: Unit, Integration, and System tests.
Tests are identified within the suite in two ways:
Unit tests are the default type and make up the bulk of the suite. They either exercise code that carries no outside dependencies or utilize isolation techniques such as mocks and fakes. They are fast and highly effective for validating defined behavior and catching regressions.
Integration tests do not interact with an Opsani Optimizer but do interact with external systems and services such as Kubernetes and Prometheus. It is common to utilize a local environment (such as Docker, Compose, kind, or Minikube) or a dedicated cloud instance to host the services to interact with. But the focus of the tests are on implementing and verifying the correct behaviors in a supportive environment.
System tests are much like integration tests except that theyn are highly prescriptive about the environment they are runnuing in and interact with a real optimizer backend as much as is practical. System tests sit at the top of the pyramid and it is expected that there are comparatively few of them, buit they deliver immense value late in a development cycle when code correctness has been established and deployment environments and compatibility concerns come to the forefront.
All test types can be implemented within any test module as appropriate by annotating the tests with pytest markers:
import pytest
@pytest.mark.integration
class TestSomething:
...
Tests without a type mark are implicitly designated as unit tests for convenience. When multiple type marks are in effect due to hierarchy, the closest mark to the test node has precedence.
There are also dedicated directories for integration and system tests at
tests/integration and tests/system respectively. These dedicated directories
are home to cross cutting concerns and scenarios that do not clearly belong to a
specific module. Tests in these directories that are marked with other types
will trigger an error.
A key part of any testing workflow is, well, running tests. pytest provides a wealth of capabilities out of the box and these have been further augmented with custom pytest-plugins within the suite.
Let's start with the basics: executing pytest standalone will execute the unit
tests in tests. Running pytest --verbose (or -v) will provide a one test per
line output which can be easier to follow in slower runs.
Individual files can be run by targetting them by name: pytest tests/connector_test.py. Individual test functions and container test classes
within the file (known as nodes in pytest parlance) can be addressed with the
tests/path/to/test_something.py::TestSomething::test_name syntax.
Tests can also be flexibly selected by marks and naming patterns. Invoking
pytest -m asyncio -k sleep will select all tests with the asyncio mark and
have the word "sleep" in their name. These arguments support a matching syntax,
look into the pytest docs for details.
The ServoX test types have specific affordances exposed through pytest. Running
pytest -T integration will select and run all integration tests, but deselect
all other types. The -T also known as --type flag supports stem matching for
brevity: pytest -T u will select the unit tests.
Because they are slow and require supplemental configuration, integration and
system tests are skipped by default. They can be enabled via the
-I, --integration and -S, --system switches, respectively. Note the difference
in behavior between the flags: pytest -I -S will result in all the tests being
selected for run whereas pytest -T sys targets the system tests exclusively.
Once the -I, -S flags have been used to enable the tests, they can be further
filtered using -m and -k. If you are thoughtful about how you name your
tests and leverage marks when it makes sense, it can become very easy to run
interesting subsets of the suite.
By default, pytest uses output buffering techniques to capture what is written
to stdout and stderr. This can become annoying if you are trying to introspect
state, print debugging info, or get a look at the servo logs. You can suppress
output capture via the -s switch. This is typically only recommended when
running a small number of tests because the output quickly becomes
incomprehensible.
If you are working through a set of failures, you can rerun the tests that
failed on the last run via the --lf, --last-failed flag. The --ff, --failed-first flag will rerun all of the tests, but run the previously failed
tests first. Similarly, --nf, --new-first will run the full suite but
prioritize new files. The pytest-picked plugin provides additional targeting
based on git working copy status -- running pytest --picked will find unstaged
files to run.
Finally, the --sw, --stepwise and --sw-skip, --stepwise-skip flags allow you
to methodically working through a stack of failures by resuming from your last
failure and then halting at the next one.
Test automation tasks are centralized into the Makefile. There are a number of
testing tasks availble including:
make test - Run all available tests.make test-unit - Run unit tests.make test-integration - Run integration tests.make test-system - Run system tests.make test-coverage - Run all available tests and generate code coverage
report.make test-kubeconfig - Generate a kubeconfig file at tests/kubeconfig. See
details in Integration Testing below.make autotest - Automatically run tests based on filesystem changes.Testing tasks will run in subprocess distributed mode by default (see below).
The test suite includes support for integration tests for running tests against
remote system components such as a Kubernetes cluster or Prometheus deployment.
Integration tests require a kubeconfig
file at tests/kubeconfig.
By convention, the default integration testing cluster is named kubetest
and the make test-kubeconfig task is provided to export the cluster details
from your primary kubeconfig, ensuring isolation.
Interaction with the Kubernetes cluster is supported by the most excellent kubetest library that provides fixtures, markers, and various testing utilities on top of pytest.
To run the integration tests, execute pytest -I to enable the
marker. Integration tests are much slower than the unit test suite
and should be designed to balance coverage and execution time.
System tests are enabled by running pytest -S. Systems tests are very similar
to integration tests in implementation and performance, but differ in that they
are prescriptively bound to particular deployment environments and interact with
an actual
Tests can also be run in cluster by packaging a development container and deploying it. The testing harness will detect the in-cluster state and utilize the active service account.
The project is configured to run CI on unit tests for all branches, tags, and pull requests opened against the repository. CI for integration and system tests is constrained by a few rules because they are so resource intensive and may be undesirable during experimental developmenmt or integrating multiple branches together.
Integration and system tests are run if any of the following conditions are met:
main.release/.bugfix/.#test:integration and/or
#test:system.The default unit test job that is executed for all pushes generates code coverage reports and XML/HTML report artifacts that are attached to the run. The integration and system test jobs report on the runtime duration of the tests to help identify and manage runtime creep.
The unit, integration, and system test jobs all utilize the pytest-xdist
plugin to split the test suite up across a set of subprocesses. This is
discussed in the Distributed Testing section.
Docker images are built and pushed to Docker Hub automatically for all
pushed refs. Release tags are handled automatically and the
opsani/servox:latest tag is advanced when a new version is released. The
main branch is built and pushed to the opsani/servox:edge tag.
The project is configured to leverage locally distributed test execution by
default. Servo workloads are heavily I/O bound and spend quite a bit
of time awaiting data from external services. This characteristic makes tests
inherently slower but also makes them very well suited for parallel execution.
The Makefile tasks and GitHub actions are configured to leverage a subprocess
divide & conquer strategy to speed things up.
This functionality is provided by pytest-xdist.
All manifests loaded through kubetest support Mustache templating.
A context dictionary is provided to the template that includes references to all
Kubernetes resources that have been loaded at render time. The namespace and
objs are likely to be the most interesting.
Just for fun, ServoX generates an ASCII art banner at startup. The font is randomized from a painstakingly selected set of style that represent the ServoX vibe. The output color is then randomized and has a 50/50 shot of being rendered as a rainbow or a randomly selected flat output color.
Don't like serendipity?
You can take control of the banner output with two environment variables:
SERVO_BANNER_FONT: Name of the Figlet
font to render the banner in.SERVO_BANNER_COLOR: The color to use when rendering the banner. Valid
options are:
REDGREENYELLOWBLUEMAGENTACYANRAINBOWpoetry version [major | minor | patch] to increment the version. This
will update the version number as version in section [tool.poetry] of
the pyproject.toml file.__cryptonym__ in
servo/__init__.py.[x.y.z] "cryptonym" - YYYY-MM-DD.v0.9.5) again the main branch.vX.Y.Z "<cryptonym>"release.yml
workflow. The framework is packaged into a Python library and published to
PyPi and Docker images are built and
published to
Docker Hub.vX.Y.Z, has been published in Docker
Hub.Please reach out to support at opsani.com.
ServoX is distributed under the terms of the Apache 2.0 Open Source license.
A copy of the license is provided in the LICENSE file at the root of the repository.
FAQs
Opsani Servo: The Next Generation
We found that servox demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.