BehaveX is a BDD testing solution designed to enhance your Behave-based testing workflow by providing additional features and performance improvements. It's particularly beneficial in the following scenarios:
BehaveX provides the following features:
@MUTE tag to test scenarios to execute them without including them in JUnit reports.-d Behave argument.@AUTORETRY tag to automatically re-execute failing scenarios. Also, you can re-run all failing scenarios using the failing_scenarios.txt file.
To install BehaveX, execute the following command:
pip install behavex
Execute BehaveX in the same way as Behave from the command line, using the behavex command. Here are some examples:
Run scenarios tagged as TAG_1 but not TAG_2:
behavex -t=@TAG_1 -t=~@TAG_2
Run scenarios tagged as TAG_1 or TAG_2:
behavex -t=@TAG_1,@TAG_2
Run scenarios tagged as TAG_1 using 4 parallel processes:
behavex -t=@TAG_1 --parallel-processes=4 --parallel-scheme=scenario
Run scenarios located at specific folders using 2 parallel processes:
behavex features/features_folder_1 features/features_folder_2 --parallel-processes=2
Run scenarios from a specific feature file using 2 parallel processes:
behavex features_folder_1/sample_feature.feature --parallel-processes=2
Run scenarios tagged as TAG_1 from a specific feature file using 2 parallel processes:
behavex features_folder_1/sample_feature.feature -t=@TAG_1 --parallel-processes=2
Run scenarios located at specific folders using 2 parallel processes:
behavex features/feature_1 features/feature_2 --parallel-processes=2
Run scenarios tagged as TAG_1, using 5 parallel processes executing a feature on each process:
behavex -t=@TAG_1 --parallel-processes=5 --parallel-scheme=feature
Perform a dry run of the scenarios tagged as TAG_1, and generate the HTML report:
behavex -t=@TAG_1 --dry-run
Run scenarios tagged as TAG_1, generating the execution evidence into a specific folder:
behavex -t=@TAG_1 -o=execution_evidence
environment.py module will run in each parallel process. This includes the before_all and after_all hooks, which will execute in every parallel process. The same is true for the before_feature and after_feature hooks when parallel execution is organized by scenario.Important: Some arguments do not apply when executing tests with more than one parallel process, such as stop and color.
BehaveX manages concurrent executions of Behave instances in multiple processes. You can perform parallel test executions by feature or scenario. When the parallel scheme is by scenario, the examples of a scenario outline are also executed in parallel.
behavex --parallel-processes=3
behavex -t=@<TAG> --parallel-processes=3
behavex -t=@<TAG> --parallel-processes=2 --parallel-scheme=scenario
behavex -t=@<TAG> --parallel-processes=5 --parallel-scheme=feature
behavex -t=@<TAG> --parallel-processes=5 --parallel-scheme=feature --show-progress-bar
BehaveX populates the Behave contexts with the worker_id user-specific data. This variable contains the id of the current behave process.
For example, if BehaveX is started with --parallel-processes 2, the first instance of behave will receive worker_id=0, and the second instance will receive worker_id=1.
This variable can be accessed within the python tests using context.config.userdata['worker_id'].
A friendly test execution report containing information related to test scenarios, execution status, evidence, and metrics. Available at:
<output_folder>/report.html
Contains information about test scenarios and execution status. Available at:
<output_folder>/report.json
One JUnit file per feature, available at:
<output_folder>/behave/*.xml
The JUnit reports have been replaced by the ones generated by the test wrapper, just to support muting tests scenarios on build servers
You can attach images or screenshots to the HTML report using your own mechanism to capture screenshots or retrieve images. Utilize the attach_image_file or attach_image_binary methods provided by the wrapper.
These methods can be called from hooks in the environment.py file or directly from step definitions.
from behavex_images import image_attachments
@given('I take a screenshot from the current page')
def step_impl(context):
image_attachments.attach_image_file(context, 'path/to/image.png')
from behavex_images import image_attachments
from behavex_images.image_attachments import AttachmentsCondition
def before_all(context):
image_attachments.set_attachments_condition(context, AttachmentsCondition.ONLY_ON_FAILURE)
def after_step(context, step):
image_attachments.attach_image_binary(context, selenium_driver.get_screenshot_as_png())
By default, images are attached to the HTML report only when the test fails. You can modify this behavior by setting the condition using the set_attachments_condition method.
For more information, check the behavex-images library, which is included with BehaveX 3.3.0 and above.
If you are using BehaveX < 3.3.0, you can still attach images to the HTML report by installing the behavex-images package with the following command:
pip install behavex-images
Providing ample evidence in test execution reports is crucial for identifying the root cause of issues. Any evidence file generated during a test scenario can be stored in a folder path provided by the wrapper for each scenario.
The evidence folder path is automatically generated and stored in the "context.evidence_path" context variable. This variable is updated by the wrapper before executing each scenario, and all files copied into that path will be accessible from the HTML report linked to the executed scenario.
The HTML report includes detailed test execution logs for each scenario. These logs are generated using the logging library and are linked to the specific test scenario. This feature allows for easy debugging and analysis of test failures.
The HTML report provides a range of metrics to help you understand the performance and effectiveness of your test suite. These metrics include:
BehaveX enhances the traditional Behave dry run feature to provide more value. The HTML report generated during a dry run can be shared with stakeholders to discuss scenario specifications and test plans.
To execute a dry run, we recommend using the following command:
behavex -t=@TAG --dry-run
In some cases, you may want to mute test scenarios that are failing but are not critical to the build process. This can be achieved by adding the @MUTE tag to the scenario. Muted scenarios will still be executed, but their failures will not be reported in the JUnit reports. However, the execution details will be visible in the HTML report.
For scenarios that are prone to intermittent failures or are affected by infrastructure issues, you can use the @AUTORETRY tag. This tag enables automatic re-execution of the scenario in case of failure.
After executing tests, if there are failing scenarios, a failing_scenarios.txt file will be generated in the output folder. This file allows you to rerun all failed scenarios using the following command:
behavex -rf=./<OUTPUT_FOLDER>/failing_scenarios.txt
or
behavex --rerun-failures=./<OUTPUT_FOLDER>/failing_scenarios.txt
To avoid overwriting the previous test report, it is recommended to specify a different output folder using the -o or --output-folder argument.
Note that the -o or --output-folder argument does not work with parallel test executions.
When running tests in parallel, you can display a progress bar in the console to monitor the test execution progress. To enable the progress bar, use the --show-progress-bar argument:
behavex -t=@TAG --parallel-processes=3 --show-progress-bar
If you are printing logs in the console, you can configure the progress bar to display updates on a new line by adding the following setting to the BehaveX configuration file:
[progress_bar]
print_updates_in_new_lines="true"
If you find this project helpful or interesting, we would appreciate it if you could give it a star (:star:). It's a simple way to show your support and let us know that you find value in our work.
By starring this repository, you help us gain visibility among other developers and contributors. It also serves as motivation for us to continue improving and maintaining this project.
Thank you in advance for your support! We truly appreciate it.
FAQs
Agile testing framework on top of Behave (BDD).
We found that behavex demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.