
Product
Introducing Scala and Kotlin Support in Socket
Socket now supports Scala and Kotlin, bringing AI-powered threat detection to JVM projects with easy manifest generation and fast, accurate scans.
causal-testing-framework
Advanced tools
A framework for causal testing using causal directed acyclic graphs.
Causal testing is a causal inference-driven framework for functional black-box testing. This framework utilises graphical causal inference (CI) techniques for the specification and functional testing of software from a black-box perspective. In this framework, we use causal directed acyclic graphs (DAGs) to express the anticipated cause-effect relationships amongst the inputs and outputs of the system-under-test and the supporting mathematical framework to design statistical procedures capable of making causal inferences. Each causal test case focuses on the causal effect of an intervention made to the system-under test. That is, a prescribed change to the input configuration of the system-under-test that is expected to cause a change to some output(s).
Python 3.10, 3.11 and 3.12
Microsoft Visual C++ 14.0+ (Windows only).
To install the latest stable release of the Causal Testing Framework:
pip install causal-testing-framework
or if you want to install with the development packages/tools:
pip install causal-testing-framework[dev]
Alternatively, you can install directly via source:
git clone https://github.com/CITCOM-project/CausalTestingFramework
cd CausalTestingFramework
then to install a specific release:
git fetch --all --tags --prune
git checkout tags/<tag> -b <branch>
pip install . # For core API only
pip install -e . # For editable install, useful for development work
For more information on how to use the Causal Testing Framework, please refer to our documentation.
[!NOTE] We recommend you use a 64 bit OS (standard in most modern machines) as we have had reports of the installation crashing on some 32 bit Debian installations.
[!NOTE] Example usage can be found in the
examples
directory.
To run the causal testing framework, you need some runtime data from your system, some causal test cases, and a causal DAG that specifies the expected causal relationships between the variables in your runtime data (and any other relevant variables that are not recorded in the data but are known to be relevant).
If you do not already have causal test cases, you can convert your causal DAG to causal tests by running the following command.
python causal_testing/testing/metamorphic_relation.py --dag_path $PATH_TO_DAG --output_path $PATH_TO_TESTS
python -m causal_testing --dag_path $PATH_TO_DAG --data_paths $PATH_TO_DATA --test_config $PATH_TO_TESTS --output $OUTPUT
The results will be saved for inspection in a JSON file located at $OUTPUT
.
In the future, we hope to add a visualisation tool to assist with this.
If you use our framework in your work, please cite the following:
This research has used version X.Y.Z (software citation) of the Causal Testing Framework (paper citation).
The paper citation should be the Causal Testing Framework paper, and the software citation should contain the specific Figshare DOI of the version used in your work.
```
@ARTICLE{Clark_etal_2023,
author = {Clark, Andrew G. and Foster, Michael and Prifling, Benedikt and Walkinshaw, Neil and Hierons, Robert M.
and Schmidt, Volker and Turner, Robert D.},
title = {Testing Causality in Scientific Modelling Software},
year = {2023},
publisher = {Association for Computing Machinery},
url = {https://doi.org/10.1145/3607184},
doi = {10.1145/3607184},
journal = {ACM Trans. Softw. Eng. Methodol.},
month = {jul},
keywords = {Software Testing, Causal Testing, Causal Inference}
}
```
```
@ARTICLE{Wild2023,
author = {Foster, Michael and Clark, Andrew G. and Somers, Richard and Wild, Christopher and Allian, Farhad and Hierons, Robert M. and Wagg, David and Walkinshaw, Neil},
title = {CITCOM Software Release},
year = {2023},
month = {nov},
url = {https://orda.shef.ac.uk/articles/software/CITCOM_Software_Release/24427516},
doi = {10.15131/shef.data.24427516.v1}
}
```
To contribute to our work, please ensure the following:
CausalTestingFramework/main
, and ensure any conflicts are resolved.We use the unittest module to develop our tests and the pytest framework as our test discovery, pylint for our code analyser, and black for our code formatting.
To find the other (optional) developer dependencies, please check pyproject.toml
.
The Causal Testing Framework is supported by the UK's Engineering and Physical Sciences Research Council (EPSRC), with the project name CITCOM - "Causal Inference for Testing of Computational Models" under the grant EP/T030526/1.
FAQs
A framework for causal testing using causal directed acyclic graphs.
We found that causal-testing-framework demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket now supports Scala and Kotlin, bringing AI-powered threat detection to JVM projects with easy manifest generation and fast, accurate scans.
Application Security
/Security News
Socket CEO Feross Aboukhadijeh and a16z partner Joel de la Garza discuss vibe coding, AI-driven software development, and how the rise of LLMs, despite their risks, still points toward a more secure and innovative future.
Research
/Security News
Threat actors hijacked Toptal’s GitHub org, publishing npm packages with malicious payloads that steal tokens and attempt to wipe victim systems.