Reaction Optimisation Goal Iteration (ROGI) Agent
The folder contains the source, resource, and Docker setup files for the Reaction Optimisation Goal Iteration (ROGI) Agent, following the suggestions on a template provide as TheWorldAvatar/JPS_BASE_LIB/python_derivation_agent/README.md
.
Purpose
The Reaction Optimisation Goal Iteration (ROGI) Agent is designed to perform iterations of reaction experiment as part of goal-driven reaction optimisation exercise. It operates based on concepts defined in OntoGoal
and orchestrates DoE Agent
, VapourtecSchedule Agent
, Vapourtec Agent
, HPLC Agent
, and HPLCPostPro Agent
to complete one iteration.
Building the Docker image
Requirements:
- Example of configurations for the agent are provided in
TheWorldAvatar/Agents/RxnOptGoalIterAgent/agent.goal.iter.env.example
file. The knowledge graph endpoints used by this agent are specified using SPARQL_QUERY_ENDPOINT
and SPARQL_UPDATE_ENDPOINT
for triple store, and FILESERVER_URL
for the file server. The credentials for knowledge graph endpoints, i.e. triple store and file server, should be provided in the same file using KG_USERNAME
, KG_PASSWORD
, FILE_SERVER_USERNAME
, FILE_SERVER_PASSWORD
. To avoid commit these information to git at deployment, developer may make a copy of this example file as agent.goal.iter.env
. As *.env
entry already exist in .gitignore
, this new created file will be omitted. Any credentials encoded are safe. The OntoAgent:Service IRI of the agent is specified using ONTOAGENT_SERVICE_IRI
. The periodically time interval to monitor asynchronous derivation is specified by DERIVATION_PERIODIC_TIMESCALE
. One may also provide DERIVATION_INSTANCE_BASE_URL
to be used by DerivationClient when creating derivations related instances. ONTOAGENT_OPERATION_HTTP_URL
can be used to specify the URL of the agent that listens the request for updating synchronous derivations, however, given the nature of the post processing Agent, this is NOT RECOMMENDED. Developers needs to ensure that this file is correctly updated before building the Docker Image.
Once the requirements have been addressed, the Image can be build via docker container, one example of which is:
(Linux)
cd /your_absolute_path_to/TheWorldAvatar/Agents/RxnOptGoalIterAgent/
docker-compose -f "docker-compose.test.yml" up -d --build
Or, simply right click docker-compose.test.yml
file and select Compose Up
option in Visual Studio Code.
Environment setup
For development and testing reasons, follow below instructions to get started.
Virtual environment setup
It is highly recommended to install pyderivationagent
packages in a virtual environment (python>=3.8). The following steps can be taken to build a virtual environment:
(Windows)
$ python -m venv <venv_name>
$ <venv_name>\Scripts\activate.bat
(<venv_name>) $
(Linux)
$ python3 -m venv <venv_name>
$ source <venv_name>/bin/activate
(<venv_name>) $
The above commands will create and activate the virtual environment <venv_name>
in the current directory.
Package installation
The following command can be used to install all required packages.
(Linux)
(<venv_name>) $ python -m pip install --upgrade pip
(<venv_name>) $ python -m pip install -r requirements.txt
(<venv_name>) $ python -m pip install -e .[dev]
As pyderivationagent
library relies on the py4jps
package, Java 11 is required. For Windows, it is recommended to obtain OpenJDK 11 from here and follow the instructions. For linux environment, one can install via:
(Linux)
$ sudo apt update
$ sudo apt install openjdk-11-jdk-headless
How to use it
The ROGI Agent is intended to use the asychronous mode
of the Derivation Framework to detect changes in instantiated OntoGoal properties (e.g. goal
, best resutl
, and restriction
) to automatically request the design/schedule/execution/analyses of a reaction experiment instances in the KG. As the agent adopts the pyderivationagent
, it also serves HTTP requests to handle synchronous derivations. However, it is (strongly) discouraged to invoke such HTTP request by ONESELF.
After successful agent start-up, an instructional page shall become available at the root (i.e. /
) of the port specified in the docker compose file. The exact address depends on where the agent container is deployed (i.e. localhost, remote VM, ...), but takes a form like http://165.232.172.16:7060/
.
Asynchronous derivation operation
Once the Agent is deployed, it periodically (defined by DERIVATION_PERIODIC_TIMESCALE
) checks the derivation that isDerivedUsing
itself (parameter ONTOAGENT_SERVICE_IRI
) and acts based on the status associated with that derivation. Over time, you may see more information about a reaction experiment is added to the knowledge graph gradually.
Prior derivation markup
For the Agent to detect outdated information, a proper mark up of the relevant derivation inputs (i.e. pure inputs) is required. (Please note, that another pre-requisite for detecting derivation inputs is the registration of the agent in the KG, i.e. REGISTER_AGENT=true
in the env file.) The following methods from the pyderivationagent
package shall be used to mark up derivation inputs within the KG (for illustration purposes only):
deriv_client = agent.derivation_client
deriv_client.updateTimestamp(inputIRI)
deriv_inputs = [goal_set_IRI, lab_IRI, chem_rxn_IRI, rxn_exp_IRI1, rxn_exp_IRI2, ...]
deriv_iri = deriv_client.createAsyncDerivationForNewInfo(agent.agentIRI, deriv_inputs)
Agent integration test
As this derivation agent modifies the knowledge graph automatically, it is recommended to run integration tests before deploying it for production. A few integration tests are provided in the tests
repository.
The dockerised integration test can be invoked via below commands:
(Linux)
cd /your_absolute_path_to/TheWorldAvatar/Agents/RxnOptGoalIterAgent
pytest -s --docker-compose=./docker-compose.test.yml --reruns 5 --reruns-delay 5
If you would like to contribute to new features for the ROGI Agent, you may use the same integration test to make sure the new features added do NOT break the original function.
Upload docker image to GitHub
Developers who add new features to the RxnOptGoalIter Agent
handle the distribution of the docker image on GitHub. If you want to add new features that suit your project and release the docker image independently, i.e. become a developer/maintainer, please contact the repository's administrator to indicate your interest.
The release procedure is currently semi-automated and requires a few items:
- Your GitHub account and password (personal access token)
- The version number x.x.x for the release
- Clone of
TheWorldAvatar
repository on your local machine
- Docker-desktop is installed and running on your local machine
Stable version release
The release process can be started by using the commands below. (REMEMBER TO CHANGE THE CORRECT VALUES FOR <absolute_path_to>
IN THE COMMANDS BELOW!) NOTE: the release process is only tested in WSL2 environment.
(Linux)
$ cd /<absolute_path_to>/TheWorldAvatar/Agents/RxnOptGoalIterAgent
$ ./upload_docker_image_to_github.sh -v x.x.x
Please follow the instructions presented in the console once the process has begun. If everything goes well, the change performed automatically during the release process should be commited, i.e., in python script Agents/RxnOptGoalIterAgent/docker-compose.github.yml
image: ghcr.io/cambridge-cares/rxn_opt_goal_iter_agent:x.x.x
NOTE: the visibility of the uploaded docker image is set as private by default, developer who uploaded the image need to change the package visibility to public manually after the upload.
Snapshot version release
If you would like to release the package in SNAPSHOT version, below commands can be used intead:
(Linux)
$ cd /<absolute_path_to>/TheWorldAvatar/Agents/RxnOptGoalIterAgent
$ ./upload_docker_image_to_github.sh -v x.x.x-SNAPSHOT
Please follow the instructions presented in the console once the process has begun. If everything goes well, commit the change in version number following the same procedure as in the stable version release.
Authors
Jiaru Bai (jb2197@cam.ac.uk)