pypx

#################################### pypx - 3.11.22 ####################################

.. image:: https://badge.fury.io/py/pypx.svg :target: https://badge.fury.io/py/pypx

.. contents:: Table of Contents

1. Overview

---

pypx is a complete client-side PACS (Picture Archive and Communications System) Query/Retrieve/Storage solution that operates in stand-alone script mode in addition to providing a set of python modules for use in other packages. The modules/API provide a simple mechanism for a python program to interact with an appropriately configured remote PACS, while the stand alone scripts offer a convenient ability to directly Query/Retrieve/Store images from the command line.

pypx was mostly developed for use in the ChRIS system as part of the pfdcm microservice; however the CLI scripts of pypx and the provided docker image offer a quick and powerful means of accessing a PACS without any additional overhead.

1.1 Complete Client Side

This solution is client-side and cannot operate fully independently of an appropriately configured PACS. Unfortunately, simply downloading this repo/tools and pointing the scripts at some PACS is insufficient. The PACS itself (which is NOT part of this repo) needs to be configured to service communications and requests from these tools. See below for more information.

1.1.1 Quick PACS Primer

A PACS exists as a separate service on a network, and pypx communicates with a pre-configured PACS when asking for Query data and when Retrieving images. Importantly, from the client perspective, data is PUSHED from the PACS, and not PULLED from the client. This means that client software in essence "asks" the PACS for images and the PACS obliges by transmitting the images over the network to a pre-configured location.

Communications with a PACS are for the most insecure and reflected a circa 1990s view/model of internetworking. When a client communicates with a PACS, it sends along with every request string identifiers unique to the client and configured in the PACS. Typical identifiers are the AETitle and sometimes additionally the CalledAETitle. The PACS examines these strings on receipt to identify/authenticate the client and also to identify a destination network IP:port to which data can be transmitted.

1.1.2 Configuring a PACS

In order to be fully complete, a destination PACS with which pypx modules wish to communicate needs to be configured with appropriate AETitle, CalledAETitle, as well as the network address IP and port of the pypx hosting machine. Configuring a PACS is obviously outside of the scope of this documentation. Consult your PACS for information on this configuration.

As a brief note, if the opensource orthanc PACS server is being used, the orthanc.json can be edited to include

.. code-block:: json

{ "DicomAet" : "CHRISV3T" }

and

.. code-block:: json

{ "CHRIS" : [ "CHRIS", "10.72.76.39", 10402 ], "CHRISLOCAL" : ["CHRISLOCAL", "192.168.1.189", 11113 ] }

where CHRISLOCAL for example defines a DICOM storescp service on the host 192.168.1.189 and port 11113 while CHRIS defines another destination storescp service .

1.1.3 Configuring pypx

Locally, however, some configuration is required and conveniently located in the script PACS_QR.sh. In the

.. code-block:: bash

function institution_set
{
    ...
}

simply add another block reflecting the variables appropriate to your remote PACS service.

1.2 Components

1.2.1 Environment

pypx can be thought of as a bridge connecting a PACS to a ChRIS instance. In between these services is a filesystem. A retrieve operation will request files from a PACS which arrive over the network and a separately configured listening service repacks these files in a specially configured location called the BASEMOUNT. Once these files are received, they can be push -ed to special ChRIS friendly storage called swift, and once there they can be register -ed to ChRIS/CUBE. Each of these services (swift and CUBE) have network locations and login details which are stored in the BASEMOUNT in <BASEMOUNT>/services/[swift,cube].json. Many different swift and CUBE configurations can in theory exist in these json files. Each configuration is identified by a key -- the SWIFTKEY for the swift service and the CUBEKEY for the CUBE service. Using these keys makes for a convenient way to push and register files without very verbose CLI.

See PACS_QR.sh -x for some in-line help on setting these keys.

1.2.2 Tools

Internally, the code wraps around DCMTK utilies as well as the PyDicom module. The following modules/scripts are provided:

• pfstorage_: Query / put files/objects into swift storage.

• px-do_: Perform various downstream utility functions once a px-find has completed.

• px-echo_: Ping the PACS to make sure it is online (echoscu).

• px-find_: Find (Query) a PACS in a variety of ways. The start point of almost all other workflows which are constructed as find then do.

• px-listen_: Deprecated listening service wrapper.

• px-move_: Move data from the PACS (movescu).

• px-push_: Push DICOM data to a remote node (either a PACS or a ChRIS swift object storage container).

• px-register_: A companion to px-push that registers files in ChRIS swift storage to the ChRIS CUBE backend.

• px-repack_: Read and repack DICOM files, organizing the destination in a human-friendly tree based layout.

• px-report_: Consume the JSON outputs of many of the tools (esp the px-find and generate various console-based reports).

• px-status_: Report on the status of query results in the BASEMOUNT.

• px-smdb_: A simple file-system based database that provides tracking and query for processed DICOM files.

1. Installation

---

2.1 Prerequisites

For all installation solutions, make sure that the machine receiving images from a PACS has approporate listening and repacking services and that the PACS itself has been configured to recognize this machine. While out of scope of this document, the simplest way to set this up is to use the pfdcm service (provided separately).

2.2 Using docker

Using the dockerized container is the recommended installation vector as the image contains all tools (dcmtk) that can interact both with a PACS as well as swift storage and CUBE without any additional software on the host system.

.. code-block:: bash

docker pull fnndsc/pypx

Alternatively, you can build a local image with

.. code-block:: bash

# If behing a proxy
PROXY=http://some.proxy.com
export UID=$(id -u)
DOCKER_BUILDKIT=1 docker build --build-arg http_proxy=${PROXY} --build-arg UID=$UID -t local/pypx .

# otherwise...
export UID=$(id -u)
DOCKER_BUILDKIT=1 docker build --build-arg UID=$UID -t local/pypx .

2.3 PyPI

For convenience, a PyPI installation is also available. This assumes additional non-python requirements such as dcmtk have been installed. This is recommended only for advanced users.

.. code-block:: bash

apt-get update
&& apt-get install -y dcmtk
&& apt-get install -y python3-pip python3-dev
&& pip3 install --upgrade pip
&& pip install pypx

1. Configuring the containerized version

---

If using the container tool images directly, take care to assure that the machine receiving PACS transmissions is available and has a listener service accessible on an exposed port. This port should be accessible to the remote PACS. Our strong recommendation is to use the companion pfdcm container/repo to receive PACS data. Note that pfdcm itself contains pypx and will handle the reception and repacking of DICOM files using the correct pypx tools.

4 Usage

---

4.1 PACS_QR.sh and workflow.sh

For the most complete example, please consult the workflow.sh_ script in the source repository. This provides a Jupyter-notebook-shell-eque overview of most if not all the possible methods to call and use these tools.

For the most convenient example, use the PACS_QR.sh script -- consult its internal help with

.. code-block:: bash

PACS_QR.sh -x

4.2 PACS_QR.sh quick-n-dirty

The PACS_QR.sh has several implicit assumptions and values that can/should be set by approprate CLI. The entire scope is beyond this simple README, however, assuming these values are set (either by using the defaults or an appropriate/custom institution_set function), the workflow is rather simple. Assuming an MRN of say 7654321,

.. code-block:: bash

Query

PACS_QR.sh -- "--PatientID 7654321"

More "low level" -- in fact PACS_QR.sh is mostly a convenience wrapper

around these low level functions. For completeness sake the low level

Query is shown here. Note that passing PACS_QR.sh -v 1 ... will

instruct PACS_QR.sh to show the actual commands it uses.

px-find
--aec CHRIS
--aet CHRISV3
--serverIP 134.174.12.21
--serverPort 104
--db /home/dicom/log
--verbosity 1
--PatientID 7654321

A similar call using the

--then report --withFeedBack

(note if --then is specified, then a --withFeedBack needs to be added

for console ouput)

Also, this report shows the SeriesInstanceUIDs in the report body

px-find
--aec CHRIS
--aet CHRISV3
--serverIP 134.174.12.21
--serverPort 104
--db /home/dicom/log
--verbosity 1
--PatientID 7654321 --then report --withFeedBack

For the most flexibility in reporting, set find results to be JSON

and pipe the json into the px-report app.

px-find
--aec CHRIS
--aet CHRISV3
--serverIP 134.174.12.21
--serverPort 104
--db /home/dicom/log
--verbosity 1 --json
--PatientID 7654321 |
px-report
--colorize dark
--printReport tabular

Retrieve

PACS_QR.sh --do retrieve -- "--PatientID 7654321"

Status

PACS_QR.sh --do status -- "--PatientID 7654321"

Push to CUBE swift storage

PACS_QR.sh --do push -- "--PatientID 7654321"

Register to CUBE internal DB

PACS_QR.sh --do register -- "--PatientID 7654321"

Note carefully the syntax of the above PACS_QR.sh commands! A -- string separates script <key>/<value> pairs from a string defining the search parameters. Note that most valid DICOM tags can be used for this string. More tags can also make a search more specific, for instance

.. code-block:: bash

"--PatientID 7654321 --StudyDate 19990909"

will limit returns only to hits performed on given StudyDate.

1. Development and debugging

---

The recommended development/debug approach is to mount source directories into the container and thus debug the containerized code from the host. Assuming you are in the source code repo root, and assuming you want to debug px-push:

.. code-block:: bash

Make sure that all env variables below are set appropriately!

docker run --rm -ti -v $PWD/pypx:/usr/local/lib/python3.8/dist-packages/pypx
-v $PWD/bin/px-push:/usr/local/bin/px-push
-v $LOCALDICOMDIR:$LOCALDICOMDIR -v $BASEMOUNT:$BASEMOUNT $PYPX
--px-push
--swiftIP $SWIFTHOST
--swiftPort $SWIFTPORT
--swiftLogin $SWIFTLOGIN
--swiftServicesPACS $SWIFTSERVICEPACS
--db $DB
--swiftPackEachDICOM
--xcrdir $LOCALDICOMDIR
--parseAllFilesWithSubStr dcm
--verbosity 1
--json

1. Additional support (incomplete)

---

Please see the relevant wiki pages for usage instructions (some are still under construction):

• pfstorage_
• px-do_
• px-echo_
• px-find_
• px-move_
• px-push_
• px-register_
• px-repack_
• px-report_
• px-status_
• px-smdb_

1. Credits

---

PyDicom_

• Author(s): darcymason_

DCMTK_

• Author(s): Dicom @ OFFIS Team

.. _px-repack: https://github.com/FNNDSC/pypx/wiki/1.-px-repack .. _px-echo: https://github.com/FNNDSC/pypx/wiki/1.-px-echo .. _px-find: https://github.com/FNNDSC/pypx/wiki/2.-px-find .. _px-report: https://github.com/FNNDSC/pypx/wiki/4.-px-report .. _px-move: https://github.com/FNNDSC/pypx/wiki/3.-px-move .. _px-push: https://github.com/FNNDSC/pypx/wiki/3.-px-push .. _px-register: https://github.com/FNNDSC/pypx/wiki/3.-px-register .. _px-do: https://github.com/FNNDSC/pypx/blob/master/bin/px-do .. _px-listen: https://github.com/FNNDSC/pypx/blob/master/bin/px-listen .. _px-status: https://github.com/FNNDSC/pypx/blob/master/bin/px-status .. _px-smdb: https://github.com/FNNDSC/pypx/wiki/3.-px-smdb .. _workflow.sh: https://github.com/FNNDSC/pypx/blob/master/workflow.sh .. _PyDicom: http://www.python.org/ .. _darcymason: https://github.com/darcymason .. _DCMTK: http://dicom.offis.de/dcmtk.php.en .. _pfstorage: https://github.com/FNNDSC/pypx/blob/master/bin/pfstorage .. _benchmarks: https://github.com/FNNDSC/pypx/wiki/Benchmarks