packtivity - pypi Package Compare versions

Comparing version 0.14.24 to 0.15.0

PKG-INFO

 Metadata-Version: 2.1
 Name: packtivity
-Version: 0.14.24
+Version: 0.15.0
 Summary: packtivity - general purpose schema + bindings for PROV activities
@@ -8,4 +8,2 @@ Home-page: https://github.com/yadage/packtivity
 Author-email: [e-mail]
-License: UNKNOWN
-Platform: UNKNOWN
 Classifier: Development Status :: 4 - Beta
@@ -16,3 +14,2 @@ Classifier: Intended Audience :: Science/Research
 Classifier: Programming Language :: Python :: 3 :: Only
-Classifier: Programming Language :: Python :: 3.6
 Classifier: Programming Language :: Python :: 3.7
@@ -22,2 +19,3 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: Implementation :: CPython
@@ -27,7 +25,90 @@ Classifier: Topic :: Scientific/Engineering
 Classifier: Operating System :: OS Independent
-Requires-Python: >=3.6
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
 Provides-Extra: celery
 License-File: LICENSE
 
-UNKNOWN
+# packtivity
 
+[![DOI](https://zenodo.org/badge/53696818.svg)](https://zenodo.org/badge/latestdoi/53696818)
+[![Coverage Status](https://coveralls.io/repos/github/diana-hep/packtivity/badge.svg)](https://coveralls.io/github/diana-hep/packtivity)
+[![Documentation Status](https://readthedocs.org/projects/packtivity/badge/?version=latest)](http://packtivity.readthedocs.io/en/latest/?badge=latest)
+[![PyPI](https://img.shields.io/pypi/v/packtivity.svg)](https://pypi.python.org/pypi/packtivity)
+
+This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities.
+
+This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas.
+
+Packtivities define
+
+* the software environment
+* parametrized process descriptions (what programs to run within these environment) and
+* produces human and machine readable outputs (as JSON) of the resulting data fragments.
+
+At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks.
+
+## Packtivity in Yadage
+
+This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows.
+
+## Example Packtivity spec
+
+This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved:
+
+    process:
+      process_type: 'string-interpolated-cmd'
+      cmd: 'DelphesHepMC  {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}'
+    publisher:
+      publisher_type: 'frompar-pub'
+      outputmap:
+        lhcofile: outputlhco
+        rootfile: outputroot
+    environment:
+      environment_type: 'docker-encapsulated'
+      image: lukasheinrich/root-delphes
+
+## Usage
+
+You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub),  all necessary parameters and attaching an external state (via the `--read` and `--write` flags).
+
+    packtivity-run -t from-github/phenochain delphes.yml \
+      -p inputhepmc="$PWD/pythia/output.hepmc" \
+      -p outputroot="'{workdir}/output.root'" \
+      -p outputlhco="'{workdir}/output.lhco'" \
+      -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
+      --read pythia --write outdir
+
+## Asynchronous Backends
+
+In order to facilitate usage of distributed resources, a number of Asynchronous
+backends can be specified. Here is an example for IPython Parallel clusters
+
+    packtivity-run -b ipcluster --asyncwait \
+      -t from-github/phenochain delphes.yml \
+      -p inputhepmc="$PWD/pythia/output.hepmc" \
+      -p outputroot="'{workdir}/output.root'" \
+      -p outputlhco="'{workdir}/output.lhco'" \
+      -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
+      --read pythia --write outdir
+
+You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag):
+
+    packtivity-run -b celery --async \
+      -t from-github/phenochain delphes.yml \
+      -p inputhepmc="$PWD/pythia/output.hepmc" \
+      -p outputroot="'{workdir}/output.root'" \
+      -p outputlhco="'{workdir}/output.lhco'" \
+      -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
+      --read pythia --write outdir
+
+And at a later point in time you can check via:
+
+    packtivity-checkproxy proxy.json
+
+## External Backends
+
+Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled
+by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy
+classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.:
+
+    export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy"

setup.py

 import os
 from setuptools import setup, find_packages
+from pathlib import Path
 
@@ -22,7 +23,10 @@ deps = [
 
+this_directory = Path(__file__).parent
 
 setup(
     name="packtivity",
-    version="0.14.24",
+    version="0.15.0",
     description="packtivity - general purpose schema + bindings for PROV activities",
+    long_description=(this_directory / "README.md").read_text(),
+    long_description_content_type="text/markdown",
     url="https://github.com/yadage/packtivity",
@@ -32,6 +36,12 @@ author="Lukas Heinrich",
     packages=find_packages(),
-    python_requires=">=3.6",
+    python_requires=">=3.7",
     include_package_data=True,
     install_requires=deps,
-    extras_require={"celery": ["celery", "redis"]},
+    extras_require={
+        "celery": [
+            "celery>=5.0.0",
+            "redis",
+            "importlib-metadata<5.0.0; python_version < '3.8'",  # FIXME: c.f. https://github.com/celery/celery/issues/7783
+        ]
+    },
     entry_points={
@@ -52,3 +62,2 @@ "console_scripts": [
         "Programming Language :: Python :: 3 :: Only",
-        "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
@@ -58,2 +67,3 @@ "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.10",
+        "Programming Language :: Python :: 3.11",
         "Programming Language :: Python :: Implementation :: CPython",
@@ -60,0 +70,0 @@ "Topic :: Scientific/Engineering",

Alert delta unavailable

Currently unable to show alert delta for PyPI packages.

Improved metrics

Total package byte prevSize

146229

3.36%

Lines of code

3309

0.3%