Security News
Introducing the Socket Python SDK
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Run multiple subprocesses asynchronous/in parallel with streamed output/non-blocking reading. Also various tools to replace shell scripts.
This provides a clean way to execute subprocesses, either one or multiple in parallel, capture their output and monitor progress:
call
with optional timeoutFileMonitor
to execute several processes in parallel and store output in a fileGroup
execution to execute jobs in parallel and capture outputAdditional tools for working with the filesystem are also included:
find
which offers much of the functionality of the shell find utilityshelljob.fs.NamedTempFile
provides a with block wrapper for temporary named filespip install shelljob
Using the Job system is the quickest approach to just run processes and log their output (by default in files named '/tmp/job_ID.log')
from shelljob import job
jm = job.FileMonitor()
jm.run([
[ 'ls', '-alR', '/usr/local' ],
'my_prog',
'build output input',
])
An array will passed directly to subprocess.Popen
, a string is first parsed with shlex.split
.
The lower level Group
class provides a simple container for more manual job management.
from shelljob import proc
g = proc.Group()
p1 = g.run( [ 'ls', '-al', '/usr/local' ] )
p2 = g.run( [ 'program', 'arg1', 'arg2' ] )
while g.is_pending():
lines = g.readlines()
for proc, line in lines:
sys.stdout.write( "{}:{}".format( proc.pid, line ) )
By default the output will be binary encoding. You can specify an encoding='utf-8'
to the run
command to use an encoded text stream instead. Be aware that if the encoding fails (the program emits an invalid sequence) the running will be interrupted. You should also use the on_error
function to check for this.
Line-endings will always be preserved.
A simplified call
function allows timeouts on subprocesses and easy acces to their output.
from shelljob import proc
# capture the output
output = proc.call( 'ls /tmp' )
# this raises a proc.Timeout exception
proc.call( 'sleep 10', timeout = 0.1 )
The 'find' funtion is a multi-faceted approach to generating listings of files.
from shelljob import fs
files = fs.find( '/usr/local', name_regex = '.*\\.so' )
print( "\n".join(files) )
Refer to the API docs for all parameters. Just let me know if there is some additional option you need.
FAQs
Run multiple subprocesses asynchronous/in parallel with streamed output/non-blocking reading. Also various tools to replace shell scripts.
We found that shelljob demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Security News
Floating dependency ranges in npm can introduce instability and security risks into your project by allowing unverified or incompatible versions to be installed automatically, leading to unpredictable behavior and potential conflicts.
Security News
A new Rust RFC proposes "Trusted Publishing" for Crates.io, introducing short-lived access tokens via OIDC to improve security and reduce risks associated with long-lived API tokens.