Wurlitzer
Capture C-level stdout/stderr pipes in Python via os.dup2
.
For more details on why this is needed, please read this blog post.
Install
pip install wurlitzer
Usage
Capture stdout/stderr in pipes:
from wurlitzer import pipes
with pipes() as (out, err):
call_some_c_function()
stdout = out.read()
Capture both stdout and stderr in a single StringIO object:
from io import StringIO
from wurlitzer import pipes, STDOUT
out = StringIO()
with pipes(stdout=out, stderr=STDOUT):
call_some_c_function()
stdout = out.getvalue()
Forward C-level stdout/stderr to Python sys.stdout/stderr,
which may already be forwarded somewhere by the environment, e.g. IPython:
from wurlitzer import sys_pipes
with sys_pipes():
call_some_c_function()
Forward C-level output to Python Logger objects (new in 3.1).
Each line of output will be a log message.
from wurlitzer import pipes, STDOUT
import logging
logger = logging.getLogger("my.log")
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler("mycode.log"))
with pipes(logger, stderr=STDOUT):
call_some_c_function()
Forward C-level output to a file (avoids GIL issues with a background thread, new in 3.1):
from wurlitzer import pipes, STDOUT
with open("log.txt", "ab") as f, pipes(f, stderr=STDOUT):
blocking_gil_holding_function()
Or even simpler, enable it as an IPython extension:
%load_ext wurlitzer
To forward all C-level output to IPython (e.g. Jupyter cell output) during execution.
Acknowledgments
This package is based on stuff we learned with @takluyver and @karies while working on capturing output from the Cling Kernel for Jupyter.
Wurlitzer?!
Wurlitzer makes pipe organs. Get it? Pipes? Naming is hard.