
Security News
Crates.io Implements Trusted Publishing Support
Crates.io adds Trusted Publishing support, enabling secure GitHub Actions-based crate releases without long-lived API tokens.
nonblocking-stream-queue
Advanced tools
This is a simple python package for nonblocking reading from streams.
It could be expanded for general nonblocking usage if desired.
A simple nonblocking queue reader is provided, with implementation from https://stackoverflow.com/questions/375427/a-non-blocking-read-on-a-subprocess-pipe-in-python/4896288#4896288 .
When constructed, the reader spawns a thread and begins reading everything from the stream.
$ python3 -m pip install nonblocking-stream-queue
import sys
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin, # object to read from
max_size=4096, # max size of each read
lines=False, # whether or not to break reads into lines
max_count=None, # max queued reads
drop_timeout=None, # time to wait for queue to drain before dropping
drop_older=False, # which end of the queue to drop from
pre_cb=None, # if set, data = (pre_cb(), read())
post_cb=None, # if set, data = post_cb(data)
drop_cb=None, # if set, call with dropped data
verbose=False, # if set, display progress filling max_count
)
print(reader.read_one()) # outputs up to 4096 characters, or None if nothing queued
print(reader.read_many()) # outputs all 4096 character chunks queued
print(''.join(reader.read_many())) # outputs all queued text
reader.block() # waits for data to be present or end, returns number of reads queued
reader.is_pumping() # False if data has ended
import sys, time
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin.buffer,
pre_cb=lambda: time.time(),
post_cb=lambda time_data_tuple: (time_data_tuple[0], time_data_tuple[1], time.time())
)
while reader.block():
start_time, data, end_time = reader.read_one()
print(start_time, data, end_time)
import sys
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin,
lines=True
)
while reader.block():
print(reader.read_one().rstrip()) # outputs each line of text that is input
There are likely many other existing solutions to this common task.
Here's one:
FAQs
Nonblocking stream reads using Queue and a pump thread
We found that nonblocking-stream-queue demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Crates.io adds Trusted Publishing support, enabling secure GitHub Actions-based crate releases without long-lived API tokens.
Research
/Security News
Undocumented protestware found in 28 npm packages disrupts UI for Russian-language users visiting Russian and Belarusian domains.
Research
/Security News
North Korean threat actors deploy 67 malicious npm packages using the newly discovered XORIndex malware loader.