Security News
pnpm 10.0.0 Blocks Lifecycle Scripts by Default
pnpm 10 blocks lifecycle scripts by default to improve security, addressing supply chain attack risks but sparking debate over compatibility and workflow changes.
Take a modern Python codebase to the next level of performance.
On their own, AsyncIO and multiprocessing are useful, but limited: AsyncIO still can't exceed the speed of GIL, and multiprocessing only works on one task at a time. But together, they can fully realize their true potential.
aiomultiprocess presents a simple interface, while running a full AsyncIO event loop on each child process, enabling levels of concurrency never before seen in a Python application. Each child process can execute multiple coroutines at once, limited only by the workload and number of cores available.
Gathering tens of thousands of network requests in seconds is as easy as:
async with Pool() as pool:
results = await pool.map(<coroutine function>, <items>)
aiomultiprocess requires Python 3.6 or newer. You can install it from PyPI:
$ pip3 install aiomultiprocess
Most of aiomultiprocess mimics the standard multiprocessing module whenever possible, while accounting for places that benefit from async functionality.
Running your asynchronous jobs on a pool of worker processes is easy:
import asyncio
from aiohttp import request
from aiomultiprocess import Pool
async def get(url):
async with request("GET", url) as response:
return await response.text("utf-8")
async def main():
urls = ["https://noswap.com", ...]
async with Pool() as pool:
async for result in pool.map(get, urls):
... # process result
if __name__ == '__main__':
# Python 3.7
asyncio.run(main())
# Python 3.6
# loop = asyncio.get_event_loop()
# loop.run_until_complete(main())
Take a look at the User Guide for more details and examples.
For further context, watch the PyCon US 2018 talk about aiomultiprocess, "Thinking Outside the GIL":
Slides available at Speaker Deck.
aiomultiprocess is copyright Amethyst Reese, and licensed under
the MIT license. I am providing code in this repository to you under an open
source license. This is my personal repository; the license you receive to
my code is from me and not from my employer. See the LICENSE
file for details.
FAQs
AsyncIO version of the standard multiprocessing module
We found that aiomultiprocess demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
pnpm 10 blocks lifecycle scripts by default to improve security, addressing supply chain attack risks but sparking debate over compatibility and workflow changes.
Product
Socket now supports uv.lock files to ensure consistent, secure dependency resolution for Python projects and enhance supply chain security.
Research
Security News
Socket researchers have discovered multiple malicious npm packages targeting Solana private keys, abusing Gmail to exfiltrate the data and drain Solana wallets.