If you want to map reduce parallel but hadoop is overkill, with Hadopy you can run map reduce in python.
To get Hadopy, either install from PyPi:
$ pip install hadopy
or clone this github project and install:
$ pip install .
Hadopy was programmed with ease-of-use in mind. To run it use one of the following command:
Linux / MacOS
$ cat example.txt | hadopy --mapper "python mapper.py" --reducer "python reducer.py"
Windows
$ type example.txt | hadopy --mapper "python mapper.py" --reducer "python reducer.py"
For more information use
$ hadopy --help
FAQs
Easy parallel map-reduce command line tool
We found that hadopy demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The EU Cyber Resilience Act is prompting compliance requests that open source maintainers may not be obligated or equipped to handle.
Security News
Crates.io adds Trusted Publishing support, enabling secure GitHub Actions-based crate releases without long-lived API tokens.
Research
/Security News
Undocumented protestware found in 28 npm packages disrupts UI for Russian-language users visiting Russian and Belarusian domains.