hadopy

Hadopy - Easy parallel map-reduce command line tool

If you want to map reduce parallel but hadoop is overkill, with Hadopy you can run map reduce in python.

Installing

To get Hadopy, either install from PyPi:

$ pip install hadopy 

or clone this github project and install:

$ pip install .

Usage

Hadopy was programmed with ease-of-use in mind. To run it use one of the following command:

Linux / MacOS

$ cat example.txt | hadopy --mapper "python mapper.py" --reducer "python reducer.py"

Windows

$ type example.txt | hadopy --mapper "python mapper.py" --reducer "python reducer.py"

For more information use

$ hadopy --help