MR Streams
MR Streams is a python utility for composing iterable map reduce filter chains
Goals:
The goals of this library are relatively simple:
- provide a stream composition syntax that supports chaining map,reduce,filter operations
- a stream object should still look and feel like a list/iterable
- a stream should have options for delayed evaluation of the stream
- to support future changes regarding how composition
Pronunciation:
- "Mister Streams" for fun
- "M-R-Streams" to sound professional
Supported operations
-
map
- applies functions to values in iterable with a built-in option for partial evaluation.
from operator import add
[*mr(range(10)).map(add, 1)]
>>[1,2,3,4,5,6,7,8,9,10]
-
filter
- applies a boolean function to values emitted in the chain. If the condition is true, the values are emitted further down the chain.
is_even = lambda x: x % 2 == 0
[*mr(range(10)).filter(is_even)]
>>[0, 2,4,6,8]
-
reduce
- reduce iterates through all objects and reduces them using a reduction function.
mr(range(10)).reduce(sum)
>> 45
-
take
- limits the number of values emitted in the stream.
[*mr(range(10)).take(3)]
>>[0,1,2]
-
tap
- applies a function passively to the stream without altering emitted values.
mr(range(4)).tap(print).reduce(sum)
>> 0
>> 1
>> 2
>> 3
>> 6
-
drain
- runs a no-op iteration that depletes the stream.
mr(range(4)).tap(print).drain()
>> 0
>> 1
>> 2
>> 3
-
chunk
- groups items in a stream into groups of size n
mr(range(4)).chunk(2).tap(print).drain()
>> [0,1]
>> [2,3]