mappingtools

MappingTools

Do stuff with Mappings and more

Introduction

This library provides utility functions for manipulating and transforming data structures which have or include Mapping-like characteristics. Including inverting dictionaries, converting class like objects to dictionaries, creating nested defaultdicts, and unwrapping complex objects.

Package
Code
Tools
CI/CD
Scans

Overview

The MappingTools library is organized into several namespaces, each containing specific functionalities for manipulating and transforming data structures. Below is a brief description of the main namespaces within the library:

collectors

This namespace contains classes and functions for collecting and categorizing data items into mappings.

• CategoryCounter: Extends a dictionary to count occurrences of data items categorized by multiple categories.
• MappingCollector: Collects key-value pairs into an internal mapping based on different modes (ALL, COUNT, DISTINCT, FIRST, LAST).
• nested_defaultdict: Creates a nested defaultdict with specified depth and factory.

operators

This namespace provides functions that perform operations on mappings.

• distinct: Yields distinct values for a specified key across multiple mappings.
• keep: Yields subsets of mappings by retaining only the specified keys.
• remove: Yields mappings with specified keys removed.
• inverse: Swaps keys and values in a dictionary.
• flattened: Converts a nested mapping structure into a single-level dictionary by flattening the keys into tuples.
• stream: Generates items from a mapping, optionally applying a factory function to each key-value pair.
• stream_dict_records: Generates dictionary records from a mapping with customizable key and value names.

transformers

This namespace includes functions that reshape objects while maintaining the consistency of their structure.

• listify: Transforms complex objects into a list of dictionaries with key and value pairs.
• simplify: Converts objects to strictly structured dictionaries.
• strictify: Applies a strict structural conversion to an object using optional converters for keys and values.
• stringify: Converts an object into a string representation by recursively processing it based on its type.

Usage

Collectors

Collectors are classes that collect data items into a Mapping.

CategoryCounter

The CategoryCounter class extends a dictionary to count occurrences of data items categorized by multiple categories. It maintains a total count of all data items and allows categorization using direct values or functions.

from mappingtools.collectors import CategoryCounter

counter = CategoryCounter()

for fruit in ['apple', 'banana', 'apple']:
    counter.update({fruit: 1}, type='fruit', char_count=len(fruit), unique_char_count=len(set(fruit)))

print(counter.total)
# Output: Counter({'apple': 2, 'banana': 1})

print(counter)
# output: CategoryCounter({'type': defaultdict(<class 'collections.Counter'>, {'fruit': Counter({'apple': 2, 'banana': 1})}), 'char_count': defaultdict(<class 'collections.Counter'>, {5: Counter({'apple': 2}), 6: Counter({'banana': 1})}), 'unique_char_count': defaultdict(<class 'collections.Counter'>, {4: Counter({'apple': 2}), 3: Counter({'banana': 1})})})

MappingCollector

A class designed to collect key-value pairs into an internal mapping based on different modes. It supports modes like ALL, COUNT, DISTINCT, FIRST, and LAST, each dictating how key-value pairs are collected.

from mappingtools.collectors import MappingCollector, MappingCollectorMode

collector = MappingCollector(MappingCollectorMode.ALL)
collector.add('a', 1)
collector.add('a', 2)
collector.collect([('b', 3), ('b', 4)])
print(collector.mapping)
# output: {'a': [1, 2], 'b': [3, 4]}

nested_defaultdict

Creates a nested defaultdict with specified depth and factory.

from mappingtools.collectors import nested_defaultdict

nested_dd = nested_defaultdict(1, list)
nested_dd[0][1].append('value')
print(nested_dd)
# output: defaultdict(<function nested_defaultdict.<locals>.factory at ...>, {0: defaultdict(<function nested_defaultdict.<locals>.factory at ...>, {1: ['value']})})

Operators

Operators are functions that perform operations on Mappings.

distinct

Yields distinct values for a specified key across multiple mappings.

from mappingtools.operators import distinct

mappings = [
    {'a': 1, 'b': 2},
    {'a': 2, 'b': 3},
    {'a': 1, 'b': 4}
]
distinct_values = list(distinct('a', *mappings))
print(distinct_values)
# output: [1, 2]

keep

Yields subsets of mappings by retaining only the specified keys.

from mappingtools.operators import keep

mappings = [
    {'a': 1, 'b': 2, 'c': 3},
    {'a': 4, 'b': 5, 'd': 6}
]
keys_to_keep = ['a', 'b']
output = list(keep(keys_to_keep, *mappings))
print(output)
# output: [{'a': 1, 'b': 2}, {'a': 4, 'b': 5}]

remove

Yields mappings with specified keys removed. It takes an iterable of keys and multiple mappings, and returns a generator of mappings with those keys excluded.

from mappingtools.operators import remove

mappings = [
    {'a': 1, 'b': 2, 'c': 3},
    {'a': 4, 'b': 5, 'd': 6}
]
keys_to_remove = ['a', 'b']
output = list(remove(keys_to_remove, *mappings))
print(output)
# output: [{'c': 3}, {'d': 6}]

inverse

Swaps keys and values in a dictionary.

from mappingtools.operators import inverse

original_mapping = {'a': {1, 2}, 'b': {3}}
inverted_mapping = inverse(original_mapping)
print(inverted_mapping)
# output: defaultdict(<class 'set'>, {1: {'a'}, 2: {'a'}, 3: {'b'}})

flattened

The flattened function takes a nested mapping structure and converts it into a single-level dictionary by flattening the keys into tuples.

from mappingtools.operators import flattened

nested_dict = {
    'a': {'b': 1, 'c': {'d': 2}},
    'e': 3
}
flat_dict = flattened(nested_dict)
print(flat_dict)
# output: {('a', 'b'): 1, ('a', 'c', 'd'): 2, ('e',): 3}

stream

Takes a mapping and an optional item factory function, and generates items from the mapping. If the item factory is provided, it applies the factory to each key-value pair before yielding.

from collections import namedtuple

from mappingtools.operators import stream


def custom_factory(key, value):
    return f"{key}: {value}"


my_mapping = {'a': 1, 'b': 2, 'c': 3}

for item in stream(my_mapping, custom_factory):
    print(item)

# output:
# a: 1
# b: 2
# c: 3


MyTuple = namedtuple('MyTuple', ['key', 'value'])
data = {'a': 1, 'b': 2}

for item in stream(data, MyTuple):
    print(item)


# output:
# MyTuple(key='a', value=1)
# MyTuple(key='b', value=2)


def record(k, v):
    return {'key': k, 'value': v}


for item in stream(data, record):
    print(item)

# output:
# {'key': 'a', 'value': 1}
# {'key': 'b', 'value': 2}

stream_dict_records

generates dictionary records from a given mapping, where each record contains a key-value pair from the mapping with customizable key and value names.

from mappingtools.operators import stream_dict_records

mapping = {'a': 1, 'b': 2}
records = stream_dict_records(mapping, key_name='letter', value_name='number')
for record in records:
    print(record)
# output:
# {'letter': 'a', 'number': 1}
# {'letter': 'b', 'number': 2}

unique_string

The unique_strings function generates an endless stream of the shortest possible strings using a specified alphabet. By default, it uses the uppercase English alphabet (string.ascii_uppercase). The function can generate strings of a fixed length or start with the shortest strings and increase the length indefinitely.

from mappingtools.operators import unique_strings

alphabet1 = 'AB'
string_length = 3
generator1 = unique_strings(alphabet1, string_length)

for s in list(generator1):
    print(s)

print('---------------')

# Example 2: Generate strings of increasing length
alphabet2 = '01'
generator2 = unique_strings(alphabet2)

for _ in range(10):
    print(next(generator2))

Transformers

Transformers are functions that reshape an object, while maintaining the consistency of the structure.

listify

Transforms complex objects into a list of dictionaries with key and value pairs.

from mappingtools.transformers import listify

wrapped_data = {'key1': {'subkey': 'value'}, 'key2': ['item1', 'item2']}
unwrapped_data = listify(wrapped_data)
print(unwrapped_data)
# output: [{'key': 'key1', 'value': [{'key': 'subkey', 'value': 'value'}]}, {'key': 'key2', 'value': ['item1', 'item2']}]

minify

The minify function is designed to shorten the keys of an object using a specified alphabet. This function can be particularly useful for reducing the size of data structures, making them more efficient for storage or transmission.

from mappingtools.transformers import minify

data = [
    {
        'first_name': 'John',
        'last_name': 'Doe',
        'age': 30,
        'address': {
            'street': '123 Main St',
            'city': 'New York',
            'state': 'CA'
        }
    },
    {
        'first_name': 'Jane',
        'last_name': 'Smith',
        'age': 25,
        'address': {
            'street': '456 Rodeo Dr',
            'city': 'Los Angeles',
            'state': 'CA'
        }
    }
]

# Minify the dictionary keys
minified_dict = minify(data)

print(minified_dict)
# [{'A': 'John', 'B': 'Doe', 'C': 30, 'D': {'E': '123 Main St', 'F': 'New York', 'G': 'CA'}}, {'A': 'Jane', 'B': 'Smith', 'C': 25, 'D': {'E': '456 Rodeo Dr', 'F': 'Los Angeles', 'G': 'CA'}}]

simplify

Converts objects to strictly structured dictionaries.

from collections import Counter
from dataclasses import dataclass
from datetime import datetime
from typing import Mapping

from mappingtools.transformers import simplify

data = {'key1': 'value1', 'key2': ['item1', 'item2']}
simplified_data = simplify(data)
print(simplified_data)
# Output: {'key1': 'value1', 'key2': ['item1', 'item2']}

counter = Counter({'a': 1, 'b': 2})
print(counter)
# Output: Counter({'b': 2, 'a': 1})

simplified_counter = simplify(counter)
print(simplified_counter)


# output: {'a': 1, 'b': 2}


@dataclass
class SampleDataClass:
    a: int
    b: int
    aa: str
    bb: str
    c: list[int]
    d: Mapping
    e: datetime


sample_datetime = datetime(2024, 7, 22, 21, 42, 17, 314159)
sample_dataclass = SampleDataClass(1, 2, '11', '22', [1, 2], {'aaa': 111, 'bbb': '222'}, sample_datetime)
print(sample_dataclass)
# output: SampleDataClass(a=1, b=2, aa='11', bb='22', c=[1, 2], d={'aaa': 111, 'bbb': '222'}, e=datetime.datetime(2024, 7, 22, 21, 42, 17, 314159))

simplified_sample_dataclass = simplify(sample_dataclass)
print(simplified_sample_dataclass)
# output: {'a': 1, 'aa': '11', 'b': 2, 'bb': '22', 'c': [1, 2], 'd': {'aaa': 111, 'bbb': '222'}, 'e': datetime.datetime(2024, 7, 22, 21, 42, 17, 314159)}

strictify

Applies a strict structural conversion to an object using optional converters for keys and values.

from mappingtools.transformers import strictify


def uppercase_key(key):
    return key.upper()


def double_value(value):
    return value * 2


data = {'a': 1, 'b': 2}
result = strictify(data, key_converter=uppercase_key, value_converter=double_value)
print(result)
# output: {'A': 2, 'B': 4}

stringify

Converts an object into a string representation by recursively processing it based on its type.

from mappingtools.transformers import stringify

data = {'key1': 'value1', 'key2': 'value2'}
result = stringify(data)

print(result)
# output: "key1=value1, key2=value2"

data = [1, 2, 3]
result = stringify(data)

print(result)
# output: "[1, 2, 3]"

Development

Ruff

ruff check src

ruff check tests

Test

Standard (cobertura) XML Coverage Report

python -m pytest tests -n auto --cov=src --cov-branch --doctest-modules --cov-report=xml

HTML Coverage Report

python -m pytest tests -n auto --cov=src --cov-branch --doctest-modules --cov-report=html