pyiotools

PLEASE NOTE:

This library is currently still under development. The API will likely undergo significant changes that may break any code you write with it. The documentation will fall out of sync with the updates regularly until development slows down. Use it at your own risk.

Overview

Provides several utilities for handling I/O:

The IOHandler class

• Api similar to argparse.ArgumentParser(). Must be used as a context manager, and while in scope the Argument.add() method will act equivalent to ArgumentParser.add_argument().
• IOHandler.process() (equivalent to ArgumentParser.parse_args()) returns a subtypes.Dict holding the argument values if no callback is provided to the IOHandler() constructor, otherwise it passes on the return value of the callback function, which will be passed a Dict as its single positional argument.
• Has various run-modes (in the provided RunMode Enum) RunMode.SMART will attempt to choose the appropriate run-mode for the situation.
• Under RunMode.COMMANDLINE argparse is used under-the-hood to process the sys.argv arguments, but with additional features and custom-built help interface that is more readable (and way prettier!)
• Under RunMode.GUI, it will programatically build a GUI to collect user input, with widgets picked based on the 'argtype' argument of Argument(). The argument defaults can be overriden at point-in-time by passing a dict of argument name-value pairs directly to IOHandler.process(). Further calls to IOHandler.process() will still have the base defaults
• Under RunMode.PROGRAMMATIC, the argument values can be passed directly to IOHandler.process() as a dict of argument name-value pairs
• IOHandler.add_subhandler() will add a new subhandler which will act as a subcommand under RunMode.COMMANDLINE, and will act as a tabbed sheet under RunMode.GUI. The handlers exist in a hierarchy, meaning that arguments passed to all parents on the way to the lowest child sheet (on the gui) or final used subcommand (in the commandline) are still handled.

The Argument class

• The Argument() constructor arguments tell the IOHandler how to handle nullability, default values, implicit coercion to the right type, whether the argument is optional, commandline aliases, conditions, dependencies, etc.

• An ArgType Enum is provided to be passed to the Argument(argtype=) constructor argument. This will let the IOHandler perform type checking and coercion. Currently the recognized types are:

  member	with IOHandler(subtypes=True)	with IOHandler(subtypes=False)
  STRING	subtypes.Str	str
  INTEGER	int	int
  FLOAT	float	float
  DECIMAL	decimal.Decimal	decimal.Decimal
  BOOLEAN	bool	bool
  LIST	subtypes.List	list
  DICT	subtypes.Dict	dict
  SET	set	set
  PATH	pathlib.Path	pathlib.Path
  FILE	pathmagic.File	pathmagic.File
  DIR	pathmagic.Dir	pathmagic.Dir
  DATETIME	subtypes.DateTime	datetime.datetime
  FRAME	subtypes.Frame	pandas.DataFrame

The Validate class

• An accessor class granting access to several Validator classes through attribute access.
• Currently supports type checking and implicit coercion of the input value to the following supported types (int, float, bool, str, list, set, dict, subtypes.DateTime, pathlib.Path, pathmagic.File, pathmagic.Dir)
• Its attributes are: Validate.Int, Validate.Float, Validate.Bool, Validate.Str, Validate.List, Validate.Set, Validate.Dict, Validate.DateTime, Validate.Path, Validate.File, Validate.Dir

The Validator classes

• Currently there are IntegerValidator, FloatValidator, BoolValidator, StringValidator, ListValidator, SetValidator, DictionaryValidator, DateTimeValidator, PathValidator, FileValidator, DirValidator
• Some of these validators are implemented as a wrapper over typepy, but the api is different.
• Validators can handle nullability as desired.
• Some validators have additional validation methods to check for values in valid ranges. For example: Validate.Int().max_value(7).is_valid(9) would return False.
• Additional conditions can be added to a validator by passing callbacks that return boolean values to Validator.add_condition()
• The validator can be reused for any number of values once initially set up.
• ListValidator and DictionaryValidator will coerce strings by using eval (safely), rather than coercing a string to a list by calling list() on it

The Gui class and its various template subclasses

• Gui class and several template subclasses that can be used alongside the various WidgetManager objects to easily set up a GUI, with the exact internals of the underlying QT classes abstracted away behind a consistent API. Makes it very quick and easy to set up a simple GUI. Is a thin wrapper around PyQt5.
• ThreePartGui class for quickly setting up Horizontal-Vertical-Horizontal guis
• HTMLGui class for Rendering HTML in a separate window

The WidgetManager class and its various widget subclasses

• Currently supports the following widgets: Label, Button, Checkbox, CheckBar, DropDown, Entry, Text, FileSelect, DirSelect, Calendar, DateTimeEdit, HtmlDisplay, ProgressBar, Table, ListTable, DictTable, WidgetFrame, HorizontalFrame, VerticalFrame
• Have a consistent API primarily using the properties WidgetManager.active, WidgetManager.state, WidgetManager.text, and WidgetManager.parent.

The Console class

• Offer choices inveractively on the console, allowing navigation using arrow keys
• Supports multi-select
• Offer YES/NO
• Hide/show console
• Clear existing lines from console

The Script class

• Uses a metaclass that wraps every method and the methods of inner classes (recursively) in a profiler, showing duration, arguments, and return value of each method call, and a repr() of the script object
• Writes this profiling information and print() statements to a log file
• Upon exiting the constructor, optionally serializes the object to the same directory as the log
• Any **kwargs passed to the constructor are stored in the Script.arguments attribute
• The Script.name attribute is automatically set to the name of the file the class is defined in
• For use with IOHandler, the Script.run_mode attribute is automatically 'smart' by default, but can be overriden by setting it as a class attribute

The Cache class

• Serializes any python object that can be pickled by Dill into a file
• Interface similar to a dict for interacting with the items in the cache: Cache.put(), Cache.get(), Cache.pop(), and Cache.setdefault()

The Serializer class

• Serialize/deserialize any object that is pickleable by Dill
• Discard unpickleable attributes recursively and replace them with LostObject instances

The Secrets class

• Serialize, then encrypt any python object and write it to a file and vice-versa.
• Encryption key must be set before first use. It will be persisted to a json config file at an os-appropriate appdir.

Installation

To install use pip:

$ pip install pyiotools

Or clone the repo:

$ git clone https://github.com/matthewgdv/iotools.git
$ python setup.py install

Usage

Usage examples coming soon.

Contributing

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

You can contribute in many ways:

Report Bugs

Report bugs at https://github.com/matthewgdv/iotools/issues

If you are reporting a bug, please include:

• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.

Fix Bugs

Look through the GitHub issues for bugs. Anything tagged with "bug" and "help wanted" is open to whoever wants to implement a fix for it.

Implement Features

Look through the GitHub issues for features. Anything tagged with "enhancement" and "help wanted" is open to whoever wants to implement it.

Write Documentation

The repository could always use more documentation, whether as part of the official docs, in docstrings, or even on the web in blog posts, articles, and such.

Submit Feedback

The best way to send feedback is to file an issue at https://github.com/matthewgdv/iotools/issues.

If you are proposing a new feature:

• Explain in detail how it would work.
• Keep the scope as narrow as possible, to make it easier to implement.
• Remember that this is a volunteer-driven project, and that contributions are welcome :)

Get Started!

Before you submit a pull request, check that it meets these guidelines:

1. If the pull request adds functionality, it should include tests and the docs should be updated. Write docstrings for any functions that are part of the external API, and add the feature to the README.md.

2. If the pull request fixes a bug, tests should be added proving that the bug has been fixed. However, no update to the docs is necessary for bugfixes.

3. The pull request should work for the newest version of Python (currently 3.7). Older versions may incidentally work, but are not officially supported.

4. Inline type hints should be used, with an emphasis on ensuring that introspection and autocompletion tools such as Jedi are able to understand the code wherever possible.

5. PEP8 guidelines should be followed where possible, but deviations from it where it makes sense and improves legibility are encouraged. The following PEP8 error codes can be safely ignored: E121, E123, E126, E226, E24, E704, W503

6. This repository intentionally disallows the PEP8 79-character limit. Therefore, any contributions adhering to this convention will be rejected. As a rule of thumb you should endeavor to stay under 200 characters except where going over preserves alignment, or where the line is mostly non-algorythmic code, such as extremely long strings or function calls.