New Research: Supply Chain Attack on Axios Pulls Malicious Dependency from npm.Details → →
Socket
Book a DemoSign in
Socket
Book a DemoSign in

hypernotes

Package Overview
Dependencies
Maintainers
1
Versions
8
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

hypernotes - pypi Package Compare versions

Comparing version
0.1.1
 to
0.2.0
+25
-14
hypernotes.egg-info/PKG-INFO
Metadata-Version: 2.1
Name: hypernotes
Version: 0.1.1
Summary: hypernotes is a lightweight Python package for taking notes on your machine learning experiments
Version: 0.2.0
Summary: A lightweight Python package for taking notes on your machine learning experiments
Home-page: https://github.com/binste/hypernotes

@@ -9,3 +9,3 @@ Author: Stefan Binder

Description: # hypernotes
[![PyPI version](https://badge.fury.io/py/hypernotes.svg)](https://badge.fury.io/py/hypernotes)
[![PyPI version](http://img.shields.io/pypi/v/hypernotes.svg?style=flat-square&color=blue)](https://pypi.python.org/pypi/hypernotes/) [![Python versions](https://img.shields.io/pypi/pyversions/hypernotes.svg?style=flat-square&color=blue)]()

@@ -26,10 +26,10 @@ hypernotes is a lightweight Python package for taking notes on your machine learning experiments. It provides a simple way to store hyperparameters, their corresponding evaluation metrics, as well as additional information and retrieve them again later for analyzing. It is written in pure Python and requires no additional dependencies.

* information about the current state of your Git repository (if there is one) such as the last commit, current branch, etc.,
* start (upon initialization) and end time (call note.end() or add to store)
* start (upon initialization) and end datetime (call note.end() or add to store)
and it provides
* a useful default dictionary structure (just print a note instance and you will see what's inside)
* a useful default dictionary structure (print a note instance and you will see what's inside)
* access to the most commonly used dictionary keys as attributes for better auto-completion support and readability (see below, for example `note.metrics`)
The notes are then saved using as *Store* instance, which uses a json file. Due to this, you can only add json serializable objects + datetime.datetime instances to a *Note*.
The notes are then saved with a *Store* instance, which uses a json file. Due to this, you should only add json serializable objects + *datetime.datetime* instances to a *Note*.

@@ -91,3 +91,3 @@ A note is uniquely identifiable by its `identifier` attribute, which is the start datetime in ISO format.

## Update notes
If you want to update notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
If you want to update notes, you can do this either directly in the json file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
```python

@@ -104,3 +104,3 @@ notes = store.load()

## Remove notes
If you want to remove notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
If you want to remove notes, you can do this either directly in the json file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
```python

@@ -112,13 +112,23 @@ notes = store.load()

## View content of a store directly in your browser
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics, parameters, etc.
## View content of a store
### Directly in your browser (no additional dependencies)
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics and parameters.
```
$ python -m hypernotes hyperstore.json --view
$ python -m hypernotes hyperstore.json
```
This only requires a modern web browser as well as an internet connection to load the JQuery and Datatables Javascript libraries.
### pandas and QGrid
Another useful option might be to load the store as a pandas dataframe (see [Load notes](#load-notes)) and then use [Qgrid](https://github.com/quantopian/qgrid) in a Jupyter notebook.
## Bonus: Store additional objects in separate experiment folders
If you want to store larger artifacts of your experiment, such as a trained model, you could create a separate folder and use the identifier of a note as part of the name.
```python
experiment_folder = f"experiment_{note.identifier}"
```
You can then store any additional objects into this folder and it will be very easy to lather on link them again to the hyperparameters and metrics stored using hypernotes.
# Other tools
Check out tools such as MLflow, Sacred, DVC, etc. if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...
Check out tools such as [MLflow](https://mlflow.org/), [Sacred](https://sacred.readthedocs.io/en/latest/index.html), or [DVC](https://dvc.org/) if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...

@@ -133,2 +143,3 @@ # Development

Code is required to be formatted with [Black](https://github.com/python/black).
Keywords: machine learning,tracking,metrics,experiments,hyperparameters,model evaluation,data science

@@ -135,0 +146,0 @@ Platform: UNKNOWN

+123
-21
hypernotes/__init__.py

@@ -8,9 +8,12 @@ import argparse

import textwrap
import types
from datetime import datetime
from json import JSONEncoder
from unittest.mock import patch
from pathlib import Path
from typing import Any, Dict, List, Optional, Set, Union, Sequence, DefaultDict
from pprint import pformat
from typing import Any, Dict, List, Optional, Set, Union, Sequence, DefaultDict, Tuple
__version__ = "0.1.1"
__version__ = "0.2.0"

@@ -35,7 +38,27 @@

def __init__(
self, text: str = "", note_data: Optional[Dict[str, dict]] = None
self, text: str = "", content: Optional[Dict[str, dict]] = None
) -> None:
if note_data is not None:
super().__init__(note_data)
"""A small wrapper around dictionaries with a default structure, which can
be used like a normal dictionary, but additionally stores useful information
such as time and date, last commit and current branch of git repository,
path to Python executable, etc.
The most commonly used dictionary keys can be accessed as attributes
for better auto-completion support and readability.
Parameters
----------
text : str, optional (default="")
Can be used to store some descriptive text about your experiment
content : Optional[Dict[str, dict]], optional (default=None)
Mainly for internal use to create Note instances out of loaded
dictionaries from a Store. If content is passed, no additional information
is added on instantiation of the class (e.g. no start datetime).
"""
if content is not None:
super().__init__(content)
self._content_passed = True
else:
self._content_passed = False
self[self._text_key] = text

@@ -71,2 +94,3 @@ self._set_up_initial_structure()

def end(self) -> None:
"""Adds the current datetime as 'end_datetime' to the note"""
self[self._end_datetime_key] = datetime.now()

@@ -171,3 +195,11 @@

def __repr__(self) -> str:
# Code and idea for patching sorted to prevent sorting by
# dictionary keys come from:
# https://stackoverflow.com/a/55661095
with patch("builtins.sorted", new=lambda l, **_: l):
r = f"Note(content={pformat(dict(self))})"
return r
class BaseStore:

@@ -281,5 +313,11 @@ """The base store class. This class cannot be used directly and mostly acts

class Store(BaseStore):
"""Implements a store for notes based on a JSON file"""
def __init__(self, path: Union[str, Path]) -> None:
"""Stores and loads Note instances in and from a json file
def __init__(self, path: Union[str, Path]) -> None:
Parameters
----------
path : Union[str, Path]
Path to the json file. If it does not yet exist, a new one will be created,
else, the Store will interact with the existing file and modify it
"""
super().__init__()

@@ -295,4 +333,17 @@ self.path = _convert_to_path(path)

def load(self, return_dataframe: bool = False):
"""Returns the full store as List[Note] with the most recent
note first, or, if return_dataframe=True, as a Pandas dataframe.
"""Loads the entire json file and returns it as a list of Note instances
with the most recent note first. Optionally, a pandas dataframe can be
returned instead.
Parameters
----------
return_dataframe : bool, optional (default=False)
If True, a pandas dataframe is returned with one row per note,
where nested structures inside the notes are resolved as far as possible
and the keys are joined with "." to form column names. This requires
the pandas package to be installed.
Returns
-------
Either List[str] or pd.DataFrame, depending on value of return_dataframe
"""

@@ -310,3 +361,15 @@ loaded_notes = self._load()

def add(self, note: Note) -> None:
"""Adds a single note to the .json file of the store"""
"""Adds the given note to the .json file of the store
Parameters
----------
note : Note
The Note instance which should be added to the store. The note
needs to consist entirely of json serializable objects or
datetime.datetime instances
Returns
-------
None
"""
# As the whole json file needs to be loaded to add a new entry,

@@ -325,5 +388,21 @@ # changes made to the file between the call to self.load and

def update(self, notes: List[Note]) -> None:
"""Updates the passed in notes in the .json file of the store"""
notes_to_be_updated = notes
def update(self, notes: Union[Note, Sequence[Note]]) -> None:
"""Updates the passed in notes in the .json file of the store
Uses the identifier attribute of the notes to find the original ones
and replaces them
Parameters
----------
notes: Union[Note, Sequence[Note]]
One or more notes which should be updated
Returns
-------
None
"""
if isinstance(notes, Note):
notes_to_be_updated = [notes]
else:
notes_to_be_updated = list(notes)
# As the whole json file needs to be loaded to add a new entry,

@@ -335,3 +414,5 @@ # changes made to the file between the call to self.load and

# then insert new version of notes
assert self._notes_are_subset(notes_subset=notes, all_notes=stored_notes), (
assert self._notes_are_subset(
notes_subset=notes_to_be_updated, all_notes=stored_notes
), (
"Some of the notes do not yet exist in the store."

@@ -341,3 +422,3 @@ + " Add them with the .add method. Nothing was updated."

new_stored_notes = self._filter_notes(
notes_to_filter_out=notes, all_notes=stored_notes
notes_to_filter_out=notes_to_be_updated, all_notes=stored_notes
)

@@ -347,6 +428,24 @@ new_stored_notes.extend(notes_to_be_updated)

def remove(self, notes: List[Note]) -> None:
"""Removes passed in notes from store"""
def remove(self, notes: Union[Note, Sequence[Note]]) -> None:
"""Removes passed in notes from store
Uses the identifier attribute of the notes to find the original ones
Parameters
----------
notes: Union[Note, Sequence[Note]]
One or more notes which should be removed
Returns
-------
None
"""
if isinstance(notes, Note):
notes_to_be_removed = [notes]
else:
notes_to_be_removed = list(notes)
stored_notes = self.load()
assert self._notes_are_subset(notes_subset=notes, all_notes=stored_notes), (
assert self._notes_are_subset(
notes_subset=notes_to_be_removed, all_notes=stored_notes
), (
"Some of the notes do not yet exist in the store."

@@ -357,3 +456,3 @@ + " Nothing was removed. Only pass in notes which already"

new_stored_notes = self._filter_notes(
notes_to_filter_out=notes, all_notes=stored_notes
notes_to_filter_out=notes_to_be_removed, all_notes=stored_notes
)

@@ -408,3 +507,6 @@ self._save_notes(new_stored_notes)

def __repr__(self) -> str:
return f"Store('{self.path}')"
class DatetimeJSONEncoder(JSONEncoder):

@@ -443,3 +545,3 @@ """Encodes datetime objects as a dictionary

def _parse_datetime(dt_str: str) -> datetime:
return datetime.fromisoformat(dt_str)
return datetime.strptime(dt_str, "%Y-%m-%dT%H:%M:%S.%f")

@@ -454,3 +556,3 @@

def _raw_dicts_to_notes(raw_dicts: List[dict]) -> List[Note]:
converted_notes = [Note(note_data=raw_note_data) for raw_note_data in raw_dicts]
converted_notes = [Note(content=raw_content) for raw_content in raw_dicts]
return converted_notes

@@ -457,0 +559,0 @@ 

+41
-25
hypernotes/__main__.py

@@ -6,2 +6,3 @@ import argparse

import webbrowser
import sys
from datetime import datetime

@@ -110,2 +111,3 @@ from pathlib import Path

def do_GET(self):
html = _format_notes_as_html(store.load())
self.send_response(200)

@@ -117,28 +119,42 @@ self.send_header("Content-type", "text/html")

if __name__ == "__main__":
# TODO: Add text
parser = argparse.ArgumentParser("")
parser.add_argument("store_path", type=str)
parser.add_argument("--view", action="store_true")
group = parser.add_mutually_exclusive_group(required=False)
parser.add_argument("--port", type=int, default=8080)
parser.add_argument("--no-browser", action="store_true")
def _parse_args(args):
parser = argparse.ArgumentParser(
"This command-line interface can be used to"
+ " get a quick glance into a store.\n\nIt will start an http server and"
+ " automatically open the relevant page in your web browser."
+ " The page will contain an interactive table showing the most relevant"
+ " information of all notes in the store such as metrics, parameters, etc."
)
parser.add_argument("store_path", type=str, help="path to json store")
parser.add_argument(
"--port", type=int, default=8080, help="port for http server (default=8080)"
)
parser.add_argument(
"--no-browser",
action="store_true",
help="can be passed to prevent automatic opening of web browser",
)
args = parser.parse_args()
if args.view:
store = Store(args.store_path)
html = _format_notes_as_html(store.load())
return parser.parse_args(args)
try:
host = "localhost"
server = HTTPServer((host, args.port), HTMLResponder)
url = f"http://localhost:{args.port}"
print(
f"Started server on {url}. Server can be stopped with control+c / ctrl+c"
)
if not args.no_browser:
webbrowser.open_new_tab(url)
server.serve_forever()
except KeyboardInterrupt:
print("Keyboard interrupt recieved. Shutting down...")
server.socket.close()
def main(raw_args):
global store
args = _parse_args(raw_args)
store = Store(args.store_path)
try:
host = "localhost"
server = HTTPServer((host, args.port), HTMLResponder)
url = f"http://localhost:{args.port}"
print(f"Started server on {url}. Server can be stopped with control+c / ctrl+c")
if not args.no_browser:
webbrowser.open_new_tab(url)
server.serve_forever()
except KeyboardInterrupt:
print("\nKeyboard interrupt recieved. Shutting down...")
server.socket.close()
if __name__ == "__main__":
main(sys.argv[1:])
+25
-14
PKG-INFO
Metadata-Version: 2.1
Name: hypernotes
Version: 0.1.1
Summary: hypernotes is a lightweight Python package for taking notes on your machine learning experiments
Version: 0.2.0
Summary: A lightweight Python package for taking notes on your machine learning experiments
Home-page: https://github.com/binste/hypernotes

@@ -9,3 +9,3 @@ Author: Stefan Binder

Description: # hypernotes
[![PyPI version](https://badge.fury.io/py/hypernotes.svg)](https://badge.fury.io/py/hypernotes)
[![PyPI version](http://img.shields.io/pypi/v/hypernotes.svg?style=flat-square&color=blue)](https://pypi.python.org/pypi/hypernotes/) [![Python versions](https://img.shields.io/pypi/pyversions/hypernotes.svg?style=flat-square&color=blue)]()

@@ -26,10 +26,10 @@ hypernotes is a lightweight Python package for taking notes on your machine learning experiments. It provides a simple way to store hyperparameters, their corresponding evaluation metrics, as well as additional information and retrieve them again later for analyzing. It is written in pure Python and requires no additional dependencies.

* information about the current state of your Git repository (if there is one) such as the last commit, current branch, etc.,
* start (upon initialization) and end time (call note.end() or add to store)
* start (upon initialization) and end datetime (call note.end() or add to store)
and it provides
* a useful default dictionary structure (just print a note instance and you will see what's inside)
* a useful default dictionary structure (print a note instance and you will see what's inside)
* access to the most commonly used dictionary keys as attributes for better auto-completion support and readability (see below, for example `note.metrics`)
The notes are then saved using as *Store* instance, which uses a json file. Due to this, you can only add json serializable objects + datetime.datetime instances to a *Note*.
The notes are then saved with a *Store* instance, which uses a json file. Due to this, you should only add json serializable objects + *datetime.datetime* instances to a *Note*.

@@ -91,3 +91,3 @@ A note is uniquely identifiable by its `identifier` attribute, which is the start datetime in ISO format.

## Update notes
If you want to update notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
If you want to update notes, you can do this either directly in the json file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
```python

@@ -104,3 +104,3 @@ notes = store.load()

## Remove notes
If you want to remove notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
If you want to remove notes, you can do this either directly in the json file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
```python

@@ -112,13 +112,23 @@ notes = store.load()

## View content of a store directly in your browser
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics, parameters, etc.
## View content of a store
### Directly in your browser (no additional dependencies)
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics and parameters.
```
$ python -m hypernotes hyperstore.json --view
$ python -m hypernotes hyperstore.json
```
This only requires a modern web browser as well as an internet connection to load the JQuery and Datatables Javascript libraries.
### pandas and QGrid
Another useful option might be to load the store as a pandas dataframe (see [Load notes](#load-notes)) and then use [Qgrid](https://github.com/quantopian/qgrid) in a Jupyter notebook.
## Bonus: Store additional objects in separate experiment folders
If you want to store larger artifacts of your experiment, such as a trained model, you could create a separate folder and use the identifier of a note as part of the name.
```python
experiment_folder = f"experiment_{note.identifier}"
```
You can then store any additional objects into this folder and it will be very easy to lather on link them again to the hyperparameters and metrics stored using hypernotes.
# Other tools
Check out tools such as MLflow, Sacred, DVC, etc. if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...
Check out tools such as [MLflow](https://mlflow.org/), [Sacred](https://sacred.readthedocs.io/en/latest/index.html), or [DVC](https://dvc.org/) if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...

@@ -133,2 +143,3 @@ # Development

Code is required to be formatted with [Black](https://github.com/python/black).
Keywords: machine learning,tracking,metrics,experiments,hyperparameters,model evaluation,data science

@@ -135,0 +146,0 @@ Platform: UNKNOWN

+24
-12
README.md
# hypernotes
[![PyPI version](https://badge.fury.io/py/hypernotes.svg)](https://badge.fury.io/py/hypernotes)
[![PyPI version](http://img.shields.io/pypi/v/hypernotes.svg?style=flat-square&color=blue)](https://pypi.python.org/pypi/hypernotes/) [![Python versions](https://img.shields.io/pypi/pyversions/hypernotes.svg?style=flat-square&color=blue)]()

@@ -18,10 +18,10 @@ hypernotes is a lightweight Python package for taking notes on your machine learning experiments. It provides a simple way to store hyperparameters, their corresponding evaluation metrics, as well as additional information and retrieve them again later for analyzing. It is written in pure Python and requires no additional dependencies.

* information about the current state of your Git repository (if there is one) such as the last commit, current branch, etc.,
* start (upon initialization) and end time (call note.end() or add to store)
* start (upon initialization) and end datetime (call note.end() or add to store)
and it provides
* a useful default dictionary structure (just print a note instance and you will see what's inside)
* a useful default dictionary structure (print a note instance and you will see what's inside)
* access to the most commonly used dictionary keys as attributes for better auto-completion support and readability (see below, for example `note.metrics`)
The notes are then saved using as *Store* instance, which uses a json file. Due to this, you can only add json serializable objects + datetime.datetime instances to a *Note*.
The notes are then saved with a *Store* instance, which uses a json file. Due to this, you should only add json serializable objects + *datetime.datetime* instances to a *Note*.

@@ -83,3 +83,3 @@ A note is uniquely identifiable by its `identifier` attribute, which is the start datetime in ISO format.

## Update notes
If you want to update notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
If you want to update notes, you can do this either directly in the json file containing the notes, or load the notes as described above, change the relevant ones, and pass them to the `update` method.
```python

@@ -96,3 +96,3 @@ notes = store.load()

## Remove notes
If you want to remove notes, you can do this either directly in the *json* file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
If you want to remove notes, you can do this either directly in the json file containing the notes, or load the notes as described above, and pass the ones which you want to remove to the `remove` method.
```python

@@ -104,13 +104,23 @@ notes = store.load()

## View content of a store directly in your browser
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics, parameters, etc.
## View content of a store
### Directly in your browser (no additional dependencies)
To get a quick glance into a store, you can use the following command. It will start an http server and automatically open the relevant page in your web browser. The page contains an interactive table which shows the most relevant information of all notes in the store such as metrics and parameters.
```
$ python -m hypernotes hyperstore.json --view
$ python -m hypernotes hyperstore.json
```
This only requires a modern web browser as well as an internet connection to load the JQuery and Datatables Javascript libraries.
### pandas and QGrid
Another useful option might be to load the store as a pandas dataframe (see [Load notes](#load-notes)) and then use [Qgrid](https://github.com/quantopian/qgrid) in a Jupyter notebook.
## Bonus: Store additional objects in separate experiment folders
If you want to store larger artifacts of your experiment, such as a trained model, you could create a separate folder and use the identifier of a note as part of the name.
```python
experiment_folder = f"experiment_{note.identifier}"
```
You can then store any additional objects into this folder and it will be very easy to lather on link them again to the hyperparameters and metrics stored using hypernotes.
# Other tools
Check out tools such as MLflow, Sacred, DVC, etc. if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...
Check out tools such as [MLflow](https://mlflow.org/), [Sacred](https://sacred.readthedocs.io/en/latest/index.html), or [DVC](https://dvc.org/) if you need better multi-user capabilities, more advanced reproducibility features, dataset versioning, ...

@@ -124,1 +134,3 @@ # Development

```
Code is required to be formatted with [Black](https://github.com/python/black).
+2
-1
setup.py
"""
Publish a new version:
- Change version number in hypernotes/__init__.py
$ git tag vX.Y.Z -m "Release X.Y.Z"

@@ -24,3 +25,3 @@ $ git push --tags

description=(
"hypernotes is a lightweight Python package for taking notes on your machine learning experiments"
"A lightweight Python package for taking notes on your machine learning experiments"
),

@@ -27,0 +28,0 @@ long_description=README,