Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

ldimbenchmark

Package Overview
Dependencies
Maintainers
1
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

ldimbenchmark

  • 0.2.67
  • PyPI
  • Socket score

Maintainers
1

ldimbenchmark version Documentation badge

LDIMBenchmark

Leakage Detection and Isolation Methods Benchmark

Instead of collecting all the different dataset to benchmark different methods on. We wanted to create a Benchmarking Tool which makes it easy to reproduce the results of the different methods locally on your own dataset.

It provides a close to real-world conditions environment and forces researchers to provide a reproducible method implementation, which is supposed to run automated on any input dataset, thus hindering custom solutions which work well in one specific case.

Usage

Installation

pip install ldimbenchmark

Python

from ldimbenchmark.datasets import DatasetLibrary, DATASETS
from ldimbenchmark import (
    LDIMBenchmark,
    BenchmarkData,
    BenchmarkLeakageResult,
)
from ldimbenchmark.classes import LDIMMethodBase
from typing import List

class YourCustomLDIMMethod(LDIMMethodBase):
    def __init__(self):
        super().__init__(
            name="YourCustomLDIMMethod",
            version="0.1.0"
        )

    def train(self, data: BenchmarkData):
        pass

    def detect(self, data: BenchmarkData) -> List[BenchmarkLeakageResult]:
        return [
            {
                "leak_start": "2020-01-01",
                "leak_end": "2020-01-02",
                "leak_area": 0.2,
                "pipe_id": "test",
            }
        ]

    def detect_datapoint(self, evaluation_data) -> BenchmarkLeakageResult:
        return {}


datasets = DatasetLibrary("datasets").download(DATASETS.BATTLEDIM)

local_methods = [YourCustomLDIMMethod()]

hyperparameters = {}

benchmark = LDIMBenchmark(
    hyperparameters, datasets, results_dir="./benchmark-results"
)
benchmark.add_local_methods(local_methods)

benchmark.run_benchmark()

benchmark.evaluate()

CLI

ldimbenchmark --help

Roadmap

  • v1: Just Leakage Detection
  • v2: Provides Benchmark of Isolation Methods

https://mathspp.com/blog/how-to-create-a-python-package-in-2022

Development

https://python-poetry.org/docs/basic-usage/

# python 3.10
# Use Environment
poetry config virtualenvs.in-project true
poetry shell
poetry install --without ci # --with ci


# Test
poetry build
cp -r dist tests/dist
cd tests
docker build . -t testmethod
pytest -s -o log_cli=true
pytest tests/test_derivation.py -k 'test_mything'
pytest --testmon
pytest --snapshot-update

# Pytest watch
ptw
ptw -- --testmon

# Watch a file during development
npm install -g nodemon
nodemon -L experiments/auto_hyperparameter.py

# Test-Publish
poetry config repositories.testpypi https://test.pypi.org/legacy/
poetry config http-basic.testpypi __token__ pypi-your-api-token-here
poetry build
poetry publish -r testpypi

# Real Publish
poetry config pypi-token.pypi pypi-your-token-here

Documentation

https://squidfunk.github.io/mkdocs-material/ https://click.palletsprojects.com/en/8.1.x/

poetry shell
mkdocs serve

TODO

LDIMBenchmark: Data Cleansing before working with them

  • per sensor type, e.g. waterflow (cut of at 0)
  • removing datapoints which are clearly a malfunction

FAQs


Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc