Security News
Fluent Assertions Faces Backlash After Abandoning Open Source Licensing
Fluent Assertions is facing backlash after dropping the Apache license for a commercial model, leaving users blindsided and questioning contributor rights.
Extremely fast time series downsampling 📈 for visualization, written in Rust.
CPython has the infamous Global Interpreter Lock, which prevents several threads from executing Python bytecode in parallel. This makes threading in Python a bad fit for CPU-bound tasks and often forces developers to accept the overhead of multiprocessing.In Rust - which is a compiled language - there is no GIL, so CPU-bound tasks can be parallelized (with Rayon) with little to no overhead.
x
: f32
, f64
, i16
, i32
, i64
, u16
, u32
, u64
, datetime64
, timedelta64
y
: f16
, f32
, f64
, i8
, i16
, i32
, i64
, u8
, u16
, u32
, u64
, datetime64
, timedelta64
, bool
f16
argminmax is 200-300x faster than numpyf16
is *not* hardware supported (i.e., no instructions for f16) by most modern CPUs!! f16
to i16
is sufficient. This mapping allows to use the hardware supported scalar and SIMD i16
instructions - while not producing any memory overhead 🎉 pip install tsdownsample
from tsdownsample import MinMaxLTTBDownsampler
import numpy as np
# Create a time series
y = np.random.randn(10_000_000)
x = np.arange(len(y))
# Downsample to 1000 points (assuming constant sampling rate)
s_ds = MinMaxLTTBDownsampler().downsample(y, n_out=1000)
# Select downsampled data
downsampled_y = y[s_ds]
# Downsample to 1000 points using the (possible irregularly spaced) x-data
s_ds = MinMaxLTTBDownsampler().downsample(x, y, n_out=1000)
# Select downsampled data
downsampled_x = x[s_ds]
downsampled_y = y[s_ds]
Each downsampling algorithm is implemented as a class that implements a downsample
method.
The signature of the downsample
method:
downsample([x], y, n_out, **kwargs) -> ndarray[uint64]
Arguments:
x
is optionalx
and y
are both positional argumentsn_out
is a mandatory keyword argument that defines the number of output values***kwargs
are optional keyword arguments (see table below):
parallel
: whether to use multi-threading (default: False
)TSDOWNSAMPLE_MAX_THREADS
ENV var (e.g. os.environ["TSDOWNSAMPLE_MAX_THREADS"] = "4"
)Returns: a ndarray[uint64]
of indices that can be used to index the original data.
*When there are gaps in the time series, fewer than n_out
indices may be returned.
The following downsampling algorithms (classes) are implemented:
Downsampler | Description | **kwargs |
---|---|---|
MinMaxDownsampler | selects the min and max value in each bin | parallel |
M4Downsampler | selects the min, max, first and last value in each bin | parallel |
LTTBDownsampler | performs the Largest Triangle Three Buckets algorithm | parallel |
MinMaxLTTBDownsampler | (new two-step algorithm 🎉) first selects n_out * minmax_ratio min and max values, then further reduces these to n_out values using the Largest Triangle Three Buckets algorithm | parallel , minmax_ratio * |
*Default value for minmax_ratio
is 4, which is empirically proven to be a good default. More details here: https://arxiv.org/abs/2305.00332
This library supports two NaN
-policies:
NaN
s (NaN
s are ignored during downsampling).NaN
once there is at least one present in the bin of the considered data.Omit NaN s | Return NaN s |
---|---|
MinMaxDownsampler | NaNMinMaxDownsampler |
M4Downsampler | NaNM4Downsampler |
MinMaxLTTBDownsampler | NaNMinMaxLTTBDownsampler |
LTTBDownsampler |
Note that NaNs are not supported for
x
-data.
Assumes;
x
-data is (non-strictly) monotonic increasing (i.e., sorted)NaN
s in x
-data👤 Jeroen Van Der Donckt
FAQs
Time series downsampling in rust
We found that tsdownsample demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Fluent Assertions is facing backlash after dropping the Apache license for a commercial model, leaving users blindsided and questioning contributor rights.
Research
Security News
Socket researchers uncover the risks of a malicious Python package targeting Discord developers.
Security News
The UK is proposing a bold ban on ransomware payments by public entities to disrupt cybercrime, protect critical services, and lead global cybersecurity efforts.