ReplayTables
Benchmarks
Getting started
Installation:
pip install ReplayTables-andnp
Basic usage:
from typing import NamedTuple
from ReplayTables.ReplayBuffer import ReplayBuffer
class Data(NamedTuple):
x: np.ndarray
a: np.ndarray
r: np.ndarray
buffer = ReplayBuffer(
max_size=100_000,
structure=Data,
rng=np.random.default_rng(0),
)
buffer.add(Data(x, a, r))
batch = buffer.sample(32)
print(batch.x.shape)
print(batch.a.shape)
print(batch.r.shape)
Prioritized Replay
An implementation of prioritized experience replay from
Schaul, Tom, et al. "Prioritized experience replay." ICLR (2016).
The defaults for this implementation strictly adhere to the defaults from the original work, though several configuration options are available.
from typing import NamedTuple
from ReplayTables.PER import PERConfig, PrioritizedReplay
class Data(NamedTuple):
a: float
b: float
config = PERConfig(
new_priority_mode='max',
uniform_probability=1e-3,
priority_exponent=0.5,
max_decay=1,
)
buffer = PrioritizedReplay(
max_size=100_000,
structure=Data,
rng=np.random.default_rng(0),
config=config,
)
buffer.add(Data(a=1, b=2))
buffer.add(Data(a=1, b=2), priority=1.3)
batch = buffer.sample(32)