tigerbeetle-node - npm Package Versions

published 0.16.10 • 3 months ago

Changelog

TigerBeetle 0.16.10

Released: 2024-10-21

Safety And Performance

#2414

Improve performance & availability during view change by ensuring a replica only repairs the portion of the WAL that is required to become primary, instead of repairing it in its entirety.
#2412

Add a unit test for Zig's stdlib sort.

Stable sort is critical for compaction correctness. Zig stdlib does have a sort fuzz test, but it doesn't cover the presorted subarray case, and doesn't check arrays much larger than the sort algorithm's on-stack cache.
#2413

Fix a bug in the MessageBus wherein connections weren't being terminated during client teardown.

Internals

#2405

Fix a bug in the benchmark wherein the usage of --account-count-hot was broken when used in conjunction with the uniform distribution.
#2409

Revamp the core_missing_prepares liveness-mode check to correctly check for the prepares that a replica should repair (after #2414).

TigerTracks 🎧

Last Train Home

0.16.9

published 0.16.9 • 3 months ago

Changelog

TigerBeetle 0.16.9

Released: 2024-10-15

Safety And Performance

#2394, #2401

TigerBeetle clients internally batch operations for improved performance. Fix a bug where an unclosed link chain could be batched before another linked chain, causing them to be treated as one long linked chain. Additionally, prevent non-batchable requests from sharing packets entirely.
#2398

AMOUNT_MAX is used as a sentinel value for things like balancing transfers to specify moving as much as possible. Correct and fix its value in the Java client. Thanks @tKe!

Features

#2274

Improve the benchmark by adding Zipfian distributed random numbers, to better simulate realistic conditions and as a precursor to approximating YCSB.
#2393

Previously, TigerBeetle's clients disallowed empty batches locally, before the request was even sent to the cluster. However, this is actually a valid protocol message - even if it's not used by the current state machine - so allow empty batches to be sent from clients.
#2384

Revamp client documentation so that each snippet is self-contained, and standardize it across all languages.

Internals

#2404

Give the DevHub a fresh coat of paint, and fix passing seeds being blue in dark mode.
#2408, #2400, #2391, #2399, #2402, #2385

Improve VOPR logging and fix a few failing seeds.

TigerTracks 🎧

99 Luftballons

0.16.8

published 0.16.8 • 3 months ago

Changelog

TigerBeetle 0.16.8

Released: 2024-10-07

Safety And Performance

#2359

Significantly reduced P100 latency by incrementally spreading the mutable table's sort during compaction. This leverages the optimization of sort algorithms for processing sequences of already sorted sub-arrays.
#2367

Improve the workload generator to support concurrent tests with different ledgers.
#2382, #2363

Fix VOPR seeds. For more awesome details about the backstory and solutions to these issues, please refer to the PR.

Features

#2358

Update the REPL to support representing the maximum integer value as -0, serving as the AMOUNT_MAX sentinel. Additionally, other negative values such as -1 can be used to represent maxInt - 1.

Also, include support for hexadecimal numbers for more convenient inputting of GUID/UUID literals (e.g. 0xa1a2a3a4_b1b2_c1c2_d1d2_e1e2e3e4e5e6).

Allow the timestamp field to be set, enabling the REPL to be used for imported events.

Internals

#2376

Use zig fetch as a replacement for downloading files, removing dependence on external tools.
#2383

Port of Rust's dbg! macro to Zig, and the corresponding CI validation to prevent code using it from being merged into main! 😎
#2370, #2373

Verify the release versions included in the multiversion binary pack at build time (not only during runtime) and improve the tigerbeetle version --verbose command's multiversion output.
#2369

Fix a multiversioning issue where the binary size exceeded the read buffer, failing to parse the executable header.
#2380

Consistently use transient_error instead of transient_failure and cleanup the StateMachine code.
#2379

Add missing links to the operations query_accounts and query_transfers in the documentation and include the declaration for QueryFilter and QueryFilterFlags in the tb_client.h header.
#2333

Clearer error message when the replica crashes due to a data file being too large, instructing the operator to increase the memory allocated for the manifest log.

TigerTracks 🎧

Creep

0.16.4

published 0.16.4 • 4 months ago

Changelog

TigerBeetle 0.16.4

Released: 2024-09-30

This release introduces "transient errors": error codes for create_transfers which depend on the state of the database (e.g. exceeds_credits). Going forward, a transfer that fails with a transient error will not succeed if retried.

See the API tracking issue and the documentation for more details.

Safety And Performance

#2345

Reduce chance of recovering_head status by recovering from torn writes in the WAL. This improves the availability of the cluster, as recovering_head replicas cannot participate in consensus until after they repair.

Features

#2335

Ensure idempotence for create_transfers' "transient errors" with new result code id_already_failed. In particular, this guards against surprising behavior when the client is running in a stateless API service.

Internals

#2334, #2362

Fix VOPR false positives.
#2115

Fix multiple commands for non-interactive REPL.

TigerTracks 🎧

Lookin' Out My Back Door

0.16.3

published 0.16.3 • 4 months ago

Changelog

TigerBeetle 0.16.3

Released: 2024-09-23

Safety And Performance

#2313

Improve cluster availability by more aggressive recovery for crashes that happen while a replica is checkpointing.
#2328

Add a more efficient recipe for balance-conditional transfers. A balance-conditional transfer is a transfer that succeeds only if the source account has more than a threshold amount of funds in it.

Features

#2327

Add a new recipe for enforcing debits_must_not_exceed_credits on some subset of transfers (this is a special case of a balance-conditional transfer, with the threshold value being equal to transferred amount).

Internals

#2330

Add triaged issue label to prevent newly opened issues from slipping through the cracks.
#2332

Add CI check for dead code.
#2323

Cleanup the source tree by removing top-level tools directory.
#2316

Make sure that process-spawning API used for build-time "scripting" consistently reports errors when the subprocess fails or hangs.

TigerTracks 🎧

Prelude in G Major

0.16.2

published 0.16.2 • 4 months ago

Changelog

TigerBeetle 0.16.2

Released: 2024-09-16

Safety And Performance

#2312

Tighten up the VSR assertions so the transition to .recovering_head can only be called from the .recovering status.
#2311

Make the primary abdicate if it is unable to process requests due to a broken clock.
#2260

Smoke integration test using the real multiversion binary.
#2270

Workload generator based on the Java client to be used in integration tests (i.e. Antithesis).

Features

#2298, #2307, #2304, #2309, #2321

Remove Tracy integration and dependencies. Add JSON traces for events with multiple running instances, such as IO, lookups, and scans.
#2300

Add the ability to filter by user_data_{128,64,32} and code in get_account_transfers and get_account_balances.

Internals

#2314, #2315

Mute the log on stderr when building client libraries. Reduce the log's severity of some entries logged as .err to .warn for less noise when running with log_level = .err.
#2310

Document and explain how time works in TigerBeetle ⏱️.
#2291

Refactor Forest.compact and remove some dead code.
#2294

Rewrite commit_dispatch, a chain of asynchronous stages calling each other, as a state machine implementation that resembles linear control flow that is much easier to read.
#2306

Fix the Node.js example that was using an incorrect enum flag. Thanks for the heads up @jorispz!
#2319

Update outdated scripts in HACKING.md.
#2317

Use git timestamps to build Docker images. This is a requirement for being deterministic in CI.
#2318

Devhub link to pending code reviews.

TigerTracks 🎧

What's On Your Mind

0.16.1

published 0.16.1 • 4 months ago

Changelog

TigerBeetle 0.16.1

Released: 2024-09-09

Safety And Performance

#2284

Improve view change efficiency; new heuristic for lagging replicas to forfeit view change.

A lagging replicas first gives a more up-to-date replica a chance to become primary by forfeiting view change. If the more up-to-date replica cannot step up as primary, the lagging replica attempts to step up as primary.
#2275, #2254, #2269, #2272

Complete rollout of the new state sync protocol.

Remove in-code remnants of the old state sync protocol. Replicas now panic if they receive messages belonging to the old protocol.

Internals

#2283

Improve log warnings for client eviction due to its version being too low/high.
#2288

Fix VOPR false positive wherein checkpoint was being updated twice in the upgrade path.
#2286

Fix typos found using codespell.
#2282

Document example for debiting multiple accounts and crediting a single account wherein the total amount to transfer to the credit account is known, but the balances of the individual debit accounts are not known.
#2281

Document the behavior of user_data_128/user_data_64/user_data_32 in the presence of pending transfers.
#2279

Inline Dockerfile in the release code, removing tools/docker/Dockerfile.
#2280, #2285

Add kcov code coverage to DevHub.
#2266, #2293

Add support for tracing IO & CPU events. This allows for coarse-grained performance analysis, for example collectively profiling IO and CPU performance (as opposed to IO or CPU in isolation).
#2273

Remove explicit header sector locks, using a common locking path for prepare and header sectors.
#2258

Change CliArgs -> CLIArgs in accordance with TigerStyle.
#2263

Vendor llvm-objcopy in the dependencies repository in accordance with our "no dependencies" policy. This ensures users don't have to manually install LLVM.
#2297

Assign correct date to the release binary date; it was earlier set to the epoch ("Jan 1 1970").
#2289

Introduce fatal errors for crashing the replica process in the face of uncorrectable errors (for example, insufficient memory/storage).
#2287

Add formatting check in the CI for the Go client.
#2278, #2292

Reduce dimensionality of configuration modes.

Removes the development configuration which was used to run the replica with asserts enabled, enabling asserts for the production configuration instead. Additionally, removes the -Dconfig CLI option, making production configuration the default.

TigerTracks 🎧

Fire on the Mountain

0.16.0

published 0.16.0 • 5 months ago

Changelog

TigerBeetle 0.16.0

Released: 2024-09-02

This release is 0.16.0 as it includes a new breaking API change around zero amount transfers, as well as the behavior around posting a full pending transfer amount or balancing as much as possible. These are all gated by the client's release version.

If you're running a client older than 0.16.0, you'll see the old behavior where zero amount transfers are disallowed, but on newer clients these are supported and will create a transfer with an amount of 0.

Additionally, the sentinel value to representing posting the full amount of a pending transfer, or doing a balancing transfer for as much as possible has changed. It's no longer 0, but instead AMOUNT_MAX.

See the tracking issue for more details.

Safety And Performance

#2221

Change how replicas that haven't finished syncing send a prepare_ok message, preventing them from falsely contributing to the durability of a checkpoint, which could potentially cause liveness issues in the event of storage faults.
#2255

The new state sync protocol regressed the behavior where the replica would try to repair the WAL before switching to state sync, and this puts the old behavior back in.

WAL repair is used when the lagging replica's log still intersects with the cluster's current log, while state sync is used when when the logs no longer intersect.
#2244

Try to repair (but not commit) prepares, even if we don't have all the headers between checkpoint and head.

This makes things consistent between the normal and repair paths, and improves concurrency while repairing.
#2253

Reject prepares on the primary if its view isn't durable, much like solo clusters.

This solves a failing VOPR seed wherein a primary accepting prepares before making its log_view durable exposes a break in its hash chain.
#2259, #2246

A few sysctls and security frameworks (eg, seccomp) might block io_uring. Print out a more helpful error message, rather than a generic "permission denied" or "system outdated".

Features

#2171

Add the new imported flag to allow user-defined timestamps when creating Accounts and Transfers from historical events.
#2220, #2237, #2238, #2239

Allow Transfers with amount=0 and change behavior for balancing and post-pending transfers, introducing the constant AMOUNT_MAX to replace the use of the zero sentinel when representing the maximum/original value in such cases. Note that this is a breaking change.

Also, explicitly define optional indexes, which previously were determined simply by not indexing zeroed values.
#2234

Introduce a new flag, Account.flags.closed, which causes an account to reject any further transfers, except for voiding two-phase transfers that are still pending.

The account flag can be set during creation or through a closing transfer. In the latter case, closed account can be re-opened by voiding or expiring the closing transfer.

Internals

#2211

Deprecates the old state sync protocol, no longer supporting both protocols simultaneously. As planned for this release, it only ignores old messages, allowing replicas to upgrade normally. In the next release, replicas would panic if they receive an old message.
#2233

Move multiversion build logic into build.zig from release.zig. This makes it much easier to build multiversion binaries as part of a regular zig build, without having to invoke CI or release process specific code that's normally part of release.zig.

It also makes it possible to build multiversion binaries on platforms that aren't x86_64 Linux.
#2215

Refactor the Multiversion API, bringing it in line with pre-existing code patterns.
#2251

Previously, TigerBeetle release numbers were based on a finicky conversion of GitHub's internal action run number to a version number.

This was error prone, and difficult to reason about before hand (what would the given version number for a release be?). Instead, make it so this very changelog is the source of truth for the version number which is explicitly set.
#2252

Change init function signatures to allow for in-place initialization. This addresses the silent stack growth caused by intermediate copy/move allocations during the initialization of large objects.

Specifically, the Forest struct can grow indefinitely depending on the number of Grooves/IndexTrees needed to support the StateMachine's custom logic, causing TigerBeetle to crash during startup due to stack-overflow.
#2265

Don't cancel in-progress GitHub actions on the main branch. In particular, this ensures that the devhub records the benchmark measurements for every merge to main, even if those merges occur in quick succession.
#2218

Make the experimental feature aof (append-only file) a runtime flag instead of a build-time setting. This simplifies operations, allowing the use of the same standard release binary in environments that require aof.
#2228

Renames the LSM constant lsm_batch_multiple to lsm_compaction_ops, providing clearer meaning on how it relates to the pace at which LSM tree compaction is triggered.
#2240

Add support for indexing flags, namely the new imported flag.

TigerTracks 🎧

0.15.6

published 0.15.6 • 5 months ago

Changelog