
Security News
/Research
Wallet-Draining npm Package Impersonates Nodemailer to Hijack Crypto Transactions
Malicious npm package impersonates Nodemailer and drains wallets by hijacking crypto transactions across multiple blockchains.
hyparquet-compressors
Advanced tools
This package provides decompressors for various compression codecs. It is designed to be used with hyparquet in order to provide full support for all parquet compression formats.
Apache Parquet is a popular columnar storage format that is widely used in data engineering, data science, and machine learning applications for efficiently storing and processing large datasets. It supports a number of different compression formats, but most parquet files use snappy compression.
Hyparquet is a fast and lightweight parquet reader that is designed to work in both node.js and the browser.
By default, hyparquet only supports uncompressed
and snappy
compressed files (the most common parquet compression codecs). The hyparquet-compressors
package extends support for all legal parquet compression formats.
hyparquet-compressors
works in both node.js and the browser. Uses js and wasm packages, no system dependencies.
To use hyparquet-compressors
with hyparquet
, simply pass the compressors
object to the parquetReadObjects
function.
import { parquetReadObjects } from 'hyparquet'
import { compressors } from 'hyparquet-compressors'
const data = await parquetReadObjects({ file, compressors })
See hyparquet repo for more info.
Parquet compression types supported with hyparquet-compressors
:
Snappy compression uses hysnappy for fast snappy decompression using a minimal WASM module.
We load the wasm module synchronously from base64 in the js file. This avoids a network request, and greatly simplifies bundling and serving wasm.
New gzip implementation adapted from fflate. Includes modifications to handle repeated back-to-back gzip streams that sometimes occur in parquet files (but are not supported by fflate).
For gzip, the output
buffer argument is optional:
output
is defined, the decompressor will write to output
until it is full.output
is undefined, the decompressor will allocate a new buffer, and expand it as needed to fit the uncompressed gzip data. Importantly, the caller should use the returned buffer.Includes a minimal port of brotli.js. Our implementation uses gzip to pre-compress the brotli dictionary, in order to minimize the bundle size.
New LZ4 implementation includes support for legacy hadoop LZ4 frame format used on some old parquet files.
Uses fzstd for Zstandard decompression.
File | Size |
---|---|
hyparquet-compressors.min.js | 116.4kb |
hyparquet-compressors.min.js.gz | 75.2kb |
FAQs
Decompressors for hyparquet
The npm package hyparquet-compressors receives a total of 5,540 weekly downloads. As such, hyparquet-compressors popularity was classified as popular.
We found that hyparquet-compressors demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
/Research
Malicious npm package impersonates Nodemailer and drains wallets by hijacking crypto transactions across multiple blockchains.
Security News
This episode explores the hard problem of reachability analysis, from static analysis limits to handling dynamic languages and massive dependency trees.
Security News
/Research
Malicious Nx npm versions stole secrets and wallet info using AI CLI tools; Socket’s AI scanner detected the supply chain attack and flagged the malware.