@apache-arrow/esnext-umd
Advanced tools
Arrow is a set of technologies that enable big data systems to process and transfer data quickly.
npm install apache-arrow
(read about how we package apache-arrow below)
Apache Arrow is a columnar memory layout specification for encoding vectors and table-like containers of flat and nested data. The Arrow spec aligns columnar data in memory to minimize cache misses and take advantage of the latest SIMD (Single input multiple data) and GPU operations on modern processors.
Apache Arrow is the emerging standard for large in-memory columnar data (Spark, Pandas, Drill, Graphistry, ...). By standardizing on a common binary interchange format, big data systems can reduce the costs and friction associated with cross-system communication.
import { readFileSync } from 'fs';
import { Table } from 'apache-arrow';
const arrow = readFileSync('simple.arrow');
const table = Table.from([arrow]);
console.log(table.toString());
/*
foo, bar, baz
1, 1, aa
null, null, null
3, null, null
4, 4, bbb
5, 5, cccc
*/
import { readFileSync } from 'fs';
import { Table } from 'apache-arrow';
const table = Table.from([
'latlong/schema.arrow',
'latlong/records.arrow'
].map((file) => readFileSync(file)));
console.log(table.toString());
/*
origin_lat, origin_lon
35.393089294433594, -97.6007308959961
35.393089294433594, -97.6007308959961
35.393089294433594, -97.6007308959961
29.533695220947266, -98.46977996826172
29.533695220947266, -98.46977996826172
*/
import { readFileSync } from 'fs';
import { Table } from 'apache-arrow';
const table = Table.from([
'latlong/schema.arrow',
'latlong/records.arrow'
].map(readFileSync));
const column = table.col('origin_lat');
const typed = column.slice();
assert(typed instanceof Float32Array);
for (let i = -1, n = column.length; ++i < n;) {
assert(column.get(i) === typed[i]);
}
import MapD from 'rxjs-mapd';
import { Table } from 'apache-arrow';
const port = 9091;
const host = `localhost`;
const db = `mapd`;
const user = `mapd`;
const password = `HyperInteractive`;
MapD.open(host, port)
.connect(db, user, password)
.flatMap((session) =>
// queryDF returns Arrow buffers
session.queryDF(`
SELECT origin_city
FROM flights
WHERE dest_city ILIKE 'dallas'
LIMIT 5`
).disconnect()
)
.map(([schema, records]) =>
// Create Arrow Table from results
Table.from(schema, records))
.map((table) =>
// Stringify the table to CSV with row numbers
table.toString({ index: true }))
.subscribe((csvStr) =>
console.log(csvStr));
/*
Index, origin_city
0, Oklahoma City
1, Oklahoma City
2, Oklahoma City
3, San Antonio
4, San Antonio
*/
See develop.md
Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved:
We prefer to receive contributions in the form of GitHub pull requests. Please send pull requests against the github.com/apache/arrow repository.
If you are looking for some ideas on what to contribute, check out the JIRA issues for the Apache Arrow project. Comment on the issue and/or contact dev@arrow.apache.org with your questions and ideas.
If you’d like to report a bug but don’t have time to fix it, you can still post it on JIRA, or email the mailing list dev@arrow.apache.org
apache-arrow is written in TypeScript, but the project is compiled to multiple JS versions and common module formats.
The base apache-arrow package includes all the compilation targets for convenience, but if you're conscientious about your node_modules footprint, we got you.
The targets are also published under the @apache-arrow namespace:
npm install apache-arrow # <-- combined es5/UMD, es2015/CommonJS/ESModules/UMD, and TypeScript package
npm install @apache-arrow/ts # standalone TypeScript package
npm install @apache-arrow/es5-cjs # standalone es5/CommonJS package
npm install @apache-arrow/es5-esm # standalone es5/ESModules package
npm install @apache-arrow/es5-umd # standalone es5/UMD package
npm install @apache-arrow/es2015-cjs # standalone es2015/CommonJS package
npm install @apache-arrow/es2015-esm # standalone es2015/ESModules package
npm install @apache-arrow/es2015-umd # standalone es2015/UMD package
npm install @apache-arrow/esnext-esm # standalone esNext/CommonJS package
npm install @apache-arrow/esnext-esm # standalone esNext/ESModules package
npm install @apache-arrow/esnext-umd # standalone esNext/UMD package
The JS community is a diverse group with a varied list of target environments and tool chains. Publishing multiple packages accommodates projects of all stripes.
If you think we missed a compilation target and it's a blocker for adoption, please open an issue.
Full list of broader Apache Arrow committers.
Full list of broader Apache Arrow projects & organizations.
Apache Arrow 0.3.0 (2017-05-05)
FAQs
Apache Arrow columnar in-memory format
The npm package @apache-arrow/esnext-umd receives a total of 15 weekly downloads. As such, @apache-arrow/esnext-umd popularity was classified as not popular.
We found that @apache-arrow/esnext-umd demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers have discovered multiple malicious npm packages targeting Solana private keys, abusing Gmail to exfiltrate the data and drain Solana wallets.
Security News
PEP 770 proposes adding SBOM support to Python packages to improve transparency and catch hidden non-Python dependencies that security tools often miss.
Security News
Socket CEO Feross Aboukhadijeh discusses open source security challenges, including zero-day attacks and supply chain risks, on the Cyber Security Council podcast.