gdal-async
Advanced tools
Bindings to GDAL (Geospatial Data Abstraction Library) with full async support
Read and write raster and vector geospatial datasets straight from Node.js with this native asynchronous GDAL binding. To get started, browse the API Documentation or examples.
When in doubt on how to use a method, check also the unit tests.
This project is a fork of https://github.com/contra/node-gdal-next which is a fork of https://github.com/naturalatlas/node-gdal.
It adds a number of features:
The default install is currently the 3.3 branch which is bundled with GDAL 3.3.2.
Support for worker_threads is planned but it is not a priority project
Pre-built binaries are provided for most recent Linux distributions, Windows 64 bit and OS X 10.15:
npm install gdal-async
By default all dependencies are the latest versions and bundled out of the box, but if you would like to link against a custom GDAL library you will have to rebuild it when installing using the following flags:
# --shared_gdal allows linking to the OS-provided libgdal, requires libgdal-dev
# (debian: sudo apt-get install libgdal-dev)
$ npm install gdal-async --build-from-source --shared_gdal
If you have multiple versions of GDAL or your library is not in its standard location, you can use the usual CXXFLAGS and LDFLAGS variables:
$ CXXFLAGS="-I/some/path/include" LDFLAGS="-L/some/path/lib" npm install gdal-async --build-from-source --shared_gdal
const gdal = require("gdal-async")
const dataset = gdal.open("sample.tif")
console.log("number of bands: " + dataset.bands.count())
console.log("width: " + dataset.rasterSize.x)
console.log("height: " + dataset.rasterSize.y)
console.log("geotransform: " + dataset.geoTransform)
console.log("srs: " + (dataset.srs ? dataset.srs.toWKT() : 'null'))
const gdal = require("gdal-async")
const dataset = gdal.open("sample.shp")
const layer = dataset.layers.get(0)
console.log("number of features: " + layer.features.count())
console.log("fields: " + layer.fields.getNames())
console.log("extent: " + JSON.stringify(layer.extent))
console.log("srs: " + (layer.srs ? layer.srs.toWKT() : 'null'))
Mixing of synchronous and asynchronous operations is supported.
Simultaneous operations on distinct dataset objects are always safe and can run it parallel.
Simultaneous operations on the same dataset object are safe too but they won't run in parallel. This is a limitation of GDAL. The only way to have multiple parallel operations on the same file is to use multiple dataset objects. Keep in mind that Node.js/libuv won't be able to detect which async contexts are waiting on each other, so if you launch 16 simultaneous operations on 4 different datasets, there is always a chance that libuv will pick 4 operations on the same dataset to run - which will take all 4 slots on the thread pool. It is recommended to either increase UV_THREADPOOL_SIZE or to make sure that every dataset has exactly one operation running at any given time. Take a look at ASYNCIO.md which explains this in detail.
Also be particularly careful when mixing synchronous and asynchronous operations in server code. If a GDAL operation is running in the background for any given Dataset, all synchronous operations on that same Dataset on the main thread will block the event loop until the background operation is finished. This includes synchronous getters and setters that might otherwise be instantaneous.. It is recommended to retrieve all values such as raster size or no data value or spatial reference before starting any I/O operations or use the new asynchronous getters introduced in 3.3.2.
Does not support worker_threads yet
The HDF5 driver is not thread safe on Windows
If the last argument of an xxxAsync function is a callback,
it will be called on completion with standard (e,r) semantics
In this case the function will return a resolved Promise
const gdal = require('gdal-async');
gdal.openAsync('sample.tif', (e, dataset) => {
if (e) {
console.error(e);
return;
}
dataset.bands.get(1).pixels.readAsync(0, 0, dataset.rasterSize.x,
dataset.rasterSize.y, (e, data) => {
if (e) {
console.error(e);
return;
}
console.log(data);
});
});
If there is no callback, the function will return a Promise
const gdal = require('gdal-async')
gdal.openAsync('sample.tif')
.then((dataset) => dataset.bands.getAsync(1))
.then((band) => band.pixels.readAsync(0, 0, band.ds.rasterSize.x, band.ds.rasterSize.y))
.then((data) => console.log(data))
.catch((e) => console.error(e))
const gdal = require('gdal-async');
try {
const ds = await gdal.openAsync('sample.tif');
const band = await ds.bands.getAsync(1);
const rasterSize = await ds.rasterSizeAsync; // starting with 3.3.2
const data = await band.pixels.readAsync(0, 0, rasterSize.x, rasterSize.y);
} catch (e => console.error(e));
TypeScript support is available beginning with gdal-async@3.1.0
import * as gdal from 'gdal-async'
const ds1: gdal.Dataset = gdal.open('sample.tif')
const ds2: Promise<gdal.Dataset> = gdal.openAsync('sample.tif')
In TypeScript gdal is a collection of named exports - as this is the preferred way of importing a native module in TypeScript and it does not impose the use of the esModuleInterop flag.
When importing from a so-called Michael Jackson Script environment (Node.js native ES6 modules), the correct import syntax is:
import gdal from 'gdal-async'
In MJS gdal is a default export - as this allows for maximum compatibility with existing CJS applications.
Built-in networking uses an embedded version of libcurl. It supports zlib compression through Node.js' own zlib support. It does not support brotli or zstd. Node.js includes brotli, but as of Node.js 16 it still does not export these symbols for use by add-ons (yes, go bug them - ask them for c-ares too).
SSL on Linux uses OpenSSL through Node.js' own support. It uses the curl trusted root CA store by default and another store can be provided through the CURL_CA_BUNDLE environment variable or GDAL config option. SSL on Windows and OSX uses the OS-provided mechanisms - Schannel and SecureTransport respectively - and thus the trusted root CA store will be the one provided by the OS.
If you are coming from node-gdal-next
gdal.Geometry.fromGeoJson now throws an Error on error instead of simply returning a null geometrygdal.DatasetBands.get{Async}() now throws an Error if an invalid band is requested instead of returning a null objectgdal.DatasetLayers.get{Async}() now throws an Error if an invalid layer is requested instead of returning a null objectgdal.FeatureDefnFields.get{Async}() now throws an Error if an invalid field definition is requested instead of returning a null objectgdal.FeatureFields.get{Async}() now throws an Error if an invalid feature is requested instead of returning a null object
null if the field exists in the definition but it is not set in the featuregdal.GDALDrivers.get() now throws an Error if an invalid driver is requested instead of returning a null objectgdal.GeometryCollectionChildren.get() now throws an Error if an invalid sub-geometry is requested instead of returning a null objectgdal.LayerFeatures.get{Async}() now throws an Error if an invalid feature is requested instead of returning a null objectgdal.LayerFields.get{Async}() now throws an Error if an invalid field is requested instead of returning a null objectgdal.LineStringPoints.get() now throws an Error if an invalid point is requested instead of returning a null objectgdal.PolygonRings.get() now throws an Error if an invalid ring is requested instead of returning a null objectgdal.Geometry methods that return a gdal.Geometry throw an Error on error instead of returning a null objectIf you are coming from node-gdal, in addition to all of the above
gdal.SpatialReference.fromEPSG(4326) with gdal.SpatialReference.fromProj4('+init=epsg:4326')ndarray from scijsThe 3.2 branch of gdal-async is compatible with ndarray from scijs, but the array must be in a positive/positive row-major stride.
A separate plugin ndarray-gdal allows zero-copy I/O, with GDAL-backed interleaving in C++ using SIMD instructions, for all possible strides both for 2D raster data and N-dimensional MDArray data. The plugin requires gdal-async@3.3 and it is not compatible with the gdal-async@3.2 branch.
When using the bundled GDAL version, the following drivers will be available:
AAIGrid, ACE2, ADRG, AIG, AVCBin, AVCE00, AirSAR, BLX, BMP, BT, carto, CEOS, COASP, COSAR, CPG, CSV, CTG, CTable2, DGN, DIMAP, DIPEx, DOQ1, DOQ2, DTED, DXF, ECRGTOC, EDIGEO, EHdr, EIR, ELAS, ENVI, ERS, ESAT, ESRI Shapefile, MapInfo File, MBTiles, FAST, FIT, FlatGeobuf, FujiBAS, GFF, GML, GPSBabel, GPSTrackMaker, GPX, GRASSASCIIGrid, GRIB, GS7BG, GSAG, GSBG, GSC, GTX, GTiff, GenBin, GeoJSON, GeoRSS, Geoconcept, GPKG, HDF5, HF2, HFA, IDA, ILWIS, INGR, IRIS, ISIS2, ISIS3, Idrisi, JAXAPALSAR, JDEM, JPEG, KMLSUPEROVERLAY, KML, KRO, L1B, LAN, LCP, LOSLAS, Leveller, MAP, MEM, Memory, MFF2, MFF, MITAB, MVT, NDF, NetCDF, NGSGEOID, NITF, NTv2, NWT_GRC, NWT_GRD, OGR_GMT, OGR_PDS, OGR_SDTS, OGR_VRT, OSM, OpenFileGDB, PAux, PCIDSK, PDS, PGDUMP, PNG, PNM, REC, RMF, ROI_PAC, RPFTOC, RS2, RST, R, S57, SAGA, SAR_CEOS, SDTS, SGI, SNODAS, SQLite, SRP, SRTMHGT, SVG, SXF, TIL, TSX, Terragen, UK .NTF, USGSDEM, VICAR, VRT, vsiaz, vsicurl, vsigs, vsigzip, vsimem, vsioss, vsis3, WAsP, WCS, WMS, WMTS, XPM, XYZ, ZMap
When compiling against a system-installed shared GDAL, all drivers and projections supported by it, should also be supported by gdal-async.
This binding was originally the product of a collaboration between Natural Atlas and Mapbox. Its contributors are Brandon Reavis, Brian Reavis, Dane Springmeyer, Zac McCormick, and others.
node-gdal-next is maintained by @contra.
The async bindings, the curve geometries, the TypeScript support, the built-in networking and the multidimensional raster model are by @mmomtchev who is the current maintainer.
Before submitting pull requests, please update the tests and make sure they all pass.
$ npm test # test against bundled gdal
$ npm run test:shared # test against most major versions
# test against shared gdal on given Linux version and Node.js version
$ npm run container dev {ubuntu|centos|fedora|debian|archlinux}:{version} 12|14|16|lts shared
Copyright © 2015–2017 Natural Atlas, Inc. & Contributors
Copyright © 2020-2021 Momtchil Momtchev, @mmomtchev & Contributors
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Release binaries with pre-built bundled GDAL are tested against the full matrix of:
Development versions are unit tested for the following targets:
| Node | OS | GDAL |
|---|---|---|
| Node.js 14.x | CentOS 8 | system installed GDAL 3.0.4 |
| Node.js 14.x | CentOS 8 | bundled GDAL 3.2.3 |
| Node.js 14.x | Debian 10 buster | system installed GDAL 2.1.2 (earliest unit-tested GDAL) |
| Node.js 14.x | Debian 10 buster | bundled GDAL 3.2.3 |
| Node.js 14.x | Fedora 33 | system installed GDAL 3.1.4 |
| Node.js 14.x | Fedora 33 | bundled GDAL 3.2.3 |
| Node.js 16.x | Arch Linux current | system installed GDAL 3.2.3 |
| Node.js 16.x | Arch Linux current | bundled GDAL 3.2.3 |
| Node.js 14.x | Ubuntu 18.04 | system installed GDAL 2.2.3 |
| Node.js 14.x | Ubuntu 18.04 | bundled GDAL 3.2.3 (glibc target platform) |
| Node.js 12.x | Ubuntu 20.04 | system installed GDAL 3.0.4 |
| Node.js 14.x | Ubuntu 20.04 | system installed GDAL 3.0.4 |
| Node.js 16.x | Ubuntu 20.04 | system installed GDAL 3.0.4 |
| Node.js 12.x | Ubuntu 20.04 | bundled GDAL 3.2.3 |
| Node.js 14.x | Ubuntu 20.04 | bundled GDAL 3.2.3 (code coverage platform) |
| Node.js 16.x | Ubuntu 20.04 | bundled GDAL 3.2.3 |
| Node.js 12.x | Windows Server 2019 | bundled GDAL 3.2.3 |
| Node.js 14.x | Windows Server 2019 | bundled GDAL 3.2.3 |
| Node.js 16.x | Windows Server 2019 | bundled GDAL 3.2.3 |
| Node.js 12.x | macOS Catalina 10.15 | bundled GDAL 3.2.3 |
| Node.js 14.x | macOS Catalina 10.15 | bundled GDAL 3.2.3 |
| Node.js 16.x | macOS Catalina 10.15 | bundled GDAL 3.2.3 |
[3.3.3] 2021-09-07
rollup-plugin-nativesgdal.vsimem.release is now always safe to useFAQs
Bindings to GDAL (Geospatial Data Abstraction Library) with full async support
The npm package gdal-async receives a total of 2,807 weekly downloads. As such, gdal-async popularity was classified as popular.
We found that gdal-async demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
TC39 met in Seattle and advanced 9 JavaScript proposals, including three to Stage 4, introducing new features and enhancements for a future ECMAScript release.
Security News
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
Security News
React's CRA deprecation announcement sparked community criticism over framework recommendations, leading to quick updates acknowledging build tools like Vite as valid alternatives.