Security News
JavaScript Leaders Demand Oracle Release the JavaScript Trademark
In an open letter, JavaScript community leaders urge Oracle to give up the JavaScript trademark, arguing that it has been effectively abandoned through nonuse.
pelias-address-deduplicator
Advanced tools
A stream for deduplicating a stream of address objects.
A stream that performs address deduplication using the robust OpenVenues deduplicator; note that it must be separately installed and running.
pelias-address-deduplicator
exports a single function:
createDeduplicateStream( requestBatchSize, maxLiveRequests, serverUrl )
, which accepts three optional arguments:
requestBatchSize
(default: 10000
): The number of addresses to buffer into a
batch before sending it to the deduplicator. The higher the number, the
less time and energy collectively spent in making requests, but the
bigger the memory consumption buildup.maxLiveRequests
(default: 4
): Since the deduper is implemented as a
standalone server and processes data more slowly than the importer feeds
it, the stream needs to rate-limit itself. maxLiveRequests
indicates
the maximum number of unresolved concurrent requests at any time; when
that number is hit, the stream will pause reading until the number of
concurrent requests falls below it.serverUrl
(default: 'http://localhost:5000'
): The HTTP base URL of the address deduplicator server.and returns a Transform
stream, which accepts un-deduplicated addresses and filters out the duplicates; note that
it'll likely be the slowest part of your data pipeline because of all the involved heavy lifting. The addresses
themselves are expected to be pelias/model Document
objects.
FAQs
A stream for deduplicating a stream of address objects.
The npm package pelias-address-deduplicator receives a total of 1 weekly downloads. As such, pelias-address-deduplicator popularity was classified as not popular.
We found that pelias-address-deduplicator demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In an open letter, JavaScript community leaders urge Oracle to give up the JavaScript trademark, arguing that it has been effectively abandoned through nonuse.
Security News
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Security News
Floating dependency ranges in npm can introduce instability and security risks into your project by allowing unverified or incompatible versions to be installed automatically, leading to unpredictable behavior and potential conflicts.