
Research
wget to Wipeout: Malicious Go Modules Fetch Destructive Payload
Socket's research uncovers three dangerous Go modules that contain obfuscated disk-wiping malware, threatening complete data loss.
@koopjs/provider-elasticsearch
Advanced tools
This provider allow Koop to fetch and transform data from an Elasticsearch instance.
From command line in the folder containing the project run:
npm install @koopjs/provider-elasticsearch --save
In your Koop instance, register the provider like:
const esProvider = require('@koopjs/provider-elasticsearch')
koop.register(esProvider)
The latest version of this provider can be installed with the Koop CLI. See the Koop CLI docs for details on setting up a Koop instance and add providers with the CLI.
To suppress KoopJS warnings from the console output, run with an environment variable of KOOP_WARNINGS="suppress". In powershell, this will look like: $env:KOOP_WARNINGS="suppress" ; node main.js
Setting an environment variable of KOOP_ENV_KEYS
allows saving of usernames and passwords to connect to elastic search
without having to store them in the configuration file. The format for this variable is
esId,user,password||esId2,user2,password2
. The esId is the id given to the connection in the configuration file as
shown below in the Basic Config File Structure section.
npm start
By default, all services only have a single layer, as defined in the index configuration. It is possible to configure one or more sub-layers in the subLayers section of the configuration. Each of these sub layers can be a predefined sub layer or you can create your own.
The following is a sample configuration file showing most capabilities
{
"esConnections": {
"esNameForServiceURLs": {
"id": "esNameForServiceURLs",
"protocol": "http://",
"port": 9200,
"hosts": [
"localhost"
],
"shapeIndices": {
"states": {
"geometryField": "geometry",
"geometryType": "Polygon",
"joinField": "NAME"
}
},
"indices": {
"myService1": {
"index": "indexName",
"allowMultiPoint": false,
"caching": {
"enabled": true,
"seconds": 600
},
"geometryField": "geometry.coordinates",
"geometryType": "geo_point",
"returnFields": [
"lastUpdate",
"createdAt",
"name"
],
"dateFields": [
"lastUpdate",
"createdAt"
],
"idField": "OBJECTID",
"subLayers": [
{"name": "geohash"}
],
"maxResults": 1000,
"maxLayerInfoResults": 1
},
"tableService": {
"index": "indexNoShape",
"allowMultiPoint": false,
"isTable": true,
"returnFields": [
"state",
"county",
"date"
],
"dateFields": [
"date"
],
"maxResults": 1000
},
"joinService": {
"index": "indexToJoin",
"allowMultiPoint": false,
"returnFields": [
"date",
"country",
"state.name"
],
"dateFields": [
"date"
],
"subLayers": [],
"shapeIndex": {
"name": "states",
"joinField": "state.name"
},
"maxResults": 1000
},
"polyService": {
"index": "polygonIndex",
"allowMultiPoint": false,
"geometryField": "geometry",
"geometryType": "MultiPolygon",
"reversePolygons": true,
"returnFields": [
"properties.date",
"properties.count",
"properties.state_name"
],
"dateFields": [
"properties.date"
],
"subLayers": [],
"maxResults": 1000
}
}
}
}
}
shapeIndices
includes all indices that will be used for their shapes on other services. All fields are mandatory.indices
this object has a property for every service to be created. The name of the property will be the service
name.Index Properties
index
is the name of the ElasticSearch indexisTable
treat this service as a table, ignoring geometryallowMultiPoint
is only important for point services and allows more than one point per featurecaching
allows local in-memory caching. Only use this for services that have no more documents than your maxResults settinggeometryField
is the full location of the geometrygeometryType
can be geo_point, Point, MultiPoint, Polyline, MultiLineString, Polygon and MultiPolygonreversePolygons
if the stored polygons do not follow the right-hand rule setting this to true will fix this.
Polygons that do not follow the right-hand rule will not be displayed as feature services without setting this to true.returnFields
includes all fields that will be returned by the feature servicedateFields
includes any return fields that should be treated as datesidField
if the index includes a field that can be treated as the OBJECTID field, this should be setsubLayers
any sub-layers to be used. Provided sub layers are geohash_aggregation
and geotile_aggregation
but custom subLayers may be used here as well.maxResults
the maximum features returned from a single requestmaxLayerInfoResults
maximum number of features to return for a layer info request (Normally 1) NOTE: Do not use this on a layer with caching enabled.shapeIndex
if the service will join to a shapeIndex for geometry list the name of the index (defined in
shapeIndices
) and the joinField from this index.vectorLayerID
the 0 based index of the subLayer to use for vector outputvectorStyle
an optional map-box style to use for the vector layerAll subLayer configurations must include a name
to map to a registered subLayer class. Other than that an options
object contains
any other needed information.
offset
which is used by both provided subLayers is the offset value passed in by esri clients. This value is in meters and is larger for higher zoom levels.aggregationFields
can be used to pass in sub-aggregations to elastic search. The fields will be added to the output for display in pop-ups and/or for use in symbology.GeoHash
precision is between 1-12 for geohash.
{
"name": "geohash_aggregation",
"options": {
"tileConfig": [
{ "precision": 8, "offset": 16 },
{ "precision": 6, "offset": 10000 },
{ "precision": 4, "offset": 32000 },
{ "precision": 2, "offset": 640000 }
],
"aggregationFields": {
"speed_avg": {
"avg": { "field": "speed"}
},
"speed_min": {
"min": { "field": "speed"}
}
}
}
}
GeoTile
precision is between 0-29 for geotile.
{
"name": "geotile_aggregation",
"options": {
"tileConfig": [
{ "precision": 22, "offset": 16 },
{ "precision": 16, "offset": 10000 },
{ "precision": 10, "offset": 32000 },
{ "precision": 4, "offset": 640000 }
],
"aggregationFields": {
"speed_avg": {
"avg": { "field": "speed"}
},
"speed_min": {
"min": { "field": "speed"}
}
}
}
}
Support for connections to Amazon OpenSearch clusters has been added. This functionality is new and had not been thoroughly tested. Please log any issues encountered when using/testing.
The OpenSearch configurations follow the same pattern as elasticsearch connections and can even be included in the same
configuration file. Create an osConnections
object sibling to the esConnections
object within the configuration
file. From there the configuration is the same as with elasticsearch connections except for cluster connection
information. All OpenSearch connection information is stored within a connectConfig
object. An example is given below.
{
"osConnections": {
"osCluster1": {
"id": "osCluster1",
"connectConfig": {
"node": "http://localhost:9200"
},
"indices": {
// same as with elastic configuration
}
}
}
}
mapReturnValues
is an object that can contain keys that are field names that in turn have their own keys equal to field
values mapped to object values. Example below. __defaultmapping
is not required.
{
"mapReturnValues": {
"fieldName": {
"returnedValue1": "mappedValue",
"__defaultmapping": "defaultValue"
}
}
}
IN DEVELOPMENT
mapFieldNames
can be used to specify a different return field than what is specified within returnFields
{
"mapFieldNames": {
"fieldName": "mappedFieldName"
}
}
[3.7.1] - 06-11-2024
FAQs
Elastic Search Provider for Koop
The npm package @koopjs/provider-elasticsearch receives a total of 5 weekly downloads. As such, @koopjs/provider-elasticsearch popularity was classified as not popular.
We found that @koopjs/provider-elasticsearch demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 7 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Socket's research uncovers three dangerous Go modules that contain obfuscated disk-wiping malware, threatening complete data loss.
Research
Socket uncovers malicious packages on PyPI using Gmail's SMTP protocol for command and control (C2) to exfiltrate data and execute commands.
Product
We redesigned Socket's first logged-in page to display rich and insightful visualizations about your repositories protected against supply chain threats.