Research
Recent Trends in Malicious Packages Targeting Discord
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
vinyl-elasticsearch
Advanced tools
Readme
Read and write Vinyl documents from and to ElasticSearch.
var gulp = require('gulp');
var elasticsearch = require('vinyl-elasticsearch');
gulp.task('search', function() {
return elasticsearch.src(
{
index: 'someindex',
size: 0,
body: {
// Begin query.
query: {
// Boolean query for matching and excluding items.
bool: {
must: [{match: {'somefield': 'somevalue'}}]
}
},
// Aggregate on the results
aggs: {
actions: {
terms: {
field: 'additionalType',
order: {'_term' : 'asc'},
size: 10,
}
}
}
}
},
{
host: 'https://localhost:9200',
log: 'trace'
}
)
.pipe(echo());
});
var gulp = require('gulp');
var elasticsearch = require('vinyl-elasticsearch');
gulp.task('default', function (){
return gulp.src('hello.json')
.pipe(
elasticsearch.dest(
{
index: 'someindex',
type: 'somedoc'
},
{
host: 'https://localhost:9200',
log: 'trace'
}
)
);
});
The configuration for the connection to ES comes from the opt
parameter, which is used when creating the client. Possible options are described at ElasticSearch Configuration.
In addtion to the ElasticSearch options, a few more have been added to help manage templates, use the AWS version of ElasticSearch, and to configure retries.
If opt.manageTemplate
is true
, then templates will be maintained automatically on the ElasticSearch host. This involves checking for one or more templates with the names indicated in templateName
, and if they don't exist (or the templateOverwrite
value is set to true
) then the appropriate template from the templateDir
is uploaded. If any of these parameters are missing then errors will be propagated back to the caller.
For example:
{
templateName: 'logstash,listenaction',
templateOverwrite: true,
templateDir: path.join(__dirname, '../fixtures/templates')
}
In addition, if opt.amazonES
is present then each message is signed. This makes it possible to use Amazon's ElasticSearch Service. Possible values for amazonES
are:
amazonES: {
region: 'someregion',
accessKey: 'accesskeyid',
secretKey: 'secretaccesskey'
}
Some ElasticSearch actions may fail for reasons that are temporary, such as servers being busy with nightly backups, or network failures. To help deal with this opts.retries
can be set to indicate the number of retries, with the default being zero.
The retry strategy uses the retry module and the backoff strategy is described there.
The configuration for the connection to ES comes from the opt
parameter, as described above.
The index, document type and id are usually derived from the file
object being passed through, using:
file.index
value;file.type
value, of if none is present, the file.base
value;file.id
value, or if none is present the file.path
value.For any of these values that is not present the corresponding value from opt
will be used.
However, to make it possible to override values across the board, any settings in glob
take priority.
The body of the document will be set to the data
property of the Vinyl file if it's present. Otherwise the buffer in the contents
property will be converted to a string and then parsed as JSON.
As of v1.7.0, writing to ES uses the bulk update API. The sequence is that each new item to be indexed is collected in memory and then the whole batch is pushed to ES in one go. The trigger to cause the push is either that the batch size has been reached (which defaults to 5000, but can be set with opt.batchSize
) or that the batch timeout has elapsed (which defaults to 5000 millisecond, but can be set with opt.batchTimeout
).
The configuration for the connection to ES comes from the opt
parameter, as described above.
The document returned will be the whole collection of search results, with the usual hits.hits
incantation. This will probably be modified soon to allow individual results to be returned as distinct Vinyl files.
FAQs
Presents ElasticSearch as a destination stream of Vinyl objects.
We found that vinyl-elasticsearch demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
Security News
Socket CEO Feross Aboukhadijeh joins a16z partners to discuss how modern, sophisticated supply chain attacks require AI-driven defenses and explore the challenges and solutions in leveraging AI for threat detection early in the development life cycle.
Security News
NIST's new AI Risk Management Framework aims to enhance the security and reliability of generative AI systems and address the unique challenges of malicious AI exploits.