Research
Recent Trends in Malicious Packages Targeting Discord
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
fd-slicer
Advanced tools
Package description
The fd-slicer npm package provides tools for creating slicer streams that allow you to read slices of a file without having to load the entire file into memory. This is particularly useful for handling large files or for applications that need to process or transmit parts of files efficiently.
Creating a slice stream from a file descriptor
This code demonstrates how to open a file, create a slicer from its file descriptor, and then create a readable stream that reads a specific part of the file (from byte 100 to 200).
const fs = require('fs');
const fdSlicer = require('fd-slicer');
fs.open('path/to/file', 'r', (err, fd) => {
if (err) throw err;
var slicer = fdSlicer.createFromFd(fd);
var stream = slicer.createReadStream({start: 100, end: 200});
stream.on('data', (chunk) => {
console.log('Data chunk:', chunk);
});
stream.on('end', () => {
console.log('Stream ended');
});
});
fs-cap is a package that provides capabilities similar to fd-slicer, allowing for the creation of capped file streams. It differs in that it focuses more on limiting the size of the data that can be read or written, rather than slicing per se.
stream-slicer offers functionality to slice streams of data, which is similar to fd-slicer's ability to slice file streams. However, stream-slicer is more generic and can be used with any type of stream, not just file descriptors.
Changelog
0.3.0
Readme
Safe fs.ReadStream
and fs.WriteStream
using the same fd.
Let's say that you want to perform a parallel upload of a file to a remote
server. To do this, we want to create multiple read streams. The first thing
you might think of is to use the {start: 0, end: 0}
API of
fs.createReadStream
. This gives you two choices:
fs.ReadStream
objects.Neither of these are acceptable options. The first one is a severe bug,
because the API docs for fs.write
state:
Note that it is unsafe to use
fs.write
multiple times on the same file without waiting for the callback. For this scenario,fs.createWriteStream
is strongly recommended.
fs.createWriteStream
will solve the problem if you only create one of them
for the file descriptor, but it will exhibit this unsafety if you create
multiple write streams per file descriptor.
The second option suffers from a race condition. For each additional time the file is opened after the first, it is possible that the file is modified. So in our parallel uploading example, we might upload a corrupt file that never existed on the client's computer.
This module solves this problem by providing createReadStream
and
createWriteStream
that operate on a shared file descriptor and provides
the convenient stream API while still allowing slicing and dicing.
This module also gives you some additional power that the builtin
fs.createWriteStream
do not give you. These features are:
var FdSlicer = require('fd-slicer');
var fs = require('fs');
fs.open("file.txt", 'r', function(err, fd) {
if (err) throw err;
var fdSlicer = new FdSlicer(fd);
var firstPart = fdSlicer.createReadStream({start: 0, end: 100});
var secondPart = fdSlicer.createReadStream({start: 100});
var firstOut = fs.createWriteStream("first.txt");
var secondOut = fs.createWriteStream("second.txt");
firstPart.pipe(firstOut);
secondPart.pipe(secondOut);
});
var FdSlicer = require('fd-slicer');
fs.open("file.txt", 'r', function(err, fd) {
if (err) throw err;
var fdSlicer = new FdSlicer(fd);
// ...
});
Make sure fd
is a properly initialized file descriptor. If you want to
use createReadStream
make sure you open it for reading and if you want
to use createWriteStream
make sure you open it for writing.
options
is an optional object which can contain:
autoClose
- if set to true
, the file descriptor will be automatically
closed once the last stream that references it is closed. Defaults to
false
. ref()
and unref()
can be used to increase or decrease the
reference count, respectively.The file descriptor passed in.
Available options
:
start
- Number. The offset into the file to start reading from. Defaults
to 0.end
- Number. Exclusive upper bound offset into the file to stop reading
from.highWaterMark
- Number. The maximum number of bytes to store in the
internal buffer before ceasing to read from the underlying resource.
Defaults to 16 KB.encoding
- String. If specified, then buffers will be decoded to strings
using the specified encoding. Defaults to null
.The ReadableStream that this returns has these additional methods:
destroy()
- stop streamingAvailable options
:
start
- Number. The offset into the file to start writing to. Defaults to
0.end
- Number. Exclusive upper bound offset into the file. If this offset
is reached, the write stream will emit an 'error' event and stop functioning.
In this situation, err.code === 'ETOOBIG'
. Defaults to Infinity
.highWaterMark
- Number. Buffer level when write()
starts returning
false. Defaults to 16KB.decodeStrings
- Boolean. Whether or not to decode strings into Buffers
before passing them to _write()
. Defaults to true
.The WritableStream that this returns has these additional methods:
destroy()
- stop streamingAnd these additional properties:
bytesWritten
- number of bytes written to the streamAnd these additional events:
bytesWritten
changes.Equivalent to fs.read
, but with concurrency protection.
callback
must be defined.
Equivalent to fs.write
, but with concurrency protection.
callback
must be defined.
Increase the autoClose
reference count by 1.
Decrease the autoClose
reference count by 1.
Emitted if fs.close
returns an error when auto closing.
Emitted when fd-slicer closes the file descriptor due to autoClose
.
FAQs
safely create multiple ReadStream or WriteStream objects from the same file descriptor
We found that fd-slicer demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
Security News
Socket CEO Feross Aboukhadijeh joins a16z partners to discuss how modern, sophisticated supply chain attacks require AI-driven defenses and explore the challenges and solutions in leveraging AI for threat detection early in the development life cycle.
Security News
NIST's new AI Risk Management Framework aims to enhance the security and reliability of generative AI systems and address the unique challenges of malicious AI exploits.