Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
@aws/s3-managed-downloader
Advanced tools
Readme
The JavaScript SDK Managed Download can be used for custom S3 downloads. The client can configure options for the Managed Download such as chunk size and number of concurrent transfers.
This package only works in Node.js versions 6+ currently. Use the npm package manager for Node.js to install the Managed Download package. Type the following command in a terminal window.
npm install @aws/s3-managed-download
You can find the full documentation for the Managed Download package here.
You can customize the part size and number of concurrent downloads for the Managed Download by setting the maxPartSize and maxConcurrency options. The maxPartSize option controls the size in bytes of each part of the transfer. The maxConcurrency option specifies the number of parts that will be downloaded in parallel and the maximum number of parts that will be buffered in memory at any time.
You will need an S3 client in order to create a managed download, so import the aws-sdk along with the managed download package.
const S3 = require('aws-sdk/clients/s3');
const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
import * as S3 from 'aws-sdk/clients/s3';
import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
Create an AWS S3 client and pass the client into the Managed Download constructor to create a Managed Download with a part size of 5MB and concurrency of 1.
const s3 = new S3();
const managedDownloader = new ManagedDownloader(s3);
const s3:S3 = new S3();
const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
Create an AWS S3 client and Managed Download options. Pass the client and the options into the Managed Download constructor to create a Managed Download with a custom part size of 10MB and concurrency of 5.
const s3 = new S3();
const options = {
maxPartSize: 10 * 1024 * 1024,
maxConcurrency: 5
};
const managedDownloader = new ManagedDownloader(s3, options);
const s3:S3 = new S3();
const options:ManagedDownloaderOptons = {
maxPartSize: 10 * 1024 * 1024,
maxConcurrency: 5
};
const managedDownloader:ManagedDownloader = new ManagedDownloader(s3, options);
Currently, the Managed Download only contains a streaming download operation which can work for an entire file or a specific range or part of a file. Here are some examples on how to use the Managed Download's getObjectStream download operation.
Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to create a local file at 'example-file-path'.
const S3 = require('aws-sdk/clients/s3');
const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
const fs = require('fs');
const s3 = new S3();
const managedDownloader = new ManagedDownloader(s3);
const params = {
Bucket: 'example-bucket',
Key: 'example-key'
};
// create a write stream for a file
const writeStream = fs.createWriteStream('example-file-path');
managedDownloader.getObjectStream(params)
.then((stream) => {
stream.pipe(writeStream);
}, (err) => {
writeStream.end();
console.error(err);
});
import * as S3 from 'aws-sdk/clients/s3';
import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
import * as fs from 'fs';
const s3:S3 = new S3();
const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
const params:GetObjectStreamInput = {
Bucket: 'example-bucket',
Key: 'example-key'
};
// create a write stream for a file
const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path');
managedDownloader.getObjectStream(params)
.then((stream) => {
stream.pipe(writeStream);
}, (err) => {
writeStream.end();
console.error(err);
});
Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to write to bytes 100-150 of a local file at 'example-file-path'.
const S3 = require('aws-sdk/clients/s3');
const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
const fs = require('fs');
const s3 = new S3();
const managedDownloader = new ManagedDownloader(s3);
const params = {
Bucket: 'example-bucket',
Key: 'example-key',
Range: '100-150'
};
// create a write stream for a file starting at byte 100
const writeStream = fs.createWriteStream('example-file-path', {start:100});
managedDownloader.getObjectStream(params)
.then((stream) => {
stream.pipe(writeStream);
}, (err) => {
writeStream.end();
console.error(err);
});
import * as S3 from 'aws-sdk/clients/s3';
import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
import * as fs from 'fs';
const s3:S3 = new S3();
const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
const params:GetObjectStreamInput = {
Bucket: 'example-bucket',
Key: 'example-key',
Range: '100-150'
};
// create a write stream for a file starting at byte 100
const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path', {start:100});
managedDownloader.getObjectStream(params)
.then((stream) => {
stream.pipe(writeStream);
}, (err) => {
writeStream.end();
console.error(err);
});
If you find any bugs or want to request a new feature, please first check the existing issues. If there is not already an issue, then open a new issue. If the issue is regarding a bug, please include the stack trace and a method to recreate it.
This library is licensed under the Apache 2.0 License.
FAQs
Managed Downloader for S3
The npm package @aws/s3-managed-downloader receives a total of 135 weekly downloads. As such, @aws/s3-managed-downloader popularity was classified as not popular.
We found that @aws/s3-managed-downloader demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 13 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.