Table of Contents
Penumbra
Encrypt/decrypt anything in the browser using streams on background threads.
Quickly and efficiently decrypt remote resources in the browser. Display the files in the DOM, or download them with conflux.
Compatibility
| .decrypt | .encrypt | .saveZip |
---|
Chrome | ✅ | ✅ | ✅ |
Edge >18 | ✅ | ✅ | ✅ |
Safari ≥14.1 | ✅ | ✅ | ✅ |
Firefox ≥102 | ✅ | ✅ | ✅ |
Safari <14.1 | 🐢 | 🐢 | 🟡 |
Firefox <102 | 🐢 | 🐢 | 🟡 |
Edge 18 | ❌ | ❌ | ❌ |
✅ = Full support with workers
🐢 = Uses main thread (lacks native WritableStream support)
🟡 = 32 MiB limit
❌ = No support
Usage
Importing Penumbra
With Yarn/NPM
yarn add @transcend-io/penumbra
npm install --save @transcend-io/penumbra
import { penumbra } from '@transcend-io/penumbra';
penumbra.get(...files).then(penumbra.save);
Vanilla JS
<script src="lib/main.penumbra.js"></script>
<script>
penumbra
.get(...files)
.then(penumbra.getTextOrURI)
.then(displayInDOM);
</script>
Check out this guide for asynchronous loading.
RemoteResource
penumbra.get()
uses RemoteResource descriptors to specify where to request resources and their various decryption parameters.
type RemoteResource = {
url: string;
mimetype?: string;
filePrefix?: string;
decryptionOptions?: PenumbraDecryptionInfo;
path?: string;
requestInit?: RequestInit;
lastModified?: Date;
size?: number;
};
.get
Fetch and decrypt remote files.
penumbra.get(...resources: RemoteResource[]): Promise<PenumbraFile[]>
.encrypt
Encrypt files.
penumbra.encrypt(options: PenumbraEncryptionOptions, ...files: PenumbraFile[]): Promise<PenumbraEncryptedFile[]>
type PenumbraEncryptionOptions = {
key: string | Buffer;
};
.encrypt() examples:
Encrypt an empty stream:
size = 4096 * 128;
addEventListener('penumbra-progress', (e) => console.log(e.type, e.detail));
addEventListener('penumbra-complete', (e) => console.log(e.type, e.detail));
file = penumbra.encrypt(null, {
stream: new Response(new Uint8Array(size)).body,
size,
});
data = [];
file.then(async ([encrypted]) => {
console.log('encryption complete');
data.push(new Uint8Array(await new Response(encrypted.stream).arrayBuffer()));
});
Encrypt and decrypt text:
const te = new self.TextEncoder();
const td = new self.TextDecoder();
const input = '[test string]';
const buffer = te.encode(input);
const { byteLength: size } = buffer;
const stream = new Response(buffer).body;
const options = null;
const file = {
stream,
size,
};
const [encrypted] = await penumbra.encrypt(options, file);
const decryptionInfo = await penumbra.getDecryptionInfo(encrypted);
const [decrypted] = await penumbra.decrypt(decryptionInfo, encrypted);
const decryptedData = await new Response(decrypted.stream).arrayBuffer();
const decryptedText = td.decode(decryptedData);
console.log('decrypted text:', decryptedText);
.getDecryptionInfo
Get decryption info for a file, including the iv, authTag, and key. This may only be called on files that have finished being encrypted.
penumbra.getDecryptionInfo(file: PenumbraFile): Promise<PenumbraDecryptionInfo>
.decrypt
Decrypt files.
penumbra.decrypt(options: PenumbraDecryptionInfo, ...files: PenumbraEncryptedFile[]): Promise<PenumbraFile[]>
const te = new TextEncoder();
const td = new TextDecoder();
const data = te.encode('test');
const { byteLength: size } = data;
const [encrypted] = await penumbra.encrypt(null, {
stream: data,
size,
});
const options = await penumbra.getDecryptionInfo(encrypted);
const [decrypted] = await penumbra.decrypt(options, encrypted);
const decryptedData = await new Response(decrypted.stream).arrayBuffer();
return td.decode(decryptedData) === 'test';
.save
Save files retrieved by Penumbra. Downloads a .zip if there are multiple files. Returns an AbortController that can be used to cancel an in-progress save stream.
penumbra.save(data: PenumbraFile[], fileName?: string): AbortController
.getBlob
Load files retrieved by Penumbra into memory as a Blob.
penumbra.getBlob(data: PenumbraFile[] | PenumbraFile | ReadableStream, type?: string): Promise<Blob>
.getTextOrURI
Get file text (if content is text) or URI (if content is not viewable).
penumbra.getTextOrURI(data: PenumbraFile[]): Promise<{ type: 'text'|'uri', data: string, mimetype: string }[]>
.saveZip
Save a zip containing files retrieved by Penumbra.
type ZipOptions = {
name?: string;
size?: number;
files?: PenumbraFile[];
controller?: AbortController;
allowDuplicates: boolean;
compressionLevel?: number;
saveBuffer?: boolean;
onProgress?(event: CustomEvent<ZipProgressDetails>): void;
onComplete?(event: CustomEvent<{}>): void;
};
penumbra.saveZip(options?: ZipOptions): PenumbraZipWriter;
interface PenumbraZipWriter extends EventTarget {
write(...files: PenumbraFile[]): Promise<number>;
close(): Promise<number>;
abort(): void;
getBuffer(): Promise<ArrayBuffer>;
getFiles(): string[];
getSize(): Promise<number>;
}
type ZipProgressDetails = {
percent: number | null;
written: number;
size: number | null;
};
Example:
const files = [
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/tortoise.jpg.enc',
name: 'tortoise.jpg',
mimetype: 'image/jpeg',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'ELry8dZ3djg8BRB+7TyXZA==',
},
},
];
const writer = penumbra.saveZip();
await writer.write(...(await penumbra.get(...files)));
await writer.close();
.setWorkerLocation
Configure the location of Penumbra's worker threads.
penumbra.setWorkerLocation(location: WorkerLocationOptions | string): Promise<void>
Examples
Display encrypted text
const decryptedText = await penumbra
.get({
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/NYT.txt.enc',
mimetype: 'text/plain',
filePrefix: 'NYT',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'gadZhS1QozjEmfmHLblzbg==',
},
})
.then((file) => penumbra.getTextOrURI(file)[0])
.then(({ data }) => {
document.getElementById('my-paragraph').innerText = data;
});
Display encrypted image
const imageSrc = await penumbra
.get({
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/tortoise.jpg.enc',
filePrefix: 'tortoise',
mimetype: 'image/jpeg',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'ELry8dZ3djg8BRB+7TyXZA==',
},
})
.then((file) => penumbra.getTextOrURI(file)[0])
.then(({ data }) => {
document.getElementById('my-img').src = data;
});
Download an encrypted file
penumbra
.get({
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/africa.topo.json.enc',
filePrefix: 'africa',
mimetype: 'image/jpeg',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'ELry8dZ3djg8BRB+7TyXZA==',
},
})
.then((file) => penumbra.save(file));
Download many encrypted files
penumbra
.get([
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/africa.topo.json.enc',
filePrefix: 'africa',
mimetype: 'image/jpeg',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'ELry8dZ3djg8BRB+7TyXZA==',
},
},
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/NYT.txt.enc',
mimetype: 'text/plain',
filePrefix: 'NYT',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'gadZhS1QozjEmfmHLblzbg==',
},
},
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/tortoise.jpg',
filePrefix: 'tortoise',
mimetype: 'image/jpeg',
},
])
.then((files) => penumbra.save({ data: files, fileName: 'example' }));
Advanced
Prepare connections for file downloads in advance
const resources = [
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/NYT.txt.enc',
filePrefix: 'NYT',
mimetype: 'text/plain',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'gadZhS1QozjEmfmHLblzbg==',
},
},
{
url: 'https://s3-us-west-2.amazonaws.com/bencmbrook/tortoise.jpg.enc',
filePrefix: 'tortoise',
mimetype: 'image/jpeg',
decryptionOptions: {
key: 'vScyqmJKqGl73mJkuwm/zPBQk0wct9eQ5wPE8laGcWM=',
iv: '6lNU+2vxJw6SFgse',
authTag: 'ELry8dZ3djg8BRB+7TyXZA==',
},
},
];
penumbra.preconnect(...resources);
penumbra.preload(...resources);
Encrypt/Decrypt Job Completion Event Emitter
You can listen to encrypt/decrypt job completion events through the penumbra-complete
event.
window.addEventListener(
'penumbra-complete',
({ detail: { id, decryptionInfo } }) => {
console.log(
`finished encryption job #${id}%. decryption options:`,
decryptionInfo,
);
},
);
Progress Event Emitter
You can listen to download and encrypt/decrypt job progress events through the penumbra-progress
event.
window.addEventListener(
'penumbra-progress',
({ detail: { percent, id, type } }) => {
console.log(`${type}% ${percent}% done for ${id}`);
},
);
Note: this feature requires the Content-Length
response header to be exposed. This works by adding Access-Control-Expose-Headers: Content-Length
to the response header (read more here and here)
On Amazon S3, this means adding the following line to your bucket policy, inside the <CORSRule>
block:
<ExposeHeader>Content-Length</ExposeHeader>
Configure worker location
penumbra.setWorkerLocation('/penumbra-workers/');
penumbra.setWorkerLocation({
base: '/penumbra-workers/',
penumbra: 'worker.penumbra.js',
StreamSaver: 'StreamSaver.js',
});
penumbra.setWorkerLocation({ penumbra: 'worker.penumbra.js' });
Waiting for the penumbra-ready
event
<script src="lib/main.penumbra.js" async defer></script>
const onReady = async ({ detail: { penumbra } } = { detail: self }) => {
await penumbra.get(...files).then(penumbra.save);
};
if (!self.penumbra) {
self.addEventListener('penumbra-ready', onReady);
} else {
onReady();
}
Querying Penumbra browser support
You can check if Penumbra is supported by the current browser by comparing penumbra.supported(): PenumbraSupportLevel
with penumbra.supported.levels
.
if (penumbra.supported() > penumbra.supported.levels.possible) {
}
enum PenumbraSupportLevel {
none = -0,
possible = 0,
size_limited = 1,
full = 2,
}
Webpack
Penumbra is compiled and bundled on npm. The recommended use is to copy in the penumbra build files into your webpack build.
We do this with copy-webpack-plugin
i.e.
const fs = require('fs');
const CopyPlugin = require('copy-webpack-plugin');
const path = require('path');
const PENUMBRA_DIRECTORY = path.join(
__dirname,
'node_modules',
'@transcend-io/penumbra',
'build',
);
module.exports = {
plugins: [
new CopyPlugin({
patterns: fs.readdirSync(PENUMBRA_DIRECTORY)
.filter((fil) => fil.indexOf('.') > 0)
.map((fil) => ({
from: `${PENUMBRA_DIRECTORY}/${fil}`,
to: `${outputPath}/${fil}`,
})),
}),
]
Contributing
yarn
yarn build
yarn test:local
yarn test:interactive
License