Security News
Cloudflare Adds Security.txt Setup Wizard
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
@socialgouv/streaming-file-encryption
Advanced tools
Streaming file encryption for Node.js using AES-256-GCM, HKDF and HMAC.
@socialgouv/streaming-file-encryption
Streaming encryption/decryption for files in Node.js made simple and secure.
pnpm add @socialgouv/streaming-file-encryption
yarn add @socialgouv/streaming-file-encryption
npm i @socialgouv/streaming-file-encryption
Note: requires Node.js v15 or superior.
To get started, you will need a main secret (64 bytes hex-encoded):
import { generateSerializedMainSecret } from '@socialgouv/streaming-file-encryption'
const serializedMainSecret = generateSerializedMainSecret()
Or do so via the CLI:
npx @socialgouv/streaming-file-encryption generate
This will be passed to your application via a secret management service (Hashicorp Vault, Kubernetes secrets, mounted secrets file, environment variable etc..)
The context
argument should be a string used to further isolate key derivation.
It can be any string (even empty if you don't wish to use it), but the same context that was used for encryption has to be provided when attempting to decrypt a file.
You could store this context in your database, but make sure it's an immutable value, like a foreign key. Random UUIDv4 are ideal for this kind of application.
import {
encryptFile,
decryptFile,
decodeMainSecret,
} from '@socialgouv/streaming-file-encryption'
import fs from 'node:fs'
import { pipeline } from 'node:stream/promises'
const mainSecret = decodeMainSecret(serializedMainSecret)
const context = 'test context'
// Encryption
await pipeline(
fs.createReadStream('path/to/cleartext.ext'),
encryptFile(mainSecret, context),
fs.createWriteStream('path/to/ciphertext.enc')
)
// Decryption
await pipeline(
fs.createReadStream('path/to/ciphertext.enc'),
decryptFile(mainSecret, context),
fs.createWriteStream('path/to/cleartext.ext')
)
By default, the underlying cipher being used is AES-256-GCM
, but you can
opt-in to using ChaCha20-Poly1305
instead:
encryptFile(mainSecret, context, 'chacha20-poly1305'),
Decryption handles files encrypted with either cipher transparently.
The main secret is passed via the environment, the context via a CLI flag, and the input and output files via stdin and stdout respectively:
export MAIN_SECRET=...
npx @socialgouv/streaming-file-encryption encrypt --context foo < image.jpg > image.jpg.sfe
npx @socialgouv/streaming-file-encryption decrypt --context foo < image.jpg.sfe > image.jpg
Note: you can use any of the following aliases for the context flag:
-c foo --ctx foo --context foo
Internally, the key used to encrypt the file will be derived from:
The security of this system requires knowing all three of these informations before being able to decrypt a file. It is then essential to keep them separate, for example:
It should require an attacker to break all three of your systems to be able to recompose individual file keys.
Encrypted files have the following properties:
There are two cryptographic parts to performing encryption and decryption:
Key derivation is done using HKDF-SHA-512
.
Symmetric encryption is done using either AES-256-GCM
or ChaCha20-Poly1305
on pages of 16kiB of cleartext at a time.
Final message authentication is done using HMAC-SHA-512
.
We generate a random 32 byte salt S
.
Two keys are derived using HKDF-SHA-512
:
One is for encryption, using the main secret as the IKM, the context as the info
and the salt S
, giving
an output key of 32 bytes (256 bits).
One is for HMAC, using the main secret, the context and a salt S + 1
(little
endian incrementation), giving an output key of 64 bytes.
Note: The purpose of the salt is to add entropy to key derivation aside from the two sources provided by the user (main secret & context). Differentiation via the salt ensures no key material is shared between encryption and HMAC.
We generate a random IV (12 bytes) IV
.
The ciphertext file starts with a 48 bytes header containing:
1a2g
for AES-256-GCM or 1c2p
for ChaCha20-Poly1305)The plain-text is broken down into pages of 16kiB (16384 bytes) to be encrypted individually with AES-256-GCM or ChaCha20-Poly1305.
Clear-text input pages are zero-padded (padding after data) to ensure a constant length of 16kiB. The actual data length is encoded on two bytes (little endian), and prepended to the page to encrypt:
The first page will use the encryption key obtained during key derivation and the randomly generated IV, and set its page index (0) as Authenticated Data, encoded as 32 bits little-endian.
Subsequent pages will then increment the IV (in a little-endian manner), and set their page index as Authenticated Data.
Note: Incrementing the IV serves two purposes: avoiding reusing IVs (nonces) across two pages (which would break the underlying cipher), and prevent reordering or skipping pages.
In an early version of the protocol where we did not use Authenticated Data (AD), this would have been enough to ensure integrity thanks to the use of the HMAC. However, we'd have known about an invalid first page only at the end of the stream, when the HMAC failed.
To detect an invalid first page earlier (truncation of the head of the file), we added the page index as AD.
The HMAC still serves to authenticate the end of the file, for which AD is not much use (unless we knew the stream length beforehand, which we assume we don't).
The AEAD authentication tag is appended after the associated ciphertext.
HMAC is computed over everything from the first byte of the version identifier in the header to the last byte of the authentication tag of the last page, and placed at the end (last 64 bytes) of the file.
Therefore, the binary file structure looks like this:
Header
vv vv vv vv aa aa aa aa aa aa aa aa aa aa aa aa | Version + IV
bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb | Salt (LSB)
bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb | Salt (MSB)
Page 1
cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc | Page 1 ciphertext
............................................... | (16386 bytes total)
cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc | Page 1 ciphertext
dd dd dd dd dd dd dd dd dd dd dd dd dd dd dd dd | Page 1 authentication tag
Page 2
ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee | Page 2 ciphertext
............................................... | (16386 bytes total)
ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee | Page 2 ciphertext
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff | Page 2 authentication tag
HMAC
xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx | HMAC-SHA-512
xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx | of everything
xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx | until
xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx | this point
Version is the ASCII string:
1a2g
(hex 31 61 32 67
) when using AES-256-GCM1c2p
(hex 31 63 32 70
) when using ChaCha20-Poly1305When decrypting a file:
For a single-page file, the final HMAC is indeed redundant with the authenticated encryption of AES-GCM / Poly1305.
However, consider an input file spanning multiple pages of 16kiB. While there are defenses in place to allow early detection of reordering or skipping of pages (incrementing the IV and using the page index as Authenticated Data), there are no guarantees that the last page is the last one that was encrypted: an attacker could drop the last N pages completely undetected, truncating the tail of the file.
Adding the HMAC (which covers everything behind it) allows authenticating the end of the file.
Pages are of a fixed size of 16kiB (+ 18 bytes to include the size and authentication tag), with the input being zero-padded before being encrypted.
This has the advantage of hiding the cleartext true size, at the cost of extra storage overhead for encrypted files.
The page size of 16kiB was chosen as a trade-off between the default buffer size in Node.js filesystem I/O streams (64kiB) and the overhead added by having constant page sizes.
While higher buffer size offered better CPU performance, they came with extra storage and RAM usage costs, so 16kiB was chosen as a middle ground.
It would indeed if the key was the same. In our case, keys are derived from both user-controlled randomness (the context), and from internal randomness (the salt). Even in the case where the context isn't used (constant), we still have 64 bytes of randomness fed into HKDF.
That being said, the pigeonhole principle states that there can be only $2^{256}$ different keys, for which collisions are infeasible.
But let's assume we do have a single key collision (you'd better check your CSPRNG). IVs are 96 bits long, and we'd need to have at least one page of overlap to compromise that key stream. To have a 50% chance of at least one page overlapping, we'd have to generate files of about $2^{94}$ pages, or $2^{(94+14)}$ bytes. That's about one trillion times the estimated size of Google datacenters.
This package is signed with sceau
.
The signature can be verified using the following public key:
sceau verify --publicKey cc5ce1aae47615906725d9859ae6c9202ca4406e14f242a4d1ef8a5a2cdadfb7
FAQs
Streaming file encryption for Node.js using AES-256-GCM, HKDF and HMAC.
The npm package @socialgouv/streaming-file-encryption receives a total of 81 weekly downloads. As such, @socialgouv/streaming-file-encryption popularity was classified as not popular.
We found that @socialgouv/streaming-file-encryption demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
Security News
The Socket Research team breaks down a malicious npm package targeting the legitimate DOMPurify library. It uses obfuscated code to hide that it is exfiltrating browser and crypto wallet data.
Security News
ENISA’s 2024 report highlights the EU’s top cybersecurity threats, including rising DDoS attacks, ransomware, supply chain vulnerabilities, and weaponized AI.