string_decoder

What is string_decoder?

The string_decoder npm package is used to decode buffer objects into strings in a manner that preserves encoded multi-byte UTF-8 and UTF-16 characters. It is particularly useful when dealing with streams of data that may contain such characters.

What are string_decoder's main functionalities?

Decoding Buffers to Strings

This feature allows for the conversion of buffer objects into strings, ensuring that multi-byte characters are not improperly split at the end of a buffer.

{"const { StringDecoder } = require('string_decoder'); const decoder = new StringDecoder('utf8'); const buffer = Buffer.from([0xE2, 0x82, 0xAC]); console.log(decoder.write(buffer)); // Outputs: '€'"}

Handling Incomplete Multi-byte Characters

This feature is useful when buffers are received in chunks and multi-byte characters are spread across multiple chunks. The StringDecoder ensures that these characters are not converted until the complete character is received.

{"const { StringDecoder } = require('string_decoder'); const decoder = new StringDecoder('utf8'); const buffers = [Buffer.from([0xE2]), Buffer.from([0x82]), Buffer.from([0xAC])]; let result = ''; buffers.forEach((buffer) => { result += decoder.write(buffer); }); result += decoder.end(); console.log(result); // Outputs: '€'"}

Other packages similar to string_decoder

iconv-lite

iconv-lite is a pure JavaScript implementation of character encoding conversion. It supports many different encodings and is more versatile than string_decoder, which only supports UTF-8, UTF-16LE/BE, and Base64. However, string_decoder is a built-in module and may be preferred for its simplicity when only these encodings are needed.

buffer

The buffer package on npm is a node.js core module for binary data manipulation. It can be used to handle binary data directly, but it does not offer the same convenience functions for decoding multi-byte characters as string_decoder does.

README

string_decoder

Node-core v8.9.4 string_decoder for userland

npm install --save string_decoder

Node-core string_decoder for userland

This package is a mirror of the string_decoder implementation in Node-core.

Full documentation may be found on the Node.js website.

As of version 1.0.0 string_decoder uses semantic versioning.

Previous versions

Previous version numbers match the versions found in Node core, e.g. 0.10.24 matches Node 0.10.24, likewise 0.11.10 matches Node 0.11.10.

Update

The build/ directory contains a build script that will scrape the source from the nodejs/node repo given a specific Node version.

Streams Working Group

string_decoder is maintained by the Streams Working Group, which oversees the development and maintenance of the Streams API within Node.js. The responsibilities of the Streams Working Group include:

• Addressing stream issues on the Node.js issue tracker.
• Authoring and editing stream documentation within the Node.js project.
• Reviewing changes to stream subclasses within the Node.js project.
• Redirecting changes to streams from the Node.js project to this project.
• Assisting in the implementation of stream providers within Node.js.
• Recommending versions of readable-stream to be included in Node.js.
• Messaging about the future of streams to give the community advance notice of changes.

See readable-stream for more details.