string_decoder

What is string_decoder?

The string_decoder npm package is used to decode buffer objects into strings in a manner that preserves encoded multi-byte UTF-8 and UTF-16 characters. It is particularly useful when dealing with streams of data that may contain such characters.

What are string_decoder's main functionalities?

Decoding Buffers to Strings

This feature allows for the conversion of buffer objects into strings, ensuring that multi-byte characters are not improperly split at the end of a buffer.

{"const { StringDecoder } = require('string_decoder'); const decoder = new StringDecoder('utf8'); const buffer = Buffer.from([0xE2, 0x82, 0xAC]); console.log(decoder.write(buffer)); // Outputs: '€'"}

Handling Incomplete Multi-byte Characters

This feature is useful when buffers are received in chunks and multi-byte characters are spread across multiple chunks. The StringDecoder ensures that these characters are not converted until the complete character is received.

{"const { StringDecoder } = require('string_decoder'); const decoder = new StringDecoder('utf8'); const buffers = [Buffer.from([0xE2]), Buffer.from([0x82]), Buffer.from([0xAC])]; let result = ''; buffers.forEach((buffer) => { result += decoder.write(buffer); }); result += decoder.end(); console.log(result); // Outputs: '€'"}

Other packages similar to string_decoder

iconv-lite

iconv-lite is a pure JavaScript implementation of character encoding conversion. It supports many different encodings and is more versatile than string_decoder, which only supports UTF-8, UTF-16LE/BE, and Base64. However, string_decoder is a built-in module and may be preferred for its simplicity when only these encodings are needed.

buffer

The buffer package on npm is a node.js core module for binary data manipulation. It can be used to handle binary data directly, but it does not offer the same convenience functions for decoding multi-byte characters as string_decoder does.

README

string_decoder.js (require('string_decoder')) from Node.js core

Copyright Joyent, Inc. and other Node contributors. See LICENCE file for details.

Version numbers match the versions found in Node core, e.g. 0.10.24 matches Node 0.10.24, likewise 0.11.10 matches Node 0.11.10. Prefer the stable version over the unstable.

The build/ directory contains a build script that will scrape the source from the joyent/node repo given a specific Node version.