Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
iconv-lite
Advanced tools
Package description
The iconv-lite npm package provides utilities for converting character encodings in pure JavaScript. It supports many different encodings and can convert to and from Buffer objects without the need for a native C++ binding. This makes it a lightweight and portable solution for encoding conversion.
Encoding Conversion
Converts text from one character encoding to another. The example shows how to decode a buffer to a string and encode a string to a buffer using Windows-1251 encoding.
const iconv = require('iconv-lite');
// Convert from an encoded buffer to js string.
const str = iconv.decode(Buffer.from([0x63, 0x61, 0x66, 0xe9]), 'win1251');
// Convert from js string to an encoded buffer.
const buf = iconv.encode('Sample input text', 'win1251');
Streaming Conversion
Provides a streaming interface for encoding conversion. This example demonstrates how to create a read stream from a file and pipe it through iconv-lite's decode stream.
const iconv = require('iconv-lite');
const fs = require('fs');
// Decode stream (from a file, for example)
const readStream = fs.createReadStream('file.txt');
const decodeStream = iconv.decodeStream('win1251');
readStream.pipe(decodeStream);
decodeStream.on('data', function(str) {
console.log(str); // converted text
});
Encoding Detection
Checks if a particular encoding is supported by iconv-lite. The example checks if UTF-8 encoding is supported.
const iconv = require('iconv-lite');
// Check if encoding is supported
const encodingSupported = iconv.encodingExists('utf-8');
console.log(encodingSupported); // true or false
The 'buffer' package is a Node.js core module that provides a way to handle binary data. It does not offer direct encoding conversion like iconv-lite, but it can be used in conjunction with other modules to achieve similar results.
The 'iconv' package is similar to iconv-lite but relies on native C++ bindings. It provides more comprehensive encoding support but is not as lightweight or portable as iconv-lite due to its compiled components.
The 'string_decoder' module is a Node.js core module that provides an API for decoding Buffer objects into strings using a specified character encoding. It is less feature-rich compared to iconv-lite and does not support as many encodings.
Readme
var iconv = require('iconv-lite');
// Convert from an encoded buffer to js string.
str = iconv.decode(new Buffer([0x68, 0x65, 0x6c, 0x6c, 0x6f]), 'win1251');
// Convert from js string to an encoded buffer.
buf = iconv.encode("Sample input string", 'win1251');
// Check if encoding is supported
iconv.encodingExists("us-ascii")
// Decode stream (from binary stream to js strings)
http.createServer(function(req, res) {
var converterStream = iconv.decodeStream('win1251');
req.pipe(converterStream);
converterStream.on('data', function(str) {
console.log(str); // Do something with decoded strings, chunk-by-chunk.
});
});
// Convert encoding streaming example
fs.createReadStream('file-in-win1251.txt')
.pipe(iconv.decodeStream('win1251'))
.pipe(iconv.encodeStream('ucs2'))
.pipe(fs.createWriteStream('file-in-ucs2.txt'));
// Sugar: all encode/decode streams have .collect(cb) method to accumulate data.
http.createServer(function(req, res) {
req.pipe(iconv.decodeStream('win1251')).collect(function(err, body) {
assert(typeof body == 'string');
console.log(body); // full request body string
});
});
// After this call all Node basic primitives will understand iconv-lite encodings.
iconv.extendNodeEncodings();
// Examples:
buf = new Buffer(str, 'win1251');
buf.write(str, 'gbk');
str = buf.toString('latin1');
assert(Buffer.isEncoding('iso-8859-15'));
Buffer.byteLength(str, 'us-ascii');
http.createServer(function(req, res) {
req.setEncoding('big5');
req.collect(function(err, body) {
console.log(body);
});
});
fs.createReadStream("file.txt", "shift_jis");
// External modules are also supported (if they use Node primitives, which they probably do).
request = require('request');
request({
url: "http://github.com/",
encoding: "cp932"
});
// To remove extensions
iconv.undoExtendNodeEncodings();
See all supported encodings on wiki.
Most singlebyte encodings are generated automatically from node-iconv. Thank you Ben Noordhuis and libiconv authors!
Multibyte encodings are generated from Unicode.org mappings and WHATWG Encoding Standard mappings. Thank you, respective authors!
Comparison with node-iconv module (1000x256kb, on MacBook Pro, Core i5/2.6 GHz, Node v0.12.0). Note: your results may vary, so please always check on your hardware.
operation iconv@2.1.4 iconv-lite@0.4.7
----------------------------------------------------------
encode('win1251') ~96 Mb/s ~320 Mb/s
decode('win1251') ~95 Mb/s ~246 Mb/s
When decoding, be sure to supply a Buffer to decode() method, otherwise bad things usually happen.
Untranslatable characters are set to � or ?. No transliteration is currently supported.
Uses BOM to determine endianness, but doesn't remove it. Use 'strip-bom' module.
Node versions 0.10.31 and 0.11.13 are buggy, don't use them (see #65, #77).
$ git clone git@github.com:ashtuchkin/iconv-lite.git
$ cd iconv-lite
$ npm install
$ npm test
$ # To view performance:
$ node test/performance.js
$ # To view test coverage:
$ npm run coverage
$ open coverage/lcov-report/index.html
FAQs
Convert character encodings in pure javascript.
The npm package iconv-lite receives a total of 55,928,682 weekly downloads. As such, iconv-lite popularity was classified as popular.
We found that iconv-lite demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.