utf8

What is utf8?

The utf8 npm package is a library for encoding and decoding UTF-8 in JavaScript. It's designed to handle Unicode strings and convert them to and from UTF-8 byte arrays. It's useful for environments where native JavaScript functions for handling UTF-8 are not available or are not performing well.

What are utf8's main functionalities?

Encoding to UTF-8

This feature allows you to encode a JavaScript string (which is typically UCS-2 or UTF-16) into a UTF-8 encoded string. This is useful when you need to ensure that your data is in UTF-8 format, for example, when interacting with APIs or databases that expect UTF-8 encoded text.

"utf8.encode('Hello World!');"

Decoding from UTF-8

This feature allows you to decode a UTF-8 encoded string back into a regular JavaScript string. This is useful when you receive UTF-8 encoded data and need to convert it to a format that can be easily manipulated in JavaScript.

"utf8.decode('48656c6c6f20576f726c6421');"

Other packages similar to utf8

iconv-lite

iconv-lite is a popular npm package that offers similar functionality to utf8. It can encode and decode various character encodings, including UTF-8, ISO-8859-1, and Windows-1251. It's more comprehensive than utf8 because it supports a wider range of encodings, but it might be overkill if you only need to work with UTF-8.

text-encoding

text-encoding is another npm package that provides TextEncoder and TextDecoder APIs to convert between UTF-8 and UTF-16. It's a polyfill for the Encoding Living Standard's API and is useful for environments that do not support these APIs natively. It's similar to utf8 but also includes support for UTF-16 and other encodings.

README

utf8.js

utf8.js is a well-tested UTF-8 encoder/decoder written in JavaScript. Unlike many other JavaScript solutions, it is designed to be a proper UTF-8 encoder/decoder: it can encode/decode any scalar Unicode code point values, as per the Encoding Standard. Here’s an online demo.

Feel free to fork if you see possible improvements!

Installation

Via npm:

npm install utf8

In a browser:

<script src="utf8.js"></script>

In Node.js:

const utf8 = require('utf8');

API

utf8.encode(string)

Encodes any given JavaScript string (string) as UTF-8, and returns the UTF-8-encoded version of the string. It throws an error if the input string contains a non-scalar value, i.e. a lone surrogate. (If you need to be able to encode non-scalar values as well, use WTF-8 instead.)

// U+00A9 COPYRIGHT SIGN; see http://codepoints.net/U+00A9
utf8.encode('\xA9');
// → '\xC2\xA9'
// U+10001 LINEAR B SYLLABLE B038 E; see http://codepoints.net/U+10001
utf8.encode('\uD800\uDC01');
// → '\xF0\x90\x80\x81'

utf8.decode(byteString)

Decodes any given UTF-8-encoded string (byteString) as UTF-8, and returns the UTF-8-decoded version of the string. It throws an error when malformed UTF-8 is detected. (If you need to be able to decode encoded non-scalar values as well, use WTF-8 instead.)

utf8.decode('\xC2\xA9');
// → '\xA9'

utf8.decode('\xF0\x90\x80\x81');
// → '\uD800\uDC01'
// → U+10001 LINEAR B SYLLABLE B038 E

utf8.version

A string representing the semantic version number.

Support

utf8.js has been tested in at least Chrome 27-39, Firefox 3-34, Safari 4-8, Opera 10-28, IE 6-11, Node.js v0.10.0, Narwhal 0.3.2, RingoJS 0.8-0.11, PhantomJS 1.9.0, and Rhino 1.7RC4.

Unit tests & code coverage

After cloning this repository, run npm install to install the dependencies needed for development and testing. You may want to install Istanbul globally using npm install istanbul -g.

Once that’s done, you can run the unit tests in Node using npm test or node tests/tests.js. To run the tests in Rhino, Ringo, Narwhal, PhantomJS, and web browsers as well, use grunt test.

To generate the code coverage report, use grunt cover.

FAQ

Why is the first release named v2.0.0? Haven’t you heard of semantic versioning?

Long before utf8.js was created, the utf8 module on npm was registered and used by another (slightly buggy) library. @ryanmcgrath was kind enough to give me access to the utf8 package on npm when I told him about utf8.js. Since there has already been a v1.0.0 release of the old library, and to avoid breaking backwards compatibility with projects that rely on the utf8 npm package, I decided the tag the first release of utf8.js as v2.0.0 and take it from there.

Author

Mathias Bynens

License

utf8.js is available under the MIT license.