The codepage npm package is a library for converting data between different character encodings. It supports a wide range of code pages and provides functionality for encoding and decoding text.
Encoding Conversion
Convert strings between different encodings. In this example, we convert a binary string to UTF-8 and vice versa using code page 437.
var cp = require('codepage');
var utf8String = cp.utils.decode(437, Buffer.from('Hello World', 'binary'));
var binaryString = cp.utils.encode(437, 'Hello World');Stream Transformation
Create a stream that transforms data from one encoding to another. Here, we create a transform stream that encodes incoming data to code page 437.
var cp = require('codepage');
var stream = require('stream');
var converter = new stream.Transform();
converter._transform = function(chunk, encoding, done) {
this.push(cp.utils.encode(437, chunk.toString()));
done();
};Handling Code Pages
Access information about different code pages. This code sample retrieves information about code page 437.
var cp = require('codepage');
var cpInfo = cp[437];
console.log(cpInfo.name);iconv-lite is a popular character encoding conversion library that works similarly to codepage. It supports many encodings and is often used for its simplicity and lightweight nature. Unlike codepage, iconv-lite does not rely on native Node.js modules, making it more portable across different platforms.
buffer-encoding is another package that offers encoding and decoding of text in Node.js. It provides a simple API for converting between different character encodings. It is similar to codepage but has a smaller footprint and fewer dependencies.
text-encoding is a polyfill for the Encoding Living Standard API provided by the WHATWG. It allows for encoding and decoding of text in the browser and Node.js. It is more comprehensive than codepage as it aims to provide a polyfill for the standard API, but it might be heavier in terms of size and complexity.
Codepages are character encodings. In many contexts, single- or double-byte character sets are used in lieu of Unicode encodings. The codepages map between characters and numbers.
In node:
var cptable = require('codepage');
In the browser:
<script src="cptable.js"></script>
<script src="cputils.js"></script>
Alternatively, use the full version in the dist folder:
<script src="cptable.full.js"></script>
The complete set of codepages is large due to some Double Byte Character Set
encodings. A much smaller file that only includes SBCS codepages is provided in
this repo (sbcs.js), as well as a file for other projects (cpexcel.js)
If you know which codepages you need, you can include individual scripts for
each codepage. The individual files are provided in the bits/ directory.
For example, to include only the Mac codepages:
<script src="bits/10000.js"></script>
<script src="bits/10006.js"></script>
<script src="bits/10007.js"></script>
<script src="bits/10029.js"></script>
<script src="bits/10079.js"></script>
<script src="bits/10081.js"></script>
All of the browser scripts define and append to the cptable object. To rename
the object, edit the JSVAR shell variable in make.sh and run the script.
The utilities functions are contained in cputils.js, which assumes that the
appropriate codepage scripts were loaded.
The script will manipulate module.exports if available . This is not always
desirable. To prevent the behavior, define DO_NOT_EXPORT_CODEPAGE.
Most codepages are indexed by number. To get the Unicode character for a given
codepoint, use the dec property:
var unicode_cp10000_255 = cptable[10000].dec[255]; // ˇ
To get the codepoint for a given character, use the enc property:
var cp10000_711 = cptable[10000].enc[String.fromCharCode(711)]; // 255
There are a few utilities that deal with strings and buffers:
var 汇总 = cptable.utils.decode(936, [0xbb,0xe3,0xd7,0xdc]);
var buf = cptable.utils.encode(936, 汇总);
var sushi= cptable.utils.decode(65001, [0xf0,0x9f,0x8d,0xa3]); // 🍣
var sbuf = cptable.utils.encode(65001, sushi);
cptable.utils.encode(CP, data, ofmt) accepts a String or Array of characters
and returns a representation controlled by ofmt:
ofmt == 'str', return a binary String (byte i is o.charCodeAt(i))ofmt == 'arr', return an Array of bytescptable.utils.decode(CP, data) accepts a byte String or Array of numbers or
Buffer and returns a JS string.
A much smaller script, including only the codepages known to be used in Excel,
is available under the name cpexcel. It exposes the same variable cptable
and is suitable as a drop-in replacement when the full codepage tables are not
needed.
In node:
var cptable = require('codepage/dist/cpexcel.full');
The make.sh script in the repo can take a manifest and generate JS source.
Usage:
$ bash make.sh path_to_manifest output_file_name JSVAR
where
JSVAR is the name of the exported variable (generally cptable)output_file_name is the output file (cpexcel.js, cptable.js, ...)path_to_manifest is the path to the manifest file.The manifest file is expected to be a CSV with 3 columns:
<codepage number>,<source>,<size>
If a source is specified, it will try to download the specified file and parse.
The file format is expected to follow the format from the unicode.org site.
The size should be 1 for a single-byte codepage and 2 for a double-byte
codepage. For mixed codepages (which use some single- and some double-byte
codes), the script assumes the mapping is a prefix code and generates efficient
JS code.
Generated scripts only include the mapping. cat a mapping with cputils.js
to produce a complete script like cpexcel.full.js.
This script uses voc. The script to build the codepage tables and
the JS source is codepage.md, so building involves voc codepage.md.
The complete list of codepages can be found in the file pages.csv.
Some codepages are easier to implement algorithmically. Since those character tables are not generated, there is no corresponding entry (they are "magic").
| CP# | Source | Description |
|---|---|---|
37 | unicode.org | IBM EBCDIC US-Canada |
437 | unicode.org | OEM United States |
500 | unicode.org | IBM EBCDIC International |
620 | NLS | Mazovia (Polish) MS-DOS |
708 | Windows 7 | Arabic (ASMO 708) |
720 | Windows 7 | Arabic (Transparent ASMO); Arabic (DOS) |
737 | unicode.org | OEM Greek (formerly 437G); Greek (DOS) |
775 | unicode.org | OEM Baltic; Baltic (DOS) |
808 | unicode.org | OEM Russian; Cyrillic + Euro symbol |
850 | unicode.org | OEM Multilingual Latin 1; Western European (DOS) |
852 | unicode.org | OEM Latin 2; Central European (DOS) |
855 | unicode.org | OEM Cyrillic (primarily Russian) |
857 | unicode.org | OEM Turkish; Turkish (DOS) |
858 | Windows 7 | OEM Multilingual Latin 1 + Euro symbol |
860 | unicode.org | OEM Portuguese; Portuguese (DOS) |
861 | unicode.org | OEM Icelandic; Icelandic (DOS) |
862 | unicode.org | OEM Hebrew; Hebrew (DOS) |
863 | unicode.org | OEM French Canadian; French Canadian (DOS) |
864 | unicode.org | OEM Arabic; Arabic (864) |
865 | unicode.org | OEM Nordic; Nordic (DOS) |
866 | unicode.org | OEM Russian; Cyrillic (DOS) |
869 | unicode.org | OEM Modern Greek; Greek, Modern (DOS) |
870 | Windows 7 | IBM EBCDIC Multilingual/ROECE (Latin 2) |
872 | unicode.org | OEM Cyrillic (primarily Russian) + Euro Symbol |
874 | unicode.org | Windows Thai |
875 | unicode.org | IBM EBCDIC Greek Modern |
895 | NLS | Kamenický (Czech) MS-DOS |
932 | unicode.org | Japanese Shift-JIS |
936 | unicode.org | Simplified Chinese GBK |
949 | unicode.org | Korean |
950 | unicode.org | Traditional Chinese Big5 |
1010 | IBM | IBM EBCDIC French |
1026 | unicode.org | IBM EBCDIC Turkish (Latin 5) |
1047 | Windows 7 | IBM EBCDIC Latin 1/Open System |
1132 | IBM | IBM EBCDIC Lao (1132 / 1133 / 1341) |
1140 | Windows 7 | IBM EBCDIC US-Canada (037 + Euro symbol) |
1141 | Windows 7 | IBM EBCDIC Germany (20273 + Euro symbol) |
1142 | Windows 7 | IBM EBCDIC Denmark-Norway (20277 + Euro symbol) |
1143 | Windows 7 | IBM EBCDIC Finland-Sweden (20278 + Euro symbol) |
1144 | Windows 7 | IBM EBCDIC Italy (20280 + Euro symbol) |
1145 | Windows 7 | IBM EBCDIC Latin America-Spain (20284 + Euro symbol) |
1146 | Windows 7 | IBM EBCDIC United Kingdom (20285 + Euro symbol) |
1147 | Windows 7 | IBM EBCDIC France (20297 + Euro symbol) |
1148 | Windows 7 | IBM EBCDIC International (500 + Euro symbol) |
1149 | Windows 7 | IBM EBCDIC Icelandic (20871 + Euro symbol) |
1200 | magic | Unicode UTF-16, little endian (BMP of ISO 10646) |
1201 | magic | Unicode UTF-16, big endian |
1250 | unicode.org | Windows Central Europe |
1251 | unicode.org | Windows Cyrillic |
1252 | unicode.org | Windows Latin I |
1253 | unicode.org | Windows Greek |
1254 | unicode.org | Windows Turkish |
1255 | unicode.org | Windows Hebrew |
1256 | unicode.org | Windows Arabic |
1257 | unicode.org | Windows Baltic |
1258 | unicode.org | Windows Vietnam |
1361 | Windows 7 | Korean (Johab) |
10000 | unicode.org | MAC Roman |
10001 | Windows 7 | Japanese (Mac) |
10002 | Windows 7 | MAC Traditional Chinese (Big5) |
10003 | Windows 7 | Korean (Mac) |
10004 | Windows 7 | Arabic (Mac) |
10005 | Windows 7 | Hebrew (Mac) |
10006 | unicode.org | Greek (Mac) |
10007 | unicode.org | Cyrillic (Mac) |
10008 | Windows 7 | MAC Simplified Chinese (GB 2312) |
10010 | Windows 7 | Romanian (Mac) |
10017 | Windows 7 | Ukrainian (Mac) |
10021 | Windows 7 | Thai (Mac) |
10029 | unicode.org | MAC Latin 2 (Central European) |
10079 | unicode.org | Icelandic (Mac) |
10081 | unicode.org | Turkish (Mac) |
10082 | Windows 7 | Croatian (Mac) |
12000 | magic | Unicode UTF-32, little endian byte order |
12001 | magic | Unicode UTF-32, big endian byte order |
20000 | Windows 7 | CNS Taiwan (Chinese Traditional) |
20001 | Windows 7 | TCA Taiwan |
20002 | Windows 7 | ETEN Taiwan (Chinese Traditional) |
20003 | Windows 7 | IBM5550 Taiwan |
20004 | Windows 7 | TeleText Taiwan |
20005 | Windows 7 | Wang Taiwan |
20105 | Windows 7 | Western European IA5 (IRV International Alphabet 5) |
20106 | Windows 7 | IA5 German (7-bit) |
20107 | Windows 7 | IA5 Swedish (7-bit) |
20108 | Windows 7 | IA5 Norwegian (7-bit) |
20127 | magic | US-ASCII (7-bit) |
20261 | Windows 7 | T.61 |
20269 | Windows 7 | ISO 6937 Non-Spacing Accent |
20273 | Windows 7 | IBM EBCDIC Germany |
20277 | Windows 7 | IBM EBCDIC Denmark-Norway |
20278 | Windows 7 | IBM EBCDIC Finland-Sweden |
20280 | Windows 7 | IBM EBCDIC Italy |
20284 | Windows 7 | IBM EBCDIC Latin America-Spain |
20285 | Windows 7 | IBM EBCDIC United Kingdom |
20290 | Windows 7 | IBM EBCDIC Japanese Katakana Extended |
20297 | Windows 7 | IBM EBCDIC France |
20420 | Windows 7 | IBM EBCDIC Arabic |
20423 | Windows 7 | IBM EBCDIC Greek |
20424 | Windows 7 | IBM EBCDIC Hebrew |
20833 | Windows 7 | IBM EBCDIC Korean Extended |
20838 | Windows 7 | IBM EBCDIC Thai |
20866 | Windows 7 | Russian Cyrillic (KOI8-R) |
20871 | Windows 7 | IBM EBCDIC Icelandic |
20880 | Windows 7 | IBM EBCDIC Cyrillic Russian |
20905 | Windows 7 | IBM EBCDIC Turkish |
20924 | Windows 7 | IBM EBCDIC Latin 1/Open System (1047 + Euro symbol) |
20932 | Windows 7 | Japanese (JIS 0208-1990 and 0212-1990) |
20936 | Windows 7 | Simplified Chinese (GB2312-80) |
20949 | Windows 7 | Korean Wansung |
21025 | Windows 7 | IBM EBCDIC Cyrillic Serbian-Bulgarian |
21027 | NLS | Extended/Ext Alpha Lowercase |
21866 | Windows 7 | Ukrainian Cyrillic (KOI8-U) |
28591 | unicode.org | ISO 8859-1 Latin 1 (Western European) |
28592 | unicode.org | ISO 8859-2 Latin 2 (Central European) |
28593 | unicode.org | ISO 8859-3 Latin 3 |
28594 | unicode.org | ISO 8859-4 Baltic |
28595 | unicode.org | ISO 8859-5 Cyrillic |
28596 | unicode.org | ISO 8859-6 Arabic |
28597 | unicode.org | ISO 8859-7 Greek |
28598 | unicode.org | ISO 8859-8 Hebrew (ISO-Visual) |
28599 | unicode.org | ISO 8859-9 Turkish |
28600 | unicode.org | ISO 8859-10 Latin 6 |
28601 | unicode.org | ISO 8859-11 Latin (Thai) |
28603 | unicode.org | ISO 8859-13 Latin 7 (Estonian) |
28604 | unicode.org | ISO 8859-14 Latin 8 (Celtic) |
28605 | unicode.org | ISO 8859-15 Latin 9 |
28606 | unicode.org | ISO 8859-15 Latin 10 |
29001 | Windows 7 | Europa 3 |
38598 | Windows 7 | ISO 8859-8 Hebrew (ISO-Logical) |
47451 | unicode.org | Atari ST/TT |
50220 | magic | ISO 2022 JIS Japanese with no halfwidth Katakana |
50221 | magic | ISO 2022 JIS Japanese with halfwidth Katakana |
50222 | magic | ISO 2022 Japanese JIS X 0201-1989 (1 byte Kana-SO/SI) |
50225 | magic | ISO 2022 Korean |
50227 | magic | ISO 2022 Simplified Chinese |
51932 | Windows 7 | EUC Japanese |
51936 | Windows 7 | EUC Simplified Chinese |
51949 | Windows 7 | EUC Korean |
52936 | Windows 7 | HZ-GB2312 Simplified Chinese |
54936 | Windows 7 | GB18030 Simplified Chinese (4 byte) |
57002 | Windows 7 | ISCII Devanagari |
57003 | Windows 7 | ISCII Bengali |
57004 | Windows 7 | ISCII Tamil |
57005 | Windows 7 | ISCII Telugu |
57006 | Windows 7 | ISCII Assamese |
57007 | Windows 7 | ISCII Oriya |
57008 | Windows 7 | ISCII Kannada |
57009 | Windows 7 | ISCII Malayalam |
57010 | Windows 7 | ISCII Gujarati |
57011 | Windows 7 | ISCII Punjabi |
65000 | magic | Unicode (UTF-7) |
65001 | magic | Unicode (UTF-8) |
unicode.org refers to the Unicode Consortium Public Mappings, a database of
various mappings between Unicode characters and respective character sets. The
tables are processed by a few scripts in the build process.
IBM refers to the IBM coded character set database. Even though IBM uses a
different numbering scheme from Windows, the IBM numbers are used when there is
no conflict. The tables are manually generated from the symbol manifests.
Windows 7 refers to direct inspection of Windows 7 machines using .NET class
System.Text.Encoding. The enclosed MakeEncoding.cs C# program brute-forces
code pages. MakeEncoding.cs deviates from unicode.org in some cases. When they
map a given code to different characters, unicode.org value is used. When
unicode.org does not prescribe a value, MakeEncoding.cs value is used.
NLS refers to the National Language Support files supplied in various versions
of Windows. In older versions of Windows (like Windows 98) these files followed
the name pattern CP_#.NLS, but newer versions use the name pattern C_#.NLS.
make test will run the nodejs-based test.
To run the in-browser tests, run a local server and go to the ctest directory.
make ctestserv will start a python SimpleHTTPServer server on port 8000.
To update the browser artifacts, run make ctest.
Please consult the attached LICENSE file for details. All rights not explicitly granted by the Apache 2.0 license are reserved by the Original Author.
FAQs
pure-JS library to handle codepages
The npm package codepage receives a total of 2,442,299 weekly downloads. As such, codepage popularity was classified as popular.
We found that codepage demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.