A javascript-only (esm/cjs) utf8 codec that is abstract-encoding compatible, well-tested and pretty efficient. Bonus: It doesn't use Nodejs' Buffer object and comes with typescript types.
import { encode, encodingLength, decode } from 'utf8-codec' // require works too!
const str = 'Hello World / こんにちは世界'
const bytes = encode(
str,
new Uint8Array(endcodingLength(str)), // own buffer supplied, optional
0 // offset, at which to write the str, optional
)
str === decode(bytes, 0, bytes.length)
The TextEncoder and TextDecoder classes exist to encode utf8 strings but they are not optimized for bigger byte processing and don't offer APIs to figure out how preemptively how many bytes are supposed to be written/read. Surprisingly this algorithm is even faster.
The other implementations found at the time do either not implement the edge cases properly and/or both directions of the codec.
FAQs
utf8 to/from bytes codec (esm/cjs)
The npm package utf8-codec receives a total of 16,115 weekly downloads. As such, utf8-codec popularity was classified as popular.
We found that utf8-codec demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.