detect-file-encoding-and-language
Advanced tools
Readme
Determine the encoding and language of text files!
For reliable encoding and language detection, use files containing 500 words or more. Smaller inputs can work as well but the results might be less accurate and in some cases incorrect.
Feel free to test the functionality of this NPM package here. Upload your own files and see if the encoding and language are detected correctly!
There are several ways in which you can use this NPM package. You can use it as a command-line interface, server-side with Node.js or client-side in the browser.
In the body section of your html file, create an input element of type file and give it an id.
// index.html
<body>
<input type="file" id="my-input-field" />
<script src="app.js"></script>
</body>
Next, load the module either by using the script tag or by using a bundler!
When loading it via the <script> tag, you can either use the CDN version or download the code itself and include it in your project. For a quickstart use the CDN version. If you want to be able to use it offline, download and include it!
// index.html
<body>
<input type="file" id="my-input-field" />
<script src="https://unpkg.com/detect-file-encoding-and-language/umd/language-encoding.min.js"></script>
<script src="app.js"></script>
</body>
Now that you've loaded the module, you can start using it.
lib inside your root directorylib create a new file and call it language-encoding.min.jsUTF-8 or UTF-8 with BOM before proceeding!language-encoding.min.js and save itlanguage-encoding.min.js via the <script> tag.// index.html
<body>
<input type="file" id="my-input-field" />
<script src="lib/language-encoding.min.js"></script>
<script src="app.js"></script>
</body>
The <script> tag exposes the languageEncoding function to everything in the DOM located beneath it. When you call it and pass in the file that you want to analyze, it'll return a Promise that you can use to retrieve the encoding, language and confidenc score as shown in the example below.
// app.js
document
.getElementById("my-input-field")
.addEventListener("change", inputHandler);
function inputHandler(e) {
const file = e.target.files[0];
languageEncoding(file).then((fileInfo) => console.log(fileInfo));
// Possible result: { language: english, encoding: UTF-8, confidence: 0.97}
}
$ npm install detect-file-encoding-and-language
// app.js
const languageEncoding = require("detect-file-encoding-and-language");
document
.getElementById("my-input-field")
.addEventListener("change", inputHandler);
function inputHandler(e) {
const file = e.target.files[0];
languageEncoding(file).then((fileInfo) => console.log(fileInfo));
// Possible result: { language: english, encoding: UTF-8, confidence: 0.97}
}
Note: This works great with frameworks such as React because they are doing the bundling for you. However, if you're using pure vanilla Javascript you will have to bundle it yourself!
$ npm install detect-file-encoding-and-language
// index.js
const languageEncoding = require("detect-file-encoding-and-language");
const pathToFile = "/home/username/documents/my-text-file.txt";
languageEncoding(pathToFile).then((fileInfo) => console.log(fileInfo));
// Possible result: { language: japanese, encoding: Shift-JIS, confidence: 1 }
$ npm install -g detect-file-encoding-and-language
Once installed you'll be able to use the command dfeal to retrieve the encoding and language of your text files.
$ dfeal "/home/user name/Documents/subtitle file.srt"
# Possible result: { language: french, encoding: CP1252, confidence: 0.99 }
or without quotation marks, using backslashes to escape spaces:
$ dfeal /home/user\ name/Documents/subtitle\ file.srt
# Possible result: { language: french, encoding: CP1252, confidence: 0.99 }
The confidence score ranges from 0 to 1. It is based on the amount of matches that were found for a particular language and the frequency of those matches. If you want to learn more about how it all works, check out the Wiki entry!
This project is licensed under the MIT License
FAQs
Charset Detector - Detect the encoding and language of text files - Use it in the browser, with Node.js, or via CLI
The npm package detect-file-encoding-and-language receives a total of 20,290 weekly downloads. As such, detect-file-encoding-and-language popularity was classified as popular.
We found that detect-file-encoding-and-language demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
A hospital in Mobile, Alabama, agreed to a settlement in a landmark ransomware death lawsuit, but is now reportedly reconsidering the agreement and refusing to pay.
Security News
A new report explores how advancements in LLMs are enhancing cyber threats, including polymorphic malware, personalized spearphishing, and the risk of hijacking customer service bots.
Security News
ESLint has approved an RFC that adds support for TypeScript configuration files, which is aimed at improving the developer experience and recognizing changes in the evolving JavaScript ecosystem.
