Readme
cChardet is high speed universal character encoding detector. - binding to uchardet_.
.. image:: https://badge.fury.io/py/cchardet.svg :target: https://badge.fury.io/py/cchardet :alt: PyPI version
.. image:: https://github.com/PyYoshi/cChardet/workflows/Build%20for%20Linux/badge.svg?branch=master :target: https://github.com/PyYoshi/cChardet/actions?query=workflow%3A%22Build+for+Linux%22 :alt: Build for Linux
.. image:: https://github.com/PyYoshi/cChardet/workflows/Build%20for%20macOS/badge.svg?branch=master :target: https://github.com/PyYoshi/cChardet/actions?query=workflow%3A%22Build+for+macOS%22 :alt: Build for macOS
.. image:: https://github.com/PyYoshi/cChardet/workflows/Build%20for%20windows/badge.svg?branch=master :target: https://github.com/PyYoshi/cChardet/actions?query=workflow%3A%22Build+for+windows%22 :alt: Build for Windows
International (Unicode)
Arabic
Bulgarian
Chinese
Croatian:
Czech
Danish
English
Esperanto
Estonian
Finnish
French
German
Greek
Hebrew
Hungarian:
Irish Gaelic
Italian
Japanese
Korean
Lithuanian
Latvian
Maltese
Polish:
Portuguese
Romanian:
Russian
Slovak
Slovene
.. code-block:: python
# -*- coding: utf-8 -*-
import cchardet as chardet
with open(r"src/tests/samples/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt", "rb") as f:
msg = f.read()
result = chardet.detect(msg)
print(result)
.. code-block:: bash
$ cd src/
$ pip install chardet
$ python tests/bench.py
Results
CPU: Intel(R) Core(TM) i5-4690 CPU @ 3.50GHz
RAM: DDR3 1600Mhz 16GB
Platform: Ubuntu 16.04 amd64
Python 3.6.1
^^^^^^^^^^^^
+-----------------+------------------+
| | Request (call/s) |
+=================+==================+
| chardet v3.0.2 | 0.35 |
+-----------------+------------------+
| cchardet v2.0.1 | 1467.77 |
+-----------------+------------------+
LICENSE
-------
See **COPYING** file.
Contact
-------
- `Issues`_
.. _uchardet: https://github.com/PyYoshi/uchardet
.. _Issues: https://github.com/PyYoshi/cChardet/issues?page=1&state=open
Platform
--------
Support
Do not Support
- `Anaconda`_
- `pyenv`_
.. _Anaconda: https://www.anaconda.com/
.. _pyenv: https://github.com/pyenv/pyenv
CHANGES
=======
2.1.7 (2020-10-27)
------------------
- support Python 3.9
- drop support for Python 3.5
2.1.6 (2020-03-17)
------------------
- drop support for Python 2.7
- support Github Actions
- update dev-dependencies
2.1.5 (2019-09-27)
------------------
- update language models (uchardet)
- add iso8859-2 test but disabled it
- support Python 3.8
- drop support for Python 3.4
2.1.4 (2018-09-27)
------------------
- disable LTO because become poor performance
2.1.3 (2018-09-26)
------------------
- support Python 3.7
2.1.2 (2018-09-26)
------------------
- enable `LTO`_ for wheel builds
- update Cython
.. _LTO: https://gcc.gnu.org/wiki/LinkTimeOptimization
2.1.1 (2017-07-01)
------------------
- fix that different results with different chuck sizes
- fix that assignments to nsSMState in nsCodingStateMachine result in unspecified behavior
- include COPYING in package
2.1.0 (2017-05-15)
------------------
- add cchardetect CLI script (`#30`_) `@craigds`_
.. _#30: https://github.com/PyYoshi/cChardet/pull/30
.. _@craigds: https://github.com/craigds
2.0.1 (2017-04-25)
------------------
- fix an issue where UTF-8 with a BOM would not be detected as UTF-8-SIG (fix `#28`_)
- pass NULL Byte to feed() / detect() (fix `#27`_)
.. _#28: https://github.com/PyYoshi/cChardet/issues/28
.. _#27: https://github.com/PyYoshi/cChardet/issues/27
2.0.0 (2017-04-06)
------------------
- Improve tests
2.0a4 (2017-04-05)
------------------
- Update uchardet repo (Fix buffer overflow)
2.0a3 (2017-03-29)
------------------
- Implement UniversalDetector (like chardet)
2.0a2 (2017-03-28)
------------------
- Update uchardet repo (Fix memory leak)
2.0a1 (2017-03-28)
------------------
- Replace `uchardet-enhanced`_ to `uchardet`_
- Remove Detector class
.. _uchardet-enhanced: https://bitbucket.org/medoc/uchardet-enhanced/overview
.. _uchardet: https://github.com/PyYoshi/uchardet
1.1.3 (2017-02-26)
------------------
- Support AArch64
1.1.2 (2017-01-08)
------------------
- Support Python 3.6
1.1.1 (2016-11-05)
------------------
- Use len() function (9e61cb9e96b138b0d18e5f9e013e144202ae4067)
- Remove detect function in _cchardet.pyx (25b581294fc0ae8f686ac9972c8549666766f695)
- Support manylinux1 wheel
1.1.0 (2016-10-17)
------------------
- Add Detector class
- Improve unit tests
FAQs
cChardet is high speed universal character encoding detector.
We found that cchardet demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Researchers have demonstrated that teams of LLM agents can exploit zero-day vulnerabilities with a 53% success rate, and the costs of using AI to do so are rapidly becoming more affordable than hiring a human penetration tester.
Security News
In an unprecedented surge, May 2024 saw the publication of over 5,000 CVEs, marking a historic milestone in cybersecurity with an average of 164 CVEs per day, nearly double the 2023 daily average.
Security News
The White House is addressing fragmented cybersecurity regulations as CISOs report spending up to 50% of their time on compliance, aiming to harmonize requirements and improve cybersecurity outcomes.