chardet

Universal character encoding detector.

chardet 7 is a ground-up, 0BSD-licensed rewrite of chardet. Same package name, same public API — drop-in replacement for chardet 5.x/6.x, just much faster and more accurate. Python 3.10+, zero runtime dependencies, works on PyPy.

Read more details about the rewrite process.

Why chardet 7?

99.7% accuracy on 3,138 test files. 359x faster than chardet 6.0.0, and +13.1pp more accurate than charset-normalizer 3.5.2 while being 1.3x faster. Language detection for every text result. MIME type detection for binary files. 0BSD licensed.

Raising the byte cap on large inputs doesn't make detection slow: detect(data, max_bytes=len(data)) on a 272 MiB file finishes in about 0.2 to 0.5 s, and the UTF-8 verdict is validated over every byte examined, never sampled. The default 200 KB cap is still the fastest path. See the performance docs for the full large-input tables.

	chardet 7.7.0 (compiled)	chardet 6.0.0	charset-normalizer 3.5.2
Accuracy (3,138 files)	99.7%	84.4%	86.6%
Speed	3,125 files/s (771 pure)	9 files/s	2,438 files/s
Language detection	93.0%	38.7%	54.8%
Peak memory	28.2 MiB	n/a	71.0 MiB
Streaming detection	yes	yes	no
Encoding era filtering	yes	no	no
Encoding filters	yes	no	yes
MIME type detection	yes	no	no
Supported encodings	99	84	99
License	0BSD	LGPL	MIT

Installation

pip install chardet

Quick Start

import chardet

chardet.detect(b"Python is a great programming language for beginners and experts alike.")
# {'encoding': 'ascii', 'confidence': 1.0, 'language': 'en', 'mime_type': 'text/plain'}

# UTF-8 English with accented characters
chardet.detect("The naïve approach doesn't always work in complex systems.".encode("utf-8"))
# {'encoding': 'utf-8', 'confidence': 0.84, 'language': 'en', 'mime_type': 'text/plain'}

# Japanese EUC-JP
chardet.detect("日本語の文字コード検出テストです。このテキストはEUC-JPでエンコードされています。正しく検出できるか確認します。".encode("euc-jp"))
# {'encoding': 'EUC-JP', 'confidence': 1.0, 'language': 'ja', 'mime_type': 'text/plain'}

# Get all candidate encodings ranked by confidence
text = "Le café est une boisson très populaire en France et dans le monde entier."
results = chardet.detect_all(text.encode("windows-1252"))
for r in results[:4]:
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1252 0.31
# iso8859-15 0.31
# ISO-8859-1 0.31
# MacRoman 0.3

Streaming Detection

For large files or network streams, use UniversalDetector to feed data incrementally:

from chardet import UniversalDetector

detector = UniversalDetector()
with open("unknown.txt", "rb") as f:
    for line in f:
        detector.feed(line)
        if detector.done:
            break
result = detector.close()
print(result)

Encoding Era Filtering

Restrict detection to specific encoding eras to reduce false positives:

from chardet import detect_all
from chardet.enums import EncodingEra

data = "Москва является столицей Российской Федерации и крупнейшим городом страны.".encode("windows-1251")

# All encoding eras are considered by default — 4 candidates across eras
for r in detect_all(data):
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1251 0.46
# MacCyrillic 0.42
# KZ1048 0.2
# ptcp154 0.2

# Restrict to modern web encodings — 1 confident result
for r in detect_all(data, encoding_era=EncodingEra.MODERN_WEB):
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1251 0.46

Encoding Filters

Restrict detection to specific encodings, or exclude encodings you don't want:

# Only consider UTF-8 and Windows-1252
chardet.detect(data, include_encodings=["utf-8", "windows-1252"])

# Drop two EBCDIC code pages (for all of them, use
# encoding_era=EncodingEra.ALL & ~EncodingEra.MAINFRAME)
chardet.detect(data, exclude_encodings=["cp037", "cp500"])

CLI

chardetect somefile.txt
# somefile.txt: utf-8 with confidence 0.99

chardetect --minimal somefile.txt
# utf-8

# Include detected language
chardetect -l somefile.txt
# somefile.txt: utf-8 en (English) with confidence 0.99

# Only consider specific encodings
chardetect -i utf-8,windows-1252 somefile.txt
# somefile.txt: utf-8 with confidence 0.99

# Pipe from stdin
cat somefile.txt | chardetect
# stdin: utf-8 with confidence 0.99

This is the beginning of the README. Read the rest at the original. Read the full README ↗