xxHash - Extremely fast hash algorithm

Website · API documentation · xxhsum manual · Format specification

xxHash is an extremely fast non-cryptographic hash algorithm, working at RAM speed limits. It is highly portable and produces identical hashes on all platforms, including little- and big-endian systems. Once finalized, algorithm outputs remain stable across xxHash releases.

The library includes the following algorithms:

• XXH32 generates 32-bit hashes, using 32-bit arithmetic.
• XXH64 generates 64-bit hashes, using 64-bit arithmetic.
• XXH3, available since v0.8.0, generates 64-bit or 128-bit hashes using vectorized arithmetic. The 128-bit variant is called XXH128.

For new applications, XXH3_64bits() is the recommended default. Use XXH3_128bits() when a 128-bit hash is required.

Benchmarks

The benchmarked reference system uses an Intel i7-9700K CPU running Ubuntu x64 20.04. The open source benchmark program is compiled with Clang v10.0 using -O3.

Hash Name	Width	Bandwidth	Small Data Velocity	Comment
XXH3 (AVX2)	64	59.4 GB/s	133.1
XXH128 (AVX2)	128	57.9 GB/s	118.1
XXH3 (SSE2)	64	31.5 GB/s	133.1
XXH128 (SSE2)	128	29.6 GB/s	118.1
memcpy, from RAM	N/A	28.0 GB/s	N/A	for reference
City64	64	22.0 GB/s	76.6
T1ha2	64	22.0 GB/s	99.0	Slightly worse collisions
City128	128	21.7 GB/s	57.7
XXH64	64	19.4 GB/s	71.0
SpookyHash	64	19.3 GB/s	53.2
Mum	64	18.0 GB/s	67.0	Slightly worse collisions
XXH32	32	9.7 GB/s	71.9
City32	32	9.1 GB/s	66.0
Murmur3	32	3.9 GB/s	56.1
SipHash	64	3.0 GB/s	43.2
FNV64	64	1.2 GB/s	62.7	Poor avalanche properties
Blake2	256	1.1 GB/s	5.1	Cryptographic
SHA1	160	0.8 GB/s	5.6	Cryptographic but broken
MD5	128	0.6 GB/s	7.8	Cryptographic but broken

For current measurements on additional platforms, see the benchmarks on xxhash.com.

Note 1: Small data velocity is a rough evaluation of an algorithm's efficiency on small data. For more detailed analysis, see the next section.

Note 2: Some algorithms feature faster than RAM speed. They can only reach their full speed potential when input is already in CPU cache (L3 or better). Otherwise, they are limited by RAM speed.

Small data

Performance on large data is only one part of the picture. Hashing is also very useful in constructions like hash tables and bloom filters. In these use cases, it is common to hash many small inputs, sometimes only a few bytes long. An algorithm's performance can be very different in such scenarios, since initialization and finalization become fixed costs. Branch misprediction also has a much greater impact.

XXH3 has been designed for excellent performance on both long and small inputs, which can be observed in the following graph:

For a more detailed analysis, see the performance comparison on the wiki.

xxHash is not a cryptographic hash function. Do not use it for signatures, password storage, or any other purpose that requires resistance to attacks.

Branch	Status
release
dev

Getting started

The default make target builds both the library and the xxhsum command line utility:

make
./xxhsum -H3 README.md

The library can then be linked as libxxhash, or compiled directly from xxhash.c. For a header-only integration, define XXH_INLINE_ALL before including xxhash.h:

#define XXH_INLINE_ALL
#include "xxhash.h"

For CMake integration, see the CMake guide. The xxhsum manual documents checksum generation, verification, benchmarking and advanced command line options.

xxhash.h is generated from the source units in lib/. It only keeps the documentation of public declarations, which IDEs display. Smaller variants, limited to some hash families, can be generated with make variants, or downloaded from each release (see limited variants). Contributors should modify these units: make regenerates xxhash.h (see AGENTS.md).

Example

The simplest API hashes a contiguous block of memory in a single call:

#include <stddef.h>
#include "xxhash.h"

XXH64_hash_t hash_buffer(const void* buffer, size_t size)
{
    return XXH3_64bits(buffer, size);
}

The API also supports incremental hashing of streams of unknown size. Complete single-shot and streaming examples are provided in the API documentation.

Quality

Speed is not the only property that matters. For non-adversarial inputs, xxHash aims to produce a uniform distribution so that any subset of the output bits can spread entries evenly in a table or index. Like any fixed-width hash, it is still subject to collisions and the birthday paradox.

This is the beginning of the README. Read the rest at the original. Read the full README ↗