🚀 Web Frameworks Benchmark

A community-maintained, reproducible comparison of backend web frameworks

Explore the results · Add a framework · Report a problem · Join the discussion

From established full-stack platforms to small HTTP routers, this repository puts hundreds of implementations behind the same tiny API and runs them in isolated containers. Today, it makes a focused part of web-stack performance easier to measure, reproduce, and discuss. The longer-term goal is a comprehensive, evidence-based guide that helps businesses choose a backend framework—not merely a leaderboard that declares one universal winner.

[!IMPORTANT] These are minimal HTTP throughput and latency tests, not a substitute for profiling a production application. Framework features, maintainability, ecosystem, security, database access, and your real workload all matter.

🧪 Benchmark contract

Every implementation is expected to listen on port 3000 and provide the same responses:

Method	Route	Expected status	Expected body
GET	/	2xx	Empty
GET	/user/:id	2xx	The id path parameter
POST	/user	2xx	Empty

Before an implementation is benchmarked, the shared RSpec contract verifies these routes. The default generated benchmark runs GET / for 15 seconds using zrk --closed and keep-alive, and records a JSON report. Concurrency and routes are configurable.

Collected fields include requests per second, total data received, run duration, and the p50, p75, p90, and p99 latency percentiles. Benchmark configuration is generated from three YAML layers:

config.yaml                  # provider and global settings
<language>/config.yaml       # language image, runtime, and engine settings
<language>/<framework>/config.yaml
                             # framework version, files, command, and overrides

📊 Results

The latest published results are available on the Web Frameworks Benchmark dashboard. Each result should be read in the context of its framework version, runtime or server variant, concurrency, benchmark revision, and hardware.

For a fair interpretation:

• compare runs produced by the same benchmark revision and machine;
• check the selected runtime/server variant, not only the framework name;
• treat small differences as noise until repeated;
• do not extrapolate this minimal API test to database-heavy or full application workloads.

🗺️ Roadmap

The project is evolving from a focused HTTP performance benchmark into a broader decision-support guide for organizations choosing a backend framework. The intended presentation is inspired by the clarity of quadrant-based market analysis, but the evaluation model will be designed specifically for backend frameworks and backed by transparent, reproducible evidence. This project is independent and is not affiliated with or endorsed by Gartner.

• ⚡ Use a modern load generator, please follow discussion in https://github.com/the-benchmarker/web-frameworks/discussions/8088
• 🔧 Optimize implementations, please follow discussion in https://github.com/the-benchmarker/web-frameworks/discussions/8093
• 🏗️ Build a real infrastructure using an public cloud
• ⚖️ Check HTTP compliance for each framework here
• 🧭 Build a framework decision model
• 💼 Publish a practical business guide

🏃 Run an implementation locally

📋 Requirements

• Git
• Docker
• Ruby and Bundler (CI uses Ruby 4)
• zrk on PATH
• jq

Install the Ruby dependencies and generate the Dockerfiles and Makefiles:

git clone https://github.com/the-benchmarker/web-frameworks.git
cd web-frameworks
bundle install
bundle exec rake config

Run the helper tests and check language version metadata with:

bundle exec rspec .spec/helpers_spec.rb
ruby .github/scripts/sync-language-versions.rb --check

Then choose an implementation—for example, javascript/fastify—and use its generated Makefile:

make -f javascript/fastify/.Makefile build
make -f javascript/fastify/.Makefile test
make -f javascript/fastify/.Makefile warmup
mkdir -p javascript/fastify/.results/10
make -f javascript/fastify/.Makefile collect
make -f javascript/fastify/.Makefile unbuild

Results are written beneath <language>/<framework>/.results/. Generated targets also include clean for removing the framework container.

🎛️ Customize a run

The generator accepts comma-separated concurrency levels and routes. Set them while generating the Makefiles:

CONCURRENCIES=64,256,512 \
ROUTES='GET:/,GET:/user/42,POST:/user' \
bundle exec rake config

Then run the same collect target. Route entries use the METHOD:/path form.

The load generator runs closed-loop over keep-alive connections: every connection sends its next request as soon as the previous response arrives, so the reported rate is the framework's sustained throughput for the whole run. Three more variables control where that work runs:

This is the beginning of the README. Read the rest at the original. Read the full README ↗