[[https://github.com/Kiyomi-Computation-Systems/chez-httparse/actions/workflows/ci.yml][https://github.com/Kiyomi-Computation-Systems/chez-httparse/actions/workflows/ci.yml/badge.svg]]
An HTTP/1.x parser for Chez Scheme: a byte-level state machine useful for web
servers reading bytes off the wire. Its based on and checked against [[https://github.com/nodejs/llhttp][llhttp]] by
replaying llhttp's recorded conformance traces and comparing them byte for byte,
at every chunk size.
You could combine this with [[https://github.com/Kiyomi-Computation-Systems/chez-libuv][chez-libuv]] to build an HTTP/1.x server like
[[https://github.com/Kiyomi-Computation-Systems/chez-http-server][chez-http-server]].
This library was generated with [[https://kiyomi-computation-systems.github.io/byte-machine/][byte-machine]] [[https://github.com/Kiyomi-Computation-Systems/byte-machine][(source)]] a Scheme state machine
generator modeled after [[https://github.com/nodejs/llparse][llparse]].
📖 [[https://kiyomi-computation-systems.github.io/chez-httparse/][Read the docs]]
#+begin_quote
✨ AI Disclosure: LLMs are used extensively in the planning and development of this project.
#+end_quote
- A state machine, not a reader. You feed it bytes which trigger callbacks.
It keeps no buffer, copies no spans, and allocates nothing in proportion to
the input: a 10 KB body costs what a 10-byte one does.
- Resumable. A token, header or body split across any number of feeds
parses exactly as one whole buffer does.
- Conformant. It passes all 274 fixtures in llhttp's corpus, every HTTP and
URL fixture upstream ships, at every chunk size from 1 to the input length.
A live differential against a built llhttp agrees on 9,557 generated inputs
as well. See [[file:docs/conformance.md][docs/conformance.md]].
- Generated and committed. =src/generated/httparse-message.sls= is emitted
from =src/httparse/grammar.sls= and checked in, so the shipped parser is
reviewable as a diff and loads without running the generator. A reference
interpreter runs beside it over every fixture and must agree.
- Pure Scheme. No C and no FFI. The only runtime dependency is the
generator's runtime library, vendored at =vendor/byte-machine=.
- The message parser, =(generated httparse-message)=. Choose request or
response, and any of llhttp's eleven lenient flags, when you make the
parser. Pauses, =skip-body= and llhttp's two upgrade hand-offs come back as
results. =parser-finish!= ends a parse as =llhttp_finish= does: a
close-delimited body completes, and a half-read message is
=invalid-eof-state=. Inside a callback, =parser-stream-offset= gives the
callback's position in the stream.
- The URL parser, =(generated httparse-url)=. Splits a request target into
schema, host, path, query and fragment spans.
- Three pure libraries with no runtime dependency: =(httparse modes)= (mode
tables), =(httparse methods)= (llhttp's method names and numbers) and
=(httparse decisions)= (llhttp's keep-alive and needs-EOF rules, and the
flag bits they read).
#+begin_src scheme
(import (generated httparse-message)
(httparse modes)
(byte-machine runtime))
(define parser
(make-parser
httparse-message-machine
(list
;; A span callback gets bounds into YOUR bytevector. Nothing is copied
;; and nothing is retained -- take what you need before returning.
(list 'on-method-data
(lambda (p ctx bv from to) (remember-method bv from to) callback-continue))
(list 'on-header-field-data
(lambda (p ctx bv from to) (remember-field bv from to) callback-continue))
;; ... the remaining callbacks. Every declared one must be bound: a
;; binding list with a hole is refused when the parser is made, not
;; quietly ignored.
)
#f
;; The mode, chosen before the first byte. Nothing in the bytes says
;; whether a request or a response is coming, and the embedder always
;; knows; the second row is llhttp's lenient bitset, and omitting it
;; parses strictly.
(list (cons 'type (httparse-message-type-value 'request))
(cons 'lenient-flags (httparse-lenient-flags-value '())))))
(let ((r (parser-feed! parser chunk 0 (bytevector-length chunk))))
(case (parser-result-status r)
((need-input) 'feed-more)
((error) (parser-error-reason (parser-result-error r)))
(else (parser-result-status r))))
#+end_src
The complete, runnable version is
[[file:test/common/example-tests.sps][test/common/example-tests.sps]], which the
test suite runs, so this example cannot drift from the library.
[[file:docs/usage.md][docs/usage.md]] walks through it, and
[[file:docs/reference.md][docs/reference.md]] lists every procedure.
- Chez Scheme 10.4.1 or newer. Discovery tries =scheme=, =chez= and
=chezscheme=, and rejects Petite.
- CMake 3.26 or newer. CI runs 3.31; development uses 4.x.
- The generator submodule: clone with =--recursive=, or run
=git submodule update --init=.
The CMake-layer specifications also run with no Scheme installed, through the
=no-scheme= preset. The optional live-llhttp differential needs Node, a C
toolchain and a built llhttp checkout; see
[[file:docs/building.md][docs/building.md]].
=make help= lists every target. The main ones:
| =make build= | configure and build the development preset |
| =make dev= | build, then run the harness self-tests |
| =make test= | build, then run the pull-request selection |
| =make test-all= | build, then run the complete selection |
| =make clean= | remove all build trees |
Each target wraps a checked-in CMake preset, so =cmake --preset dev= and
=ctest --preset pr= work the same way without =make=, including on Windows.
[[file:docs/building.md][docs/building.md]] covers the presets, the lanes and the
opt-in checks.
The libraries are pure R6RS. Nothing under =src/= imports =(chezscheme)=:
the grammars import =(rnrs)= and =(byte-machine graph)=, and the generated
parsers import =(rnrs)= and =(byte-machine runtime)= only. The generator's
own =src/= is the same. A port to another R6RS system is a tooling job, not a
library one.
Chez-specific code is confined to build and test time:
- the test harness, through nine named imports: =machine-type=,
=scheme-version=, =getenv=, =system=, =statistics=, =sstats-bytes=,
=real-time=, =with-output-to-string= and =display-condition=
- =tools/emit-machine.ss=, for command-line arguments, file output and =exit=
- =cmake/HttparseChez.cmake=, which finds the executable
Most of those imports have a direct equivalent elsewhere. =statistics= and
=sstats-bytes= do not: allocation accounting has no portable spelling, so the
allocation lane is Chez-only.
Expect two differences on another system. Library resolution differs (Racket
wants =#lang r6rs=, Guile has its own search rules). And the generated parser
is a large set of mutually recursive definitions that relies on proper tail
calls, a shape Chez compiles well, so the figures in
[[file:docs/performance.md][docs/performance.md]] would need re-measuring.
Every Scheme lane takes a =BACKEND= argument (=cmake/HttparseSchemeLane.cmake=),
which is where a second runtime would attach.
No port has been attempted; portability here is by construction, not by
measurement.
Start with installing and usage; the rest is by task.
- [[file:docs/installing.md][Installing]]: with Akku, or putting the libraries on Chez's library path
- [[file:docs/usage.md][Usage]]: making a parser, feeding it, reading results, its lifecycle
- [[file:docs/callbacks.md][Callbacks]]: the 25 callbacks, spans and stream offsets
- [[file:docs/modes.md][Modes]]: request or response, and llhttp's lenient flags
- [[file:docs/errors.md][Errors]]: reading an error and mapping it to a response
- [[file:docs/pausing.md][Pausing and upgrades]]: pausing, resuming, and the upgrade hand-offs
- [[file:docs/connection.md][Parser state and the connection]]: parser fields, keep-alive or close
- [[file:docs/url.md][The URL parser]]: splitting a request target
- [[file:docs/reference.md][API reference]]: every documented binding
- [[file:docs/conformance.md][Conformance]]: what is checked against llhttp, and how
- [[file:docs/performance.md][Performance]]: allocation and throughput, measured
- [[file:docs/building.md][Building and testing]]: presets, lanes and the opt-in checks
- [[https://github.com/Kiyomi-Computation-Systems/chez-httparse/tree/main/.plans/decisions][Design decision records]]: why the code is the way it is (in the repository, not in release archives)
- =CHANGELOG.md=: release history
[[file:CONTRIBUTING.md][CONTRIBUTING.md]] says how to set up, which checks a pull request must pass,
and the house rules.
Report a vulnerability privately; [[file:SECURITY.md][SECURITY.md]] says how.
© [[https://kiyomicomputation.systems/][Kiyomi Computation Systems LLC]]. BSD 3-Clause; see [[file:LICENSE][LICENSE]].
The parser is a port of llhttp's, and its grammars, method table and error
reasons derive from llhttp (MIT, © 2018 Fedor Indutny). llhttp's
conformance corpus is vendored under =test/vendor/llhttp/= as test data and is
not part of the library. [[file:NOTICE][NOTICE]] has the attributions and
llhttp's license text.
