uriparse

• uriparse

[[https://github.com/Kiyomi-Computation-Systems/uriparse/actions/workflows/ci.yml][https://github.com/Kiyomi-Computation-Systems/uriparse/actions/workflows/ci.yml/badge.svg?branch=main]]

A URI parser for Chez Scheme, written in portable R6RS. It follows [[https://www.rfc-editor.org/rfc/rfc3986][RFC 3986]].

It's built with a byte-level state machine useful for web servers reading bytes off the wire. It also includes an ergonomic library for reading URIs as strings and parsing them into a record of components.

You could combine this with [[https://github.com/Kiyomi-Computation-Systems/chez-libuv][chez-libuv]] and [[https://github.com/Kiyomi-Computation-Systems/chez-httparse][chez-httparse]] to build an HTTP/1.x server.

This library was generated with [[https://kiyomi-computation-systems.github.io/byte-machine/][byte-machine]] [[https://github.com/Kiyomi-Computation-Systems/byte-machine][(source)]] a Scheme state machine generator modeled after [[https://github.com/nodejs/llparse][llparse]].

📖 [[https://kiyomi-computation-systems.github.io/uriparse/][Read the docs]]

#+begin_quote ✨ AI Disclosure: LLMs are used extensively in the planning and development of this project. #+end_quote

** Why

uriparse is built for servers that read URIs straight off the wire.

• Bytes in, bytes out. A URI is parsed as a bytevector, not a string, so nothing is decoded or guessed before the parse.
• Streaming. Input can arrive in any number of chunks, and the result is the same however it was split.
• Reusable. One collector can serve every request on a connection. Resetting it reallocates nothing.
• No copies to validate. A check can run over a range of your own buffer in place.
• Lenient parsing, strict validation, kept apart. uri-parse splits whatever it can. uri-parse-validate rejects exactly what the RFC grammar rejects. You pick the one you need.
• Absent is not empty. http://a has no query and http://a? has an empty one, and the results say so: #f for absent, an empty bytevector for empty.

If you just have a URI in a string, you don't need any of that: (uri easy) takes strings and gives back strings.

** What it does

• Read URIs as strings. (uri easy) parses a string into a record and answers with strings: the host, the port as a number, decoded path segments and query pairs. It also resolves, normalizes and compares.
• Split queries. (uri query) reads the key=value&… pairs out of a query. It splits first and decodes after, so an encoded & stays in its value.
• Parse. Split a URI into scheme, authority, path, query and fragment. Three modes: an absolute URI (uri), a relative reference (relative-ref), or either (uri-reference).
• Validate. Check each component's characters, plus the authority's userinfo, host (registered name, IPv4, IPv6 or IPvFuture) and port.
• Resolve. Resolve a relative reference against a base URI, including removing . and .. segments, and turn components back into a URI. Every resolution example in RFC 3986 passes.
• Normalize. Lowercase the scheme and host, uppercase percent-encoding hex, and decode percent-encoded characters that never needed encoding.
• Parse CONNECT targets. Read the bare host:port form an HTTP CONNECT request uses (RFC 9112's authority-form).

** Using it

Install it with [[https://akkuscm.org/packages/chez-uri/][Akku]], which also fetches [[https://github.com/Kiyomi-Computation-Systems/byte-machine][byte-machine]], the runtime it depends on:

#+begin_src sh akku update && akku install chez-uri #+end_src

Or clone it. The libraries live in src/, and byte-machine is a git submodule in vendor/byte-machine:

#+begin_src sh git clone --recurse-submodules https://github.com/Kiyomi-Computation-Systems/uriparse.git #+end_src

[[file:docs/installing.md][docs/installing.md]] has both ways in full.

Read a URI from a string:

#+begin_src scheme (import (rnrs) (uri easy))

(define u (string->uri "https://[email protected]:8443/a/caf%C3%A9?q=a%26b&tag#top"))

(for-each (lambda (x) (write x) (newline)) (list (uri-host u) (uri-port u) (uri-path-segments u) (uri-query-pairs u) (uri->string (uri-resolve u "../x?y")))) #+end_src

#+begin_example "example.com" 8443 ("a" "café") (("q" . "a&b") ("tag" . #f)) "https://[email protected]:8443/x?y" #+end_example

[[file:docs/easy.md][docs/easy.md]] covers the rest of (uri easy), and [[file:docs/README.md][docs/]] has a guide to each part of the library and an [[file:docs/reference.md][API reference]].

A server works in bytes. Parse a URI and read its components:

#+begin_src scheme (import (rnrs) (uri collect) (uri components))

;; The parser works on bytes, so the input is UTF-8. (define result (uri-parse 'uri (string->utf8 "https://example.com/path?q=1#top")))

;; Print one component, or "" when the URI has none. (define (show name bv) (display name) (display ": ") (display (if bv (utf8->string bv) "")) (newline))

(when (uri-parse-ok? result) (let ((c (uri-parse-components result))) (show "scheme" (uri-components-scheme c)) (show "authority" (uri-components-authority c)) (show "path" (uri-components-path c)) (show "query" (uri-components-query c)) (show "fragment" (uri-components-fragment c)))) #+end_src

#+begin_src sh scheme --libdirs "src:vendor/byte-machine/src" --script parse.scm #+end_src

#+begin_example scheme: https authority: example.com path: /path query: q=1 fragment: top #+end_example

Where to go next:

| To… | use | |--------------------------------------------+-----------------------------------------------------------| | read a URI you have as a string | (uri easy), see [[file:docs/easy.md][docs/easy.md]] | | split a query into key/value pairs | query-pairs, form-pairs in (uri query) | | parse input that arrives in chunks | make-uri-collector, uri-collect-feed!, uri-collect-finish! | | reuse a collector for the next request | uri-collector-reset! | | parse and validate in one call | uri-parse-validate | | validate a result you already have | uri-parse-result-validate | | validate one component in your own buffer | validate-component in (uri charset) | | validate an authority in your own buffer | validate-authority, authority-host-span in (uri host) | | parse a CONNECT target | uri-parse-authority-form, uri-parse-authority-form-validate | | resolve a reference against a base | (uri resolve) | | normalize | (uri normalize) | | turn components back into a URI | (uri recompose) |

Each exported procedure carries a type signature in a comment above its definition, and has an entry in [[file:docs/reference.md][docs/reference.md]].

** Scope and limits

• uri-parse is deliberately lenient: http://a/b c parses. Run uri-parse-validate when you need the input to be a valid URI; it rejects that one with invalid-pchar.
• Validation accepts exactly the inputs the RFC 3986 grammar accepts. This is tested against an exhaustive sweep of short inputs, the RFC's own examples, and a hand-written adversarial set.
• (uri normalize) does not remove . and .. segments; resolution does. If you want both, normalize first and then resolve. (uri easy)'s uri-normalize does both, in that order.
• (uri easy) accepts only ASCII, as RFC 3986 does: encode text with uri-encode before putting it in a URI. Its decoding must produce UTF-8: %FF raises invalid-utf-8 instead of becoming �.
• Resolution is strict: http:g resolved against http://a/b/c/d;p?q stays http:g. The older, looser behaviour the RFC mentions is not supported.
• Recomposing gives back what was parsed. It does not repair components built by hand into something ambiguous, such as a scheme-less path whose first segment contains :. Resolution can produce one such shape too: against a base with no authority, foo:/a and .//bar resolve to the path //bar, which recomposes to foo://bar, a URI whose authority is bar. (uri easy) writes that path as /.//bar instead, which names the same path and parses back unchanged.
• Only Chez Scheme is tested. CI runs on Linux amd64; development happens on macOS. Windows and arm64 are untested.

** Developing

*** Requirements

• Chez Scheme 10.4.1 or newer, the full system rather than Petite (the compiled test lanes need compile-program)
• CMake 3.26 or newer

The build looks for scheme, chez and chezscheme, in that order. To point it somewhere else:

#+begin_src sh cmake --preset dev -DURI_CHEZ_EXECUTABLE=/absolute/path/to/scheme #+end_src

*** Building and testing

#+begin_src sh make # list every target make test # the pull-request selection make test-all # every test make no-scheme # the CMake layer alone, on a machine with no Scheme make check-akku # install with Akku as a consumer would (needs akku) make check-help # every target is described, and the contributor docs name real ones #+end_src

The Scheme examples in this README and in docs/ are run by the tests, so they print what they say they print. Another test holds docs/reference.md to the libraries' exports: a new export needs an entry.

*** What is distributed

Akku.manifest describes the package for the [[https://akkuscm.org/][Akku]] index. The Akku package and a GitHub source archive are both built the way git archive builds them, so they leave out what .gitattributes marks export-ignore: .plans/, .superpowers/ and AGENTS.md. Those are the design records and agent instructions, kept here for working on uriparse, not for using it.

From the package, Akku installs only the libraries in src/. .akkuignore keeps the docs site generator's site/ libraries off a consumer's library path.

make check-akku checks all three: what the package holds, which libraries install, and that a program reads a URI with only Akku's library path.

*** The docs site

docs/ is also published as a website, rendered by [[https://github.com/Kiyomi-Computation-Systems/chez-cmark-gfm][chez-cmark-gfm]] 2.0.0. It is not a dependency of the library, so you fetch it yourself and point CMARK_GFM_ROOT at it. It needs the cmark-gfm shared library installed (brew install cmark-gfm, apt install cmark-gfm).

Get it from [[https://akkuscm.org/packages/chez-cmark-gfm/][Akku]], in any directory outside this repository:

#+begin_src sh akku update && akku install chez-cmark-gfm #+end_src

or clone it at the tag CI uses. Clone on an Apple silicon Mac: Akku 1.1.0 has a known crash there (akku issue #69).

#+begin_src sh git clone --branch v2.0.0 https://github.com/Kiyomi-Computation-Systems/chez-cmark-gfm.git #+end_src

Then, with CMARK_GFM_ROOT naming that directory:

#+begin_src sh make check-site CMARK_GFM_ROOT=/path/to/it # links, anchors, nav make site CMARK_GFM_ROOT=/path/to/it # build/site/index.html #+end_src

docs/README.md becomes the home page. A new page goes in the nav in site/pages.sls; make check-site fails until it is there. The site is published to GitHub Pages by .github/workflows/pages.yml when a v* tag is pushed.

*** Generated files

Two kinds of file are generated, committed, and checked by a test that regenerates them and compares. Edit the source, not the output:

• src/generated/uri-scanner.sls comes from (uri grammar). Run make parser-refresh after changing the grammar.
• test/conformance/rfc3986/ is extracted from the RFC text in test/vendor/. Run make corpus-refresh after changing the extractor.

*** Layout

| src/uri/ | the libraries | | src/generated/ | the generated scanner | | tools/ | the program that writes the generated scanner | | docs/ | guides for users, and the API reference | | site/ | the docs site generator; build-site.sps runs it | | cmake/ | build modules: Chez discovery, test registration, corpus extraction | | test/common/ | the test programs | | test/conformance/ | test data: RFC examples and hand-written negative cases | | test/site/ | the docs site generator's tests and make check-site | | test/support/ | the test harness and the build-layer specifications | | test/akku/ | the Akku install check and the program it runs | | vendor/ | the byte-machine submodule | | .plans/ | plans and decision records; not distributed | | .superpowers/ | the development ledger; not distributed |

Release history is in [[file:CHANGELOG.md][CHANGELOG.md]].

** Contributing

[[file:CONTRIBUTING.md][CONTRIBUTING.md]] says how to set up, which checks a pull request must pass, and the house rules.

** Security

Report a vulnerability privately; [[file:SECURITY.md][SECURITY.md]] says how.

** License

© [[https://kiyomicomputation.systems/][Kiyomi Computation Systems LLC]]. Licensed under BSD 3-Clause — see [[file:LICENSE][LICENSE]]. Third-party material, including the RFC 3986 text used by the tests, is listed in [[file:NOTICE][NOTICE]].