English | 简体中文
Refined-X
Your public interface for the agentic web
Publish once for people, search engines, and AI agents.
Refined-X is an opinionated personal publishing starter built with
Astro and Starlight.
Write in Markdown and YAML; Refined-X turns the same public corpus into:
- a calm, editorial website for human readers;
- Markdown mirrors,
llms.txt, and structured JSON for language models;
- OpenAPI and MCP discovery metadata for programmatic clients;
- an optional NLWeb-compatible Ask service for live, grounded Q&A.
Your content stays in your own repository or knowledge vault. The site is
static by default, and the AI service is optional.
Live demo ·
Ask the demo ·
Use this template ·
Production example
Why Refined-X?
Most personal sites publish HTML and stop there. That works for browsers, but
agents must still extract meaning from navigation, layout, and scripts.
Refined-X publishes one source corpus through three surfaces:
| Surface | What it provides |
|---|
| Human-readable | Articles, series, projects, profile, answers, topics, light/dark themes |
| Machine-readable | Per-page Markdown, llms.txt, llms-full.txt, JSON APIs, OpenAPI |
| Agent-queryable | Static curated search, optional NLWeb /ask, optional MCP ask tool |
flowchart LR
A["Markdown + YAML"] --> B["Refined-X build"]
B --> C["Editorial website"]
B --> D["Agent-readable surfaces"]
D --> E["Optional Public Ask"]
What makes it different
One corpus, multiple outputs
Articles, answers, projects, series, and public profile data share an explicit
content schema. Refined-X generates the website and every machine-readable
surface from that same source of truth.
Content independent from the theme
contentRoot, publicDir, and outDir are configurable. Keep a personal
vault or monorepo outside the template, and use Refined-X only as the publishing
layer.
Useful without an AI backend
The default site is fully static. /ask searches curated answers and public
articles without a model, database, or runtime bill.
Live Q&A when you want it
The optional Public Ask Worker adds grounded retrieval and summarization through
a restricted NLWeb v0.55-compatible /ask endpoint and a dual-era Streamable HTTP MCP
server (one ask tool on /mcp for modern 2026-07-28 and legacy 2025 clients).
Offline dual-era integration runs in CI; product-client verification is recorded in
docs/mcp-client-support-matrix.md (core: Claude Code modern + Codex CLI legacy on a synthetic mock backend; other clients stay not_run).
It includes quotas, rate limits, browser verification, source links,
and explicit capability boundaries.
Designed for reading
Agent support is not allowed to turn the site into a dashboard. Refined-X keeps
an editorial, monochrome visual system with restrained motion and accessible
light/dark themes.
Quick start
This repository is a GitHub template. Select
Use this template,
or run:
npm create astro@latest -- --template tower1229/Refined-X
cd <project>
npm install
npm run dev
Then open the local URL printed by Astro.
Before deploying:
npm run check
npm run test:public-ask
npm run test:related
npm run build
npm run verify
Choose a deployment mode
| Mode | Infrastructure | Result |
|---|
| Static | GitHub Pages, Cloudflare Pages, Netlify, or any static host | Website, local Ask search, Markdown, llms.txt, JSON, OpenAPI, discovery metadata |
| Static + external vault | Static host plus an external contentRoot | Same outputs while content remains outside the template |
| Live Ask | Static site plus the reference Cloudflare Worker | Grounded browser answers, NLWeb /ask, MCP ask, health endpoint |
Start static. Add Live Ask only when conversational access is useful.
Deploy static site
| Path | Guide |
|---|
| GitHub Pages | Copy deploy/user-github-pages.yml → enable Pages (GitHub Actions) |
| Cloudflare Pages | Build npm run build, output dist, Node 24 |
Step-by-step: docs/deploy-static.md.
Cloudflare Pages: connect the repo in the dashboard (git integration docs). GitHub Pages: copy the workflow linked above.
Content model
The default public corpus lives in content/:
content/
articles/**/*.md
answers/**/*.md
pages/**/*.md
profile/
person.yaml
cooperation.yaml
resume.md
projects/*.{yaml,yml,json}
series/
series.json
*.yaml
Article frontmatter is intentionally explicit:
---
title: Building for humans and agents
description: A short description for readers and search engines.
contentType: article
pubDate: 2026-07-01
slug: humans-and-agents
series: notes
tags:
- publishing
- agents
llmSummary: A concise, evidence-grounded summary for machine-readable outputs.
---
The schema validates required dates, slugs, summaries, answer fields, and
content types during the build.
Optional seoTitle overrides browser and sharing titles without changing the article H1.
Use seoImage for a deliberate sharing image and updatedDate for actual content changes;
the latter must not precede pubDate.
With assetSource configured, asset collection preserves original URLs and generates
content-addressed WebP variants and dimensions. Markdown images use this manifest for
responsive sources and lazy loading. GIF, animated images, and SVG retain their format.
JPEG candidates use dimensions after EXIF orientation, so width descriptors are unique
and reflect the encoded pixels. Image cache v3 includes platform, architecture, and Sharp/native-codec versions; it bypasses older caches after encoder upgrades.
List previously public originals in retainedAssets when retiring content so a clean
checkout can still rebuild those URLs. Spectral Latin subsets are self-hosted; their
license is in src/assets/fonts/Spectral-OFL.txt.
Before cleanup, collection rejects source/output overlap (including symlinks) and protects declared content and history inputs from deletion. A content root may contain the public directory, but cannot reside inside the generated asset directory.
Each collection rebuilds the generated asset directory; public and dist preview leftovers
are not history. Set REFINED_X_ASSET_HISTORY to an explicit publication snapshot to retain
old derived URLs. Its asset-history.json declares version: 1, sourceRevision, and
files: [{ file, sha256 }]; blobs reside in optimized/ and are verified before restoration.
Hosting adapters obtain snapshots from pushed artifacts, separately from build caches.
Maintain feedUpdatedDate and pageUpdatedDates when feed, listing, or static-page content
changes cannot be represented by article dates alone. Never refresh them merely because
a build ran. Collections, including archive pagination, use the newer of their computed
member date and authored date. RSS discovery is available in the shared head and the footer.
Run npm run test:site for image, link, deployment-base, and date regressions, then
npm run verify against the built output. Hosting adapters can pass resolveRoute to
auditSite in scripts/site-audit.mjs to check deployment redirects before filesystem
fallback. Inline CSS static URLs and data-src resources are checked as well. Hosting
adapters maintain concrete examples for historical dynamic routes. Length heuristics and empty ALT are editorial warnings, not fixed SEO limits.
Metadata names and canonical relation tokens are compared case-insensitively, including copied legacy HTML.
Pagefind output uses its pinned public getFiles() API followed by awaited Node writes
and SHA-256 readback. Both build and verify reject empty or malformed entries,
missing language metadata/WASM, invalid compressed payloads, and missing or changed
bundle files. Deploy the entire pagefind/ directory, including refined-x-manifest.json.
The built-in Ask UI still searches /api/search-index.json; Pagefind is a separate
static index for integrations. npm run test:search exercises 100 real native exports;
npm run test:search-build injects corrupt artifacts into actual Astro builds and
checks both gates. See search integrity notes for the
failure diagnosis and validation limits. The memory export temporarily holds the
whole search bundle in RAM; large sites should measure build memory before upgrading.
Configure your site
Security maintenance: v2.1.1 updates both the site and optional Worker lockfiles.
Run npm ci separately in this directory and examples/public-ask-worker, then
run npm audit in each; CI audits production and development dependencies at all
severity levels. Registry availability and newly disclosed advisories can fail that gate.
The Worker pins MCP SDK 2.3.0 and Wrangler 4.153.0; regenerate Worker types and run
the offline protocol suite before updating an instance. Updating the template does
not deploy an existing Worker. See security upgrade notes
for exact versions, advisory sources and acceptance boundaries.
For most installations, create an instance.config.mjs overlay. You can also
edit the defaults in site.config.mjs:
export default {
site: "https://example.com",
title: "Your Name",
locale: "en",
timeZone: "UTC",
contentRoot: "./content",
publicDir: "./public",
outDir: "./dist",
brand: {
persona: "Your Name",
homeHeading: "Your Name",
homeLede: "What you publish and why it matters.",
},
};
Common options:
| Field | Default | Purpose |
|---|
locale | en | UI language pack: en or zh-CN |
contentRoot | ./content | Public Markdown/YAML corpus |
publicDir | ./public | Static assets |
outDir | ./dist | Build output |
assetSource | unset | Optional external image library |
brand.* | demo values | Public identity and home-page copy |
ask.* | empty | Optional Public Ask / MCP / health URLs; protocolProfile defaults to undeclared (set dual-era only after deployment acceptance) |
discovery.awp | false | Optional AWP 0.2 manifests; when true, emits byte-identical /agent.json and /.well-known/agent.json (phase-1 static reads only); llms does not auto-recommend them; subpath deploy limits — see plan §8.3 |
comments.* | empty | Optional giscus repository and category |
Relative paths resolve from the Refined-X package root.
Enable article comments
Refined-X can load giscus on article pages and keep the
discussion data in a public GitHub Discussions repository. Enable Discussions,
install the giscus GitHub App, create a discussion category, then copy the four
public identifiers from the giscus configuration page:
export default {
comments: {
repo: "owner/repository",
repoId: "R_...",
category: "Comments",
categoryId: "DIC_...",
},
};
Leave all four fields empty to disable comments. A partial configuration fails
the build instead of silently publishing a broken surface. Refined-X uses a
stable article:<entry.id> discussion key, follows the site locale and theme,
and loads the third-party iframe lazily. Visitors need a GitHub account to
participate.
Agent-readable surfaces
Every build exposes a predictable public interface:
| Endpoint | Purpose |
|---|
/llms.txt | Compact site map and important links for agents |
/llms-full.txt | Full public text corpus |
/<page>.md | Clean Markdown mirror of a public page |
/api/profile.json | Structured public identity |
/api/articles.json | Article catalog |
/api/topics.json | Topic catalog |
/api/search-index.json | Static Ask/search corpus |
/openapi.json | API and optional Ask/MCP contract (only declares configured remotes) |
/.well-known/about.json | Site capability summary |
These endpoints make the site easier to ingest and connect. They do not assume
that every agent automatically discovers or invokes them.
Legacy draft MCP discovery paths (/.well-known/mcp.json,
/.well-known/mcp/catalog.json, /.well-known/mcp/server-card.json) were
removed in a breaking change. Prefer the configured MCP endpoint URL,
OpenAPI, and about.json. Do not expect /.well-known/ai-catalog.json.
Enable Live Ask
Optional Cloudflare Worker for grounded browser answers, NLWeb POST /ask, and
MCP ask. Package: examples/public-ask-worker.
Deploy checklist and troubleshooting: docs/deploy-live-ask.md.
After the Worker is up, point the static site at it:
export default {
ask: {
askUrl: "https://ask.example.com/ask",
mcpUrl: "https://ask.example.com/mcp",
healthUrl: "https://ask.example.com/health",
// Optional. Default undeclared — do not claim modern dual-era until accepted.
// protocolProfile: "dual-era",
},
};
Set PUBLIC_TURNSTILE_SITE_KEY at Astro build time when using browser generation.
The hosted demo uses https://ask-demo.refined-x.com/mcp.
Live Ask intentionally does not support long-term memory, arbitrary actions,
elicitation, or impersonating the site owner.
Use an external vault
Refined-X can live as a submodule inside a personal data repository:
git submodule add [email protected]:tower1229/Refined-X.git 90_Website/Template
Place instance.config.mjs next to the submodule, or set
REFINED_X_INSTANCE_CONFIG:
export default {
contentRoot: "../../20_Publish",
publicDir: "../../30_Assets/Public",
outDir: "../../dist",
};
Instance-specific settings stay outside the template, so upstream updates do
not overwrite your identity or content.
Design
See DESIGN.md for the visual system, typography, component
rules, motion boundaries, and accessibility guidance.
Scope
Refined-X is:
- a static-first personal publishing starter;
- an opinionated public content schema;
- a reference implementation for agent-readable and agent-queryable surfaces.
Refined-X is not:
- a hosted CMS;
- a private personal agent;
- a long-term memory service;
- a promise of automatic MCP discovery in every client;
- a claim that every MCP client in the extended matrix has been product-verified (see
docs/mcp-client-support-matrix.md).
Contributing
Issues, implementation reports, documentation improvements, and pull requests
are welcome. If you launch a site with Refined-X, open a showcase issue so it
can be included in the community gallery.
License
MIT
