# auctionhouse.fyi CDN v1 The files auctionhouse.fyi publishes on its CDN, and the live messages that point at them. Design: `docs/design/2026-09-29-delivery-design.md` ("push a pointer, pull from the CDN"). Fixtures: `protocol/cdn-fixtures/`. The pure rules (paths, the capability rule, the gap and checkpoint logic, file schemas) are `shared/src/cdn.ts`, used by the Worker, the web and the desktop app. Change this file and its fixtures first, then every implementation together (AGENTS.md rule 10). ## Hosts and bases | Deployment | API | CDN base (`cdnBase`) | |---|---|---| | production | `https://auctionhouse.fyi` | `https://cdn.auctionhouse.fyi/` (R2 `ahf-data`) | | dev | `https://dev.auctionhouse.fyi` | `https://cdn-dev.auctionhouse.fyi/` (R2 `ahf-data-dev`) | | PR preview `pr-` | the Preview URL | `https://cdn-dev.auctionhouse.fyi/pr//` (the preview's `AHF_R2_PREFIX`) | `GET /api/v1/config` answers `cdnBase` (api-v1 "Environment"). Every **path** below is relative to the CDN base (no leading slash); the R2 key is `AHF_R2_PREFIX + path`. Inside CDN files, references are paths; in API answers and live messages they are absolute **URLs** (`cdnBase + path`). The CDN is R2 behind a custom domain with a Cache Everything rule, Smart Tiered Cache and CORS for the deployment's origins (ops). Every file is JSON; `.json.gz` files are stored with `Content-Encoding: gzip`, so browsers and `fetch` decompress them. ## Caching - **Immutable files** (their path names their version): `Cache-Control: public, max-age=31536000, immutable`, set as the R2 object's `cacheControl`. They are written once and never rewritten. - **Pointers**: small files that name the newest immutable file: `Cache-Control: public, max-age=` as the table says. - A client asks only for files it was told exist (from a pointer, a live message or `GET /api/v1/ah/:market/files`), never a guessed future one: a 404 would be cached at the edge. ## The capability rule A file whose content needs a feature (or a limit) the anonymous role doesn't have carries a **capability** `` in its path: 22 base64url characters, 128 random bits from a CSPRNG, made by the market object and never derivable from anything public. Its scope is the table's; one cap per scope. A caller learns a cap only through an authenticated channel, and only when its roles grant what the file needs (`shared/src/entitlements.ts`): | File | Needs | Cap scope | |---|---|---| | live changes, checkpoints | `prices.live` | one file | | minute candle blocks | `candles.minute` | (market, `m1`, UTC day of the block's start) | | hour candle blocks older than 7 days (the capped copies) | `limit(historyDays)` above the anonymous role's | (market, `h1`, UTC day of the hour) | | full day candles (90 days) | `limit(historyDays)` above the anonymous role's | one file | | order books | `books` | one file | | volume estimates | `depth` | one file | Nothing an anonymous caller may see sits behind a cap (snapshots, public day candles, item data, icons). The CDN serves a capability file to whoever has the URL: sharing a URL shares the data, as sharing the data itself would; the files expire (Retention), so shared URLs die. ## Files `market` is a market id (api-v1 Conventions); `hour` = `floor(ms / 3600000)`, `day` = `floor(ms / 86400000)`, `block` = `floor(ms / 600000)` (10 minutes); keys are item keys; `AuctionRow` and candles are api-v1's (`[t, open, high, low, close, quantity]`, `t` Unix ms of the candle's start). Lists are ascending by key, candles ascending by `t`. **Market depth is paid** (api-v1 "Market depth is paid"): public files (snapshots and their pointers, the public hour blocks and 7-day day candles, item data) carry overall quantities but no volume estimates and nothing from order books; those appear only in capped files. ### Hourly snapshot (public) - `snap/v1//.json.gz` (immutable): the hourly snapshot as api-v1 "Hourly snapshot" defines it, as of `hour`'s start. Written by the hourly cron (minute 1). - `snap/v1//latest.json` (pointer, 60 s): `{v: 1, market, hour, at, path}`: `at` = the hour's start (Unix ms), `path` the immutable file. ### Recent changes (public) What home, lists and item cards need without fetching candle blocks: each item's price, quantity, 1-hour, 24-hour and 7-day change and a 7-day sparkline, as of the hour. - `recent/v1//.json.gz` (immutable): `{v: 1, market, at, items: [key, price | null, quantity, ch1h, ch24h, ch7d, spark][]}`, written by the hourly cron with the snapshot. `at` is the hour's start; `price` and `quantity` the row as of `at` (the snapshot's). A change is `(price − then) / then`, rounded to 4 decimals, where `then` is the close of the item's newest hour candle that started more than 1 hour, 24 hours, or 7 days before `at` (null when either is missing). `spark` is 28 closes, one per 6 hours over the 7 days before `at` (the newest close up to the end of each window; null before the item's first). Only keys with a row. - The file also carries hourly index series for the 7 days before `at` (the home page's 1D and 7D charts): `index: [hour, value][]`, the market index, and `categories: {herbs, ore, cloth, leather, enchanting, elemental, consumables, gear: [hour, value][]}`, each from hour `H − 168` (value 100) to `H − 1` (the last finished hour; `H` = `at`'s hour), values rounded to 3 decimals. The rule is wowzers' market index (`shared/src/marketindex.ts` `chainIndex`, zod-free; fixture `protocol/cdn-fixtures/index.json`), run on hour closes: chain-linked, each hour the weighted mean of the basket's log returns (each clipped to ±ln 3) against the hour before, a price carried from up to 72 hours back; weights √(fair × sold per day), the quantity listed (mean over the window's candles) when there's no volume, `fair` the lower median of the newest 7 day closes before `at`'s day (else the price); the basket is the 50 heaviest items (20 per category) with at least 5 hour candles in the window. Categories by the item data's class and subclass (herbs 7/9, ore 7/7, cloth 7/5, leather 7/6, enchanting 7/12, elemental 7/10, consumables class 0, gear classes 2 and 4). Only the index values are public: no weights, volumes or item lists. - `recent/v1//latest.json` (pointer, 60 s): `{v: 1, market, hour, at, path}`. ### Item data (public) - `items/v1//items.json.gz`, `…/search.json.gz`: the generated item data and search index (api-v1 "Item data"), written by the generator. - `items/v1/-/items.json.gz`, `…/search.json.gz` (immutable): the same merged with the catalogue; `hash` = the first 16 hex digits of SHA-256 of the merged `items.json` text. Written by the catalogue merge (at most every 10 minutes after a name changed). - `items/v1/current.json` (pointer, 300 s): `{v: 1, build, hash: string | null, items, search, at}`: `items` and `search` are paths; `hash` null for the generator's own files. ### Icons (public) `icons/.webp`, `icons/id/.webp` (immutable). ### Live changes and checkpoints (`prices.live`) Every merge of a market (a scan or a book, `disagrees` ones aside; api-v1 "Uploads") gets the next **sequence number** `seq` of that market (1, 2, 3, …; it never repeats or goes back). - `live/v1//-.json.gz` (immutable): `{v: 1, market, seq, at, rows}`: `at` the market's newest scan after the merge; `rows` every row the merge changed (its own items, and the rows a complete scan marked gone), as `AuctionRow`s. - `live/v1//c-.json.gz` (immutable): a **checkpoint**: `{v: 1, market, seq, at, rows}` with the whole table after merge `seq`. Written after merge 1 and after every merge whose `seq` is a multiple of 50 (`CHECKPOINT_EVERY`), so the newest checkpoint is at most 49 merges behind. A client's table is the checkpoint's rows plus each later change file applied in `seq` order, a row replacing its key's row when there is none or its `at` is ≥ the stored one's (api-v1 "Market rules" › AuctionRow and merge, the client rule). ### Candle blocks Written by the market object once the period is over plus a grace of 5 minutes (`SEAL_GRACE_MS`; late uploads have that long to count), for every period from the market's first merge on, empty or not. A written period is **sealed**: a scan arriving later for it changes the store and the API's answers, not the file. The newest sealed period of each resolution is `sealed: {m1: block, h1: hour, d1: day}` in `hello`, `sealed` messages and `GET /api/v1/ah/:market/files`. Chart windows (api-v1 "Chart windows"): everyone gets hourly candles of finished hours for the last 7 days (the anonymous role's `historyDays`, `PUBLIC_DAYS`) and 7 days of day candles; minute candles need `candles.minute`, and older hours and 90 days of day candles need the higher `historyDays`. - `candles/v1/m1///.json.gz` (`candles.minute`, immutable): `{v: 1, market, res: "m1", t: × 600000, items: [key, candles][]}`: the minute candles of the 10 minutes of `block`, per item with any. - `candles/v1/h1//.json.gz` (public, immutable): `{v: 1, market, res: "h1", t: × 3600000, items: [key, [candle]][]}`: the finished hour's candle per item with one. Kept 8 days, so the last 7 days of hours are public. - `candles/v1/h1c///.json.gz` (`historyDays` above 7, immutable): the same hour, capped, kept 15 days (hour candles' retention is 14 days): what paid clients read for hours older than 7 days. - `candles/v1/d1//.json.gz` (public, immutable): `{v: 1, market, res: "d1", t: × 86400000, days: 7, items: [key, candles][]}`: every item's day candles of the 7 days `day − 6 … day`. - `candles/v1/d1c///.json.gz` (`historyDays` above 7, immutable): the same with `days: 90` (`day − 89 … day`). The cap of an `m1` block or a capped `h1` copy is the cap of (market, resolution, the UTC day of the block's start): a client that knows a day's cap can build every path of that day. The market object makes each day's caps when the day begins. ### Volume estimates (`depth`) - `volume/v1///.json.gz` (immutable): `{v: 1, market, t: × 86400000, days: 90, items: [key, [day, listed, sold, expired][]][]}`: every item's volume estimates (api-v1 "Market rules" › Order books) of the 90 days up to `day`, written when the day is sealed. Its URL is `caps.volume`. ### Order books (`books`) - `books/v1//-.json.gz` (immutable): the stored book, the upload's body (`{market, at, complete, auctions, entries}`; api-v1 "Market rules" › Order books). Its URL comes with the `ah` notice of its merge and from `GET /api/v1/ah/:market/books` and `…/files`. ## Retention R2 lifecycle rules (by prefix, in whole days; ops) delete: `live/` after 3 days (72 h), `books/` after 30 days, `candles/v1/m1/` after 3 days, `candles/v1/h1/` after 8 days, `candles/v1/h1c/` after 15 days, `candles/v1/d1/` and `candles/v1/d1c/` after 2 days (each day file holds the whole history), `snap/` and `recent/` after 8 days (the cron also deletes hourly snapshot copies after 7 days), `volume/` after 2 days. Pointers are rewritten long before. Previews' files live under `pr//` and go with the preview (the janitor). ## Live messages `GET /api/v1/live?market=` upgrades to a WebSocket for **one market** (api-v1 "Live"): the Worker checks the credential once, then hands the socket to a hub object of that market (`hub::`, at most 2,000 sockets each). To follow several markets, open several sockets. The server sends: - `{t: "hello", market, seq, at, checkpoint: {seq, url} | null, changes: [seq, url][], sealed: {m1, h1, d1}, caps: Caps, roles, features, now}` first: the newest `seq` (0 before the first merge) and its scan `at`, the newest checkpoint and every change after it (with `prices.live`; else `checkpoint` null and `changes` empty), the newest sealed periods, and the caller's caps. - `{t: "ah", market, seq, url, at, book?: url}` after every merge: `url` the change file (with `prices.live`; the socket requires `live`, which comes with it today), `book` the book file when the merge was a book (with `books`). - `{t: "sealed", market, res: "m1" | "h1" | "d1", start, url}` when a candle file is written (`start` the period's start, Unix ms): `url` the file the caller may read (`h1`: the public copy; `d1`: the full one with the `historyDays` limit above 7, else the public one; `m1` only with `candles.minute`, else not sent). - `{t: "alert", alert}` when one of the caller's alerts on this market fires. - `{t: "roles", roles, features}` when the caller's features change; `{t: "auth", until}`, `{t: "error", error}` as api-v1 says. **Caps:** `{m1?: {[day]: cap}, h1?: {[day]: cap}, d1?: url, volume?: url}`: `volume` the newest volume file's URL, with `depth`; `m1` for the 3 retained days (today's included) with `candles.minute`; `h1` (the capped copies) for the 15 retained days and `d1` (the newest full day file's URL) with the `historyDays` limit above 7. An anonymous or free caller has `{}`: it reads the public `h1` and `d1` files, whose paths need no cap. ## Catching up (the gap rule; `shared/src/cdn.ts` `catchUp`, fixture `gap.json`) A client holds `have` (the `seq` its table is at, or null for none), knows some change URLs by `seq` and maybe a checkpoint, and learns a `target` seq (from `hello` or an `ah` notice). `catchUp(have, target, known, checkpoint, askedFiles)` answers the next step (`askedFiles`: whether the client already asked `…/files` for this target): 1. `have ≥ target`: `{op: "none"}`. 2. `have` isn't null, `target − have ≤ 50`, and every seq `have + 1 … target` is known: `{op: "changes", urls}` in seq order. 3. Else a checkpoint with `seq ≤ target` and (`have` null or `checkpoint.seq > have`) whose following seqs up to `target` are all known: `{op: "checkpoint", url, then: urls}`. 4. Else, when `askedFiles` is false, `{op: "files"}`: ask `GET /api/v1/ah/:market/files` for the newest checkpoint and changes, then run `catchUp` again with `askedFiles` true; when it is true, `{op: "table"}`: load `GET /api/v1/ah/:market` (the API; its answer has `seq`) and set `have` to that `seq`. A notice whose `seq` is exactly `have + 1` is case 2 with one URL. A client never fetches a change it already applied.