# Code snippets quote a pinned source

> For agents: start with the [agent guide](https://ratstack.sh/llms.txt). Every page is Markdown by default; add `Accept: text/html` for HTML.

New on /news: [Foldkit 0.171.0 requires result Message schemas](/news).

**Design brief; shipped at `a646f7ffd00273d85fbe1d905d9cdb194fdc0f60`.**

`@rat-stack/code-snippets` gives an excerpt its exact source, checks its metadata, and builds both reader views from one model.
See [Write a wiki page](/skills/write-a-wiki-page) for authoring steps and exact repair messages.

Lore explains design choices. [Systems](/systems) documents shipped behavior, as set out in the [project vision](https://github.com/joelhooks/rat-stack/blob/main/VISION.md).

## The problem

A copied excerpt can stop matching its source.
A reader needs the exact revision, file, and selected lines.
An author needs errors that explain how to repair a broken reference.

The package reads [pinned sources](/lore/pinned-sources-can-report-drift) during content generation.
It produces [one snippet model for both renderers](/lore/one-snippet-model-two-renderers).
It reports changed selected lines as warnings without replacing the pinned text.

## The authoring syntax

A Markdown code fence is a block delimited by backticks. It differs from rat-stack's enforcement fence.

Code fences are the only authoring syntax. Authors do not write `<Code>` elements.
The internal component renderer remains an implementation detail.

Fence metadata means options on the opening line of a Markdown code block.
It has these parts:

- An optional language token.
- Whitespace-separated `key=value` pairs.
- At most one highlight set, such as `{3,5-7}`.
- Unquoted or double-quoted values.

The accepted keys are `repo`, `path`, `at`, `lines`, `title`, and `nocheck`.
Unknown keys, duplicate keys, and malformed metadata are errors.

Every published TypeScript fence without `path` must typecheck.
The content build writes each one to `apps/site/dist/doc-fences` as its own module.
`pnpm check` typechecks that directory with `apps/site/tsconfig.doc-fences.json`.
A fragment that cannot compile takes `nocheck="<reason>"`.
The content build logs each opt-out with its page and line, so the count stays visible without failing a build.

The shipped request records content, selection, and source repository, commit, file, and line numbers:

packages/code-snippets/src/model.ts at c936b94c957ada7199df3ab7b39a330d6e1c644f; lines 17-34; highlighted 23,29. Title: Decoded CodeRequest.

```typescript
export class CodeRequest extends Schema.Class<
  CodeRequest,
  { readonly brand: unique symbol }
>("CodeRequest")({
  body: Schema.String,
  commit: Schema.String,
  configured: Schema.Boolean,
  highlights: Schema.Array(RangeSchema),
  language: Schema.String,
  line: Schema.Int,
  path: Schema.String,
  rangeText: Schema.String,
  ranges: Schema.Array(RangeSchema),
  reference: Schema.Boolean,
  repo: Schema.String,
  sourcePath: Schema.String,
  title: Schema.String,
}) {}
```

`configured` distinguishes metadata fences from ordinary fences. `ranges` carries decoded selections.

A source reference has an empty body. These selected lines enforce the pin and path rules:

packages/code-snippets/src/scanner.ts at a646f7ffd00273d85fbe1d905d9cdb194fdc0f60; lines 180-190,192-197; highlighted 189,193-197. Title: Pinned revision and relative path.

```typescript
const validCommit = (value: string) => {
  for (let index = 0; index < value.length; index += 1) {
    const char = value.charAt(index);

    if (!(char >= "0" && char <= "9") && !(char >= "a" && char <= "f")) {
      return false;
    }
  }

  return value.length === 40;
};
const validPath = (value: string) =>
  !value.startsWith("/") &&
  !value.includes(":") &&
  !value
    .split("/")
    .some((part) => part === ".." || part === "" || part === ".");
```

Paths reject traversal, empty segments, dots, and colons.

For a source reference:

- `repo` identifies a configured repository. It defaults to `rat-stack`.
- `path` names a relative file path.
- `at` pins a full 40-character lowercase commit SHA, never a branch or tag.
- `lines` selects positive, ordered, non-overlapping inclusive ranges.
- Highlights use original source line numbers and stay inside visible ranges.
- Configured fences show at most 25 source lines. Collapsed gaps do not count.

The cap also covers inline fences with metadata. Ordinary fences without metadata keep their renderer and are not capped by this rule.

A `path` reference with body text fails with `CodeBodyWithReference`.

An explicit language wins. An unknown explicit language is an error.
Without a language, a source reference infers it from the file extension.

An unmapped extension or an unlabelled inline fence uses `markdown`.
The adapter maps `svx` to Markdown and `txt` to text; it accepts `javascript` explicitly.

[Schema decoding](/lore/schemas-define-the-boundary) parses unknown metadata into a `CodeRequest` at the boundary.
Later stages receive decoded values.

## The pipeline

Preflight checks every independent snippet job before any generated output is written.

Diagram: Collect produces CodeRequest; resolve produces ResolvedCode; highlight produces HighlightedFile; decorate produces CodeSnippet; preflight permits HTML and agent Markdown emission.

```text
fence nodes
    │ collect and decode
    ▼
CodeRequest
    │ resolve pinned Git object
    ▼
ResolvedCode
    │ tokenize the whole file
    ▼
HighlightedFile
    │ select lines, highlights, and gaps
    ▼
CodeSnippet
    │ all jobs pass preflight
    ├──────────────────────┐
    ▼                      ▼
HTML                   agent Markdown
```

Each stage adds facts. Both renderers use the same final model.

- **Collect**: Walk Markdown syntax-tree code nodes; scan metadata; Schema-decode requests and source locations.
- **Resolve**: Read source text; determine file length and language; check ranges; derive a source link.
- **Highlight**: Produce tokens for every source line.
- **Decorate**: Select numbered lines; attach highlights, collapsed gaps, source details, and diagnostics.
- **Emit**: Write assets only after every independent job passes preflight.

[Whole-file tokenization](/lore/tokenize-before-slicing) comes before range selection.
This preserves the preceding source context for multiline comments and strings.

## Ports and adapters

A port is an application-owned interface. An adapter implements that interface for a provider.
The package follows [ports and adapters](/lore/hexagonal-architecture).

Its core owns stage schemas, errors, and selection rules. Core imports neither Git nor Shiki.

`SourceRepository` resolves repository id, commit, and path through a decoded registry.
Each entry contains `id`, `adapter`, and `location`. Optional fields are `linkTemplate` and the bootstrap `remote`.

The `/git` adapter reads any configured local Git repository.
Mischief configures `rat-stack` to the clone used for generation.
Link templates substitute `{sha}`, `{path}`, `{start}`, and `{end}`.
A GitHub, GitLab, Forgejo, or link-free entry needs no core change.

The [highlighter port](/lore/engine-neutral-tokens-keep-adapters-replaceable) exposes:

- Per-line tokens containing `text`, `role`, and `fontStyle`.
- Language, alias, and theme catalogues.
- An engine fingerprint: an identifier for the highlighter, version, and settings used in cache keys.

The `/shiki` adapter uses the build-only Shiki 4.6.0 pin.
Shiki token types stay inside that adapter.
A `/plain` adapter supplies uncoloured tokens for core tests.

A service is a named interface. A Layer describes how to build a service and supply its dependencies.
A Scope owns resource cleanup.

[Layers](/lore/layer-constructor-pattern) provide the adapters. Shiki acquisition and disposal stay inside a scoped Layer.

Mischief collects Markdown syntax-tree nodes, chooses component renderers, and owns markup, CSS, and reporting.
Syntax styles obey [the fence](/lore/the-fence), including the existing accent rule.

### Cache boundaries

During one build, caches reuse successful results whose inputs cannot change:

- Blob cache: repository, commit, and path.
- Whole-file token cache: content key and engine settings.
- Render cache: the canonical snippet content key.

Snippet decoration also depends on ranges and highlights.
HEAD, diagnostics, and failures do not enter the immutable cache.
There is no persistent cache initially. Content-addressed output and Turbo provide reuse between builds.

## Errors as interface

[Errors are part of the interface](/lore/errors-are-part-of-the-interface).
Each typed error keeps the source location, request context, and a repair instruction.
Range errors include requested ranges and actual file length when available.
Invalid metadata retains raw values because no decoded request exists.

- **Metadata**:
  - Shipped tags: `CodeInvalidProps`, `CodeUnknownKey`, `CodeDuplicateKey`, `CodeMalformedMeta`, `CodeBodyWithReference`
  - Repair: Correct the fence metadata or remove reference body text.
- **Registry**:
  - Shipped tags: `CodeUnknownRepo`, `CodeRepositoryConfigInvalid`
  - Repair: Select a registered id or repair the registry.
- **Source**:
  - Shipped tags: `CodeMissingCommit`, `CodeMissingPath`, `CodeSourceUnavailable`
  - Repair: Correct the pin or path; fetch missing shallow history before generation.
- **Selection**:
  - Shipped tags: `CodeInvalidRanges`, `CodeRangeOutOfBounds`, `CodeHighlightOutsideRanges`, `CodeLineCapExceeded`
  - Repair: Use ordered, visible ranges within file bounds and the line cap.
- **Engine**:
  - Shipped tags: `UnknownLanguage`, `CodeHighlightFailed`
  - Repair: Select a supported language or repair the highlighter adapter.

The report-level `CodeBuildFailed` holds collected failures.
[Error accumulation](/lore/accumulate-independent-errors) uses [`Effect.validate`](https://github.com/Effect-TS/effect/blob/289a53f7b1fc40049219c6e69022db2fb859c356/packages/effect/src/Effect.ts#L612-L619). `validateAll` is not the operation used here.

A dependent stage stops on failure. Unrelated jobs still run.
No asset writer runs before successful preflight.

[Source drift](/lore/pinned-sources-can-report-drift) produces diagnostics while allowing the build to continue.
The [error model](/lore/error-model) separates expected failures from defects.

## The testing approach

The package ships property tests for these rules. Mischief also tests the renderer integration.

Use a small plain-array reference model for visible line numbers, gaps, and highlights.
Do not copy production selection logic into the model.
See [model-based testing](/lore/model-based-testing) and [tests that earn their place](/lore/tests-that-earn-their-place).

- **P1**: Printing then parsing valid metadata preserves the request.
- **P2**: Every arbitrary metadata string produces a request or a typed failure; parsing never crashes.
- **P3**: Accepted ranges are positive, ordered, disjoint, and strictly within source bounds.
- **P4**: Visible line numbers, collapsed gaps, and highlights match the independent array model.
- **P5**: Selected tokens match whole-file Shiki tokens, including multiline comments and template strings.
- **P6**: HTML, agent Markdown, and pinned source agree on selected text, repository, commit, file, and line numbers.
- **P7**: Exactly `k` independent failures produce `k` diagnostics and no asset writes.
- **P8**: Repeated rendering with an identical content key produces identical bytes. Every output-affecting input participates in the key.

A changed key need not change output bytes. An engine version change can preserve token output.

Use [property-based testing and shrinking](/lore/property-based-testing).
Effect 4 exposes [`Arbitrary.schema`](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/effect/src/Arbitrary.ts) through `effect/Arbitrary`.
[`it.prop` and `it.effect.prop`](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/vitest/src/index.ts#L53-L165) accept Schema or Arbitrary inputs.

The package uses 40 runs per property.
P5 exercises Shiki. Other core properties use the plain adapter.

The test suite also checks:

- One exact-message example per error tag at the interface that reports errors.
- Byte comparisons for ordinary fences under the decided default language rule.
- A planted range or slicing bug that makes its property fail.
- Startup bundle checks for the build-only boundary.

Use [tests that earn their place](/lore/tests-that-earn-their-place) for planted-bug checks.
[Agents explore, tests remember](/lore/agents-explore-tests-remember) connects exploration with repeatable behavior checks.

## Prior art and where we differ

- **[rehype-pretty-code](https://rehype-pretty.pages.dev/)**:
  - Reused idea: Fence titles and `{1-3,4}` line highlights.
  - Shipped difference: Add repository, commit, path, and selection keys.
- **[Expressive Code text markers](https://expressive-code.com/key-features/text-markers/)**:
  - Reused idea: Line sets in fence metadata.
  - Shipped difference: Reject malformed or unsupported metadata instead of accepting a broad marker language.
- **[Expressive Code MetaOptions](https://github.com/expressive-code/expressive-code/blob/main/packages/%40expressive-code/core/src/helpers/meta-options.ts)**:
  - Reused idea: A dedicated metadata parser.
  - Shipped difference: Use an allowlist and reject duplicate keys. Its accessors can return the last matching value.
- **[Docusaurus Theme GitHub Codeblock](https://github.com/saucelabs/docusaurus-theme-github-codeblock)**:
  - Reused idea: Import excerpts from GitHub reference URLs.
  - Shipped difference: Resolve local Git objects instead of downloading code.
- **[remark-github-code-import](https://github.com/haocheng6/remark-github-code-import/blob/main/src/utils.ts)**:
  - Reused idea: Import excerpts from GitHub URLs, including commit permalinks.
  - Shipped difference: Keep generation offline; use a configured repository registry.

The scanner accepts only the grammar documented above, a stricter subset than the general `MetaOptions` parser.

## Build-only limits

There is no remote source adapter, mirror synchronization, or persistent snippet cache in this design.
A shallow clone must already contain the pinned commit and file-content Git object.
Generation performs no checkout and no network fetch.

Run `pnpm sources:fetch` explicitly before generation when pinned objects are missing. It uses the configured bootstrap remote and verifies the requested commits.

[Build-time work stays out of requests](/lore/build-time-work-stays-out-of-requests).
The Worker receives static assets. Git access, Shiki, and the snippet package stay in the build.

There is no state machine yet.
This is one scoped Effect over immutable local objects.
It has no retry policy, resumable acquisition, or externally driven state transitions.
A machine would repeat the stage call graph without owning a lifecycle.

[Lifecycles are machines](/lore/lifecycles-are-machines) applies when remote acquisition adds retries, cancellation, or resumability.
That future adapter must use declared Effect actors rather than inline side effects.

See [the glossary](/glossary) for fence metadata and the linked concepts.

## Sources

1. [Shipped code-snippets package](https://github.com/joelhooks/rat-stack/tree/a646f7ffd00273d85fbe1d905d9cdb194fdc0f60/packages/code-snippets)
   rat-stack. The shipped grammar, pipeline, adapters, and exact-message tests. Accessed 2026-10-02.

2. [Component registry](https://github.com/joelhooks/rat-stack/blob/main/apps/site/scripts/component-registry.ts)
   rat-stack. Human and agent renderers consume the shipped snippet model. Accessed 2026-10-02.

3. [Content generator](https://github.com/joelhooks/rat-stack/blob/main/apps/site/scripts/generate-content.ts)
   rat-stack. Existing build-time highlighting and content generation. Accessed 2026-10-02.

4. [rehype-pretty-code](https://rehype-pretty.pages.dev/)
   rehype-pretty-code. Fence titles and line highlighting syntax. Accessed 2026-10-02.

5. [Text markers](https://expressive-code.com/key-features/text-markers/)
   Expressive Code. Line sets in fence metadata. Accessed 2026-10-02.

6. [MetaOptions parser](https://github.com/expressive-code/expressive-code/blob/main/packages/%40expressive-code/core/src/helpers/meta-options.ts)
   Expressive Code. General metadata parsing and last-value accessors. Accessed 2026-10-02.

7. [Docusaurus Theme GitHub Codeblock](https://github.com/saucelabs/docusaurus-theme-github-codeblock)
   Sauce Labs. Code imports from GitHub reference URLs. Accessed 2026-10-02.

8. [remark-github-code-import utilities](https://github.com/haocheng6/remark-github-code-import/blob/main/src/utils.ts)
   haocheng6. Network fetch from raw\.githubusercontent.com. Accessed 2026-10-02.

9. [Effect.validate, Effect 4.1.4](https://github.com/Effect-TS/effect/blob/289a53f7b1fc40049219c6e69022db2fb859c356/packages/effect/src/Effect.ts#L612-L619)
   Effect contributors. Evaluate all independent jobs and collect failures. Accessed 2026-11-14.

10. [Effect Schema](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/effect/src/Schema.ts)
    Effect contributors. Decode unknown input and define tagged errors. Accessed 2026-10-02.

11. [Effect Layer](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/effect/src/Layer.ts)
    Effect contributors. Provide service adapters. Accessed 2026-10-02.

12. [Effect Vitest property APIs](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/vitest/src/index.ts#L53-L165)
    Effect contributors. Schema or Arbitrary inputs, shrinking, and check options. Accessed 2026-10-02.

13. [Effect Arbitrary](https://github.com/Effect-TS/effect/blob/effect%404.0.0/packages/effect/src/Arbitrary.ts)
    Effect contributors. Schema-derived generators and shrinking. Accessed 2026-10-02.

14. [Shiki installation and usage](https://shiki.style/guide/install)
    Shiki. Whole-file token output through codeToTokens. Accessed 2026-10-02.

## Lore on this page

- [The fence](/lore/the-fence)
- [Accumulate independent errors before writing output](/lore/accumulate-independent-errors)
- [Scopes release resources when work ends](/lore/scopes-own-resources)
- [Errors are part of the interface](/lore/errors-are-part-of-the-interface)
- [Property-based testing](/lore/property-based-testing)
- [Pin the source and report drift separately](/lore/pinned-sources-can-report-drift)
- [Workers answer requests; Durable Objects own named state](/lore/workers-answer-requests-durable-objects-own-named-state)

## Linked from

- Accumulate independent errors before writing output → Check every independent job before any job writes an asset. → [Read page](https://ratstack.sh/lore/accumulate-independent-errors)

- Agent guide → An Effect stack so pure (aspirational) Kit Langton will blush. → [Read page](https://ratstack.sh/llms.txt)

- Build and run rat-stack (README.md) → What is in the repo, how the example works, and how to run it. → [Read page](https://ratstack.sh/README.md)

- Build-time work stays out of requests → Git reads and highlighting produce static assets before the Worker handles requests. → [Read page](https://ratstack.sh/lore/build-time-work-stays-out-of-requests)

- Change log → What changed in the files served here, newest first. → [Read page](https://ratstack.sh/log)

- Engine-neutral tokens keep adapters replaceable → The highlighter port exposes text, roles, and font styles rather than Shiki types. → [Read page](https://ratstack.sh/lore/engine-neutral-tokens-keep-adapters-replaceable)

- Errors are part of the interface → An author error needs a location, the rejected input, and a repair. → [Read page](https://ratstack.sh/lore/errors-are-part-of-the-interface)

- Full agent guide → An Effect stack so pure (aspirational) Kit Langton will blush. → [Read page](https://ratstack.sh/llms-full.txt)

- Learn the stack | rat-stack → Hands-on guides to Effect actions, XState lifecycles, and the seams between stack pieces. → [Read page](https://ratstack.sh/skills)

- One snippet model feeds both renderers → HTML and agent Markdown derive their code, repository, commit, path, and line numbers from the same snippet. → [Read page](https://ratstack.sh/lore/one-snippet-model-two-renderers)

- Pin the source and report drift separately → A pinned excerpt stays fixed while a separate diagnostic compares it with HEAD. → [Read page](https://ratstack.sh/lore/pinned-sources-can-report-drift)

- Rat Stack lore | rat-stack → Short, source-grounded notes on the ideas and decisions behind rat-stack. → [Read page](https://ratstack.sh/lore)

- Source change history (log.md) → The generated change log as Markdown. → [Read page](https://ratstack.sh/log.md)

- The rat-stack bestiary eats excuses → A field guide to eight documented rat-stack creatures, with real habitats, enforceable behaviour, and strictly imaginary temperaments. → [Read page](https://ratstack.sh/lore/shitrat-bestiary)

- Tokenize the whole file before slicing → Select lines after tokenization so preceding code still guides recognition of multiline strings and comments. → [Read page](https://ratstack.sh/lore/tokenize-before-slicing)

- write-a-wiki-page → Write or revise public wiki pages with short prose, cited claims, meaningful visuals, and pinned code snippets. → [Read page](https://ratstack.sh/skills/write-a-wiki-page)

## Unlinked mentions

- [Agent rules for rat-stack (AGENTS.md)](https://ratstack.sh/AGENTS.md) → npm adds provenance.

- [Effect + Alchemy peers](https://ratstack.sh/resources/peers) → Provenance records the source snapshot and author license confirmation.

- [gardener](https://ratstack.sh/skills/gardener) → Before adding a necessary exact-version exception, inspect the registry integrity, provenance, tarball file list, package scripts, and dependency changes.

- [The rat-stack bestiary eats excuses](https://ratstack.sh/lore/shitrat-bestiary) → A doc-fence is a published TypeScript code block that the content build writes out and pnpm check typechecks.
