# Capture first, shape later

> For agents: start with the [agent guide](https://ratstack.sh/llms.txt). Every page is Markdown by default; add `Accept: text/html` for HTML.

New on /news: [Foldkit 0.171.0 requires result Message schemas](/news).

An event you didn't store can't be analyzed later. Shape decisions change, but a lost event stays lost. Event recording stores everything it is allowed to store. Queries select and interpret the stored fields when they run.

Cloudflare Basin makes this concrete. Events that do not match a structured stream's schema "are accepted during ingestion but will be dropped during processing," the docs say. They also say "schema modifications are not supported after stream creation."

An unstructured stream instead accepts "any valid JSON without validation" into "a single value column." [Manage streams](https://developers.cloudflare.com/basin-pipelines/streams/manage-streams/)

Snowplow reached the same rule years earlier. Its pipelines are "non-lossy," it says. When validation or enrichment fails, payloads are "stored into a bad rows storage solution … instead of being discarded." [snowplow-event-recovery](https://github.com/snowplow-incubator/snowplow-event-recovery)

In rat-stack, the `packages/events` cartridge owns event recording and its resources.
It sends one JSON event record per request to an unstructured Basin stream. A pipeline copies it unchanged (`SELECT *`) into an Iceberg table, and queries select and interpret its fields later.

Size and rate limits still apply. The stream also requires the `{ "value": <event> }` row wrapper; omitting it drops rows during processing. [Analytics](/systems/analytics) documents that boundary.

Capture first keeps only permitted request facts. The event record holds method, path, status, and referrer origin and path. Its query string has credential, email, code, and session keys removed.

The request body is never read. The IP is stored only as a salted hash, computed with an additional secret value.

Visitor identity is a setting:

- **`daily`**: Hash of IP, user agent, and day; no cookie.
- **`persistent`**: One first-party id per browser.

The server sets the persistent id. WebKit "deletes all cookies created in JavaScript and all other script-writeable storage after 7 days of no user interaction." [Tracking Prevention in WebKit](https://webkit.org/tracking-prevention/)

Segment's identify guidance joins people later by a stable database id. It advises against using an email address. It gives the reason: "because database IDs never change." [Spec: Identify](https://segment.com/docs/connections/spec/identify/)

Property tests check these rules. See [property-based testing](/lore/property-based-testing) and [model-based testing](/lore/model-based-testing).

## Sources

1. [Manage streams](https://developers.cloudflare.com/basin-pipelines/streams/manage-streams/)
   Cloudflare docs. Stream validation rules; used for capture-first unstructured ingestion. Accessed 2026-10-01.

2. [GitHub - snowplow-incubator/snowplow-event-recovery · GitHub](https://github.com/snowplow-incubator/snowplow-event-recovery)
   GitHub snowplow-incubator/snowplow-event-recovery. Event recovery design; used for retaining failed payloads rather than discarding them. Accessed 2026-10-01.

3. [Spec: Identify](https://segment.com/docs/connections/spec/identify/)
   Segment docs. Identify guidance; used for joining people with stable database ids. Accessed 2026-10-01.

4. [Tracking Prevention in WebKit](https://webkit.org/tracking-prevention/)
   WebKit. Storage expiry rules; used for the server-set persistent visitor cookie. Accessed 2026-10-01.

5. [rat-stack/packages/events at main · joelhooks/rat-stack · GitHub](https://github.com/joelhooks/rat-stack/tree/main/packages/events)
   GitHub joelhooks/rat-stack. Events implementation; used for the envelope, redaction, and identity modes. Accessed 2026-10-01.

## Lore on this page

- [A cartridge owns one job and its resources](/lore/cartridges)
- [Property-based testing](/lore/property-based-testing)

## Linked from

- Agent guide → An Effect stack so pure (aspirational) Kit Langton will blush. → [Read page](https://ratstack.sh/llms.txt)

- Analytics → One event per HTTP request, stored unchanged for queries to select and interpret later. → [Read page](https://ratstack.sh/systems/analytics)

- Change log → What changed in the files served here, newest first. → [Read page](https://ratstack.sh/log)

- Full agent guide → An Effect stack so pure (aspirational) Kit Langton will blush. → [Read page](https://ratstack.sh/llms-full.txt)

- Property-based testing → State a rule that must hold for every input, then let a generator search for a failing input. → [Read page](https://ratstack.sh/lore/property-based-testing)

- Queues, Pipelines, Workflows and K2 do different jobs → Pick a Cloudflare product for work outside a request by the unit you track: a message, a row, a step, or a record in a log. → [Read page](https://ratstack.sh/lore/queues-pipelines-workflows-and-k2-do-different-jobs)

- Rat Stack lore | rat-stack → Short, source-grounded notes on the ideas and decisions behind rat-stack. → [Read page](https://ratstack.sh/lore)

- Source change history (log.md) → The generated change log as Markdown. → [Read page](https://ratstack.sh/log.md)
