Skip to content

The canonical model

Neurowire has exactly one in-memory representation of a feed: the NeurowireFeed. Every parser (RSS, Atom, RDF, JSON Feed, HTML auto-detect, taps) produces this shape, and every serializer (NWF, atom, rss, json, md) consumes it. Nothing else in the system reads the upstream format directly. This single model is the contract that lets any input format turn into any output format.

The model lives in packages/core/src/model.ts and is defined with zod schemas, so the same definition both types the code and validates unknown input at runtime.

Design: list metadata only, no article bodies

Neurowire deliberately models a feed as a list of articles, not the articles themselves. An entry carries title, link, dates, a short summary, authors, and tags. It does not carry the full HTML article body.

This keeps the model small, the outputs compact, and fetching cheap (one listing page or feed, not N article pages). If you want the full text, follow the entry's link.

TIP

This is why the nwf format can be so terse and why a mesh of dozens of sources stays small. The model never holds article content.

NeurowireFeed

FieldTypeRequiredNotes
idstringyesStable identifier for the feed (usually the source URL).
titlestringyesHuman-readable feed title.
homestringnoThe website the feed represents (the "alternate" link).
selfstringnoThe canonical URL of the feed document itself.
updatedstringyesLast-updated timestamp (ISO 8601).
authorsPerson[]noFeed-level authors.
generator{ name, version? }noWhat produced the feed. Neurowire stamps its own here.
entriesNeurowireEntry[]yesThe articles, newest first by convention.

NeurowireEntry

FieldTypeRequiredNotes
idstringyesStable per-entry identifier (see synthetic ids below).
titlestringyesArticle title.
linkstringyesURL of the article.
publishedstringnoFirst-published timestamp (ISO 8601).
updatedstringnoLast-updated timestamp (ISO 8601).
summarystringnoShort description or excerpt. Not the full body.
authorsPerson[]noPer-entry authors.
tagsstring[]noCategories or labels.
source{ name?, url? }noWhere this entry came from. Used by meshes to tag entries by source.

Person

FieldTypeRequired
namestringyes
urlstringno
emailstringno

The zod schemas

The schemas mirror the tables above. EntrySchema and FeedSchema are exported alongside their inferred types, and parseNeurowireFeed(data) validates an unknown value (throwing a ZodError on bad input).

ts
export const EntrySchema = z.object({
  id: z.string(),
  title: z.string(),
  link: z.string(),
  published: z.string().optional(),
  updated: z.string().optional(),
  summary: z.string().optional(),
  authors: z.array(PersonSchema).optional(),
  tags: z.array(z.string()).optional(),
  source: z
    .object({ name: z.string().optional(), url: z.string().optional() })
    .optional(),
})

export const FeedSchema = z.object({
  id: z.string(),
  title: z.string(),
  home: z.string().optional(),
  self: z.string().optional(),
  updated: z.string(),
  authors: z.array(PersonSchema).optional(),
  generator: z.object({ name: z.string(), version: z.string().optional() }).optional(),
  entries: z.array(EntrySchema),
})

Stable synthetic entry ids (content hashing)

Many sources, especially scraped HTML listings, give an entry no real GUID. To keep deduplication and round-trips stable across every format, Neurowire derives a deterministic id from the entry's content when none is present.

The logic lives in packages/core/src/id.ts:

  • hashHex(input) is a pure FNV-1a (64-bit) hash returned as a fixed 16-char lowercase hex string. It uses no node:crypto, so core stays portable.
  • stableId(link, title) hashes \${link}\n${title}`and returnsurn:nwf:<16-char hex>`.

During ingestion, an entry that already has a real id keeps it. An entry with an empty id gets stableId(link, title) stamped in. The same (link, title) always produces the same urn, so the same article keeps the same id across fetches and across output formats.

urn:nwf:9f1c4b2a6d3e0f87

TIP

Because the id is derived from the link and title, an article that is re-fetched (or merged into a mesh) is recognized as the same entry and deduplicated, even when the upstream source never gave it an id.