Library usage

Neurowire is a library first. The CLI and API are thin wrappers over the same functions. This page shows the common entry points; see the reference for full symbol lists.

The split is deliberate: @neurowire/ingest does network work (fetch, detect, parse, merge), @neurowire/core is pure (the model, serializers, filtering, merging), @neurowire/web renders HTML.

Fetch a single feed

fetchFeed(url, options) fetches a website, RSS, or Atom URL and normalizes it to a NeurowireFeed.

import { fetchFeed } from '@neurowire/ingest'

const feed = await fetchFeed('https://blog.rust-lang.org/feed.xml')
console.log(feed.title, feed.entries.length)

For an HTML page, fetchFeed follows a declared feed link, then a registry tap, then heuristic auto-detection. Options include template (force a CSS-selector template), timeoutMs, retries, backoffMs, maxDepth, signal, and cache.

Fetch a mesh

fetchMesh(mesh, options) fetches every source in parallel and merges them into one newest-first feed (tagged by source, deduped). Sources that fail are skipped and never fatal unless all of them fail.

import { fetchMesh } from '@neurowire/ingest'
import type { Mesh } from '@neurowire/core'

const mesh: Mesh = {
  name: 'AI News',
  sources: [
    { name: 'Claude Blog', url: 'https://claude.com/blog' },
    { name: 'Simon Willison', url: 'https://simonwillison.net/atom/everything/' },
  ],
}

const feed = await fetchMesh(mesh, { limit: 20 })

Pass onSourceError to override the default stderr warning for partial failures.

Fetch a construct

A construct is a bundle of meshes. fetchConstruct(construct, options) keeps the per-mesh grouping in a FetchedConstruct (one merged feed per mesh). flattenConstruct collapses it into a single feed for the serializers.

import { fetchConstruct, flattenConstruct, createConfigMeshResolver } from '@neurowire/ingest'
import type { Construct } from '@neurowire/core'

const construct: Construct = {
  name: 'Daily',
  meshes: [
    'ai-news', // a { ref } shorthand, resolved by the resolver below
    { name: 'Releases', sources: [{ name: 'Claude Code', url: 'https://github.com/anthropics/claude-code/releases.atom' }] },
  ],
}

const fetched = await fetchConstruct(construct, { resolver: createConfigMeshResolver() })
const feed = flattenConstruct(fetched) // one feed, entries tagged by mesh

{ ref } members need a MeshResolver. createConfigMeshResolver() reads ~/.config/neurowire/meshes; you can supply any (ref: string) => Mesh | undefined function. Meshes are fetched with bounded concurrency (default 2).

Parse an already-fetched document

When you already have the bytes (no network), ingestDocument(doc, options) turns a RawDocument into a feed. Useful for tests or custom transports.

import { ingestDocument } from '@neurowire/ingest'

const feed = await ingestDocument({
  url: 'https://example.com/blog',
  contentType: 'text/html',
  body: '<html>...</html>',
})

Register taps

To resolve feed-less sites (e.g. claude.com), register the curated taps once at startup.

import { registerAllTaps } from '@neurowire/taps'

registerAllTaps() // built-ins plus ~/.config/neurowire/taps and NEUROWIRE_TAPS

See Taps for adding your own.

Serialize a feed

serialize(feed, format) from core renders a NeurowireFeed to a string. Formats: nwf, atom, rss, json, md.

import { serialize, FORMATS, MEDIA_TYPES } from '@neurowire/core'

const atom = serialize(feed, 'atom')
const json = serialize(feed, 'json')

console.log(FORMATS)        // ['atom', 'rss', 'json', 'md', 'nwf']
console.log(MEDIA_TYPES.json) // 'application/feed+json; charset=utf-8'

The direct serializers (toAtom, toRss, toJsonFeed, toMarkdown, toNwf) and the NWF round-trip helpers (fromNwf, validateNwf) are also exported. See The model and the core reference.

Archive to a journal

A journal is an append-only archive of what a source has published. openJournalStore persists one per id as NWFJ segments; entries the journal already holds are dropped on append, so archiving on a timer is safe.

import { openJournalStore } from '@neurowire/ingest'

const store = openJournalStore() // ~/.config/neurowire/journal, or { dir }

const { added, head } = store.append('ai', feed.entries, {
  id: feed.id,
  title: feed.title,
  home: feed.home,
})
console.log(`added ${added}, head is now ${head.seq}`)

Read it back from a cursor to get only what arrived since last time:

const { entries, tooOld } = store.since('ai', head)

tooOld is true when compaction dropped the entries the cursor pointed past, so the delta cannot be complete and you should re-read the whole journal.

Query an archive with the same filter, window, sort, and limit semantics a live feed uses:

const { entries, scanned, skipped } = store.query('ai', {
  filter: { include: [{ field: 'tag', pattern: 'rust' }] },
  from: Date.now() - 30 * 86_400_000,
  sort: 'date',
  limit: 20,
})

skipped lists segment files the plan ruled out from the manifest alone and never opened. To hand the result to a serializer, wrap it back into a feed:

import { journalToFeed, serialize } from '@neurowire/core'

const records = store.read('ai')
const md = serialize(journalToFeed({ records, head: store.head('ai'), issues: [] }), 'md')

The format itself (createJournalEncoder, parseJournal, readJournalSince, journalHead, verifyJournal, queryJournal) is pure and lives in core, so you can journal to something other than the filesystem. See the core and ingest references.

Render HTML

@neurowire/web is the only package that emits HTML (core stays format-pure). All three functions return self-contained pages with inline CSS and no external requests.

import { toHtml, toConstructHtml, toConstructPages } from '@neurowire/web'

// a single feed or mesh -> one page
const page = toHtml(feed)

// a fetched construct -> an overview page of recap cards
const overview = toConstructHtml(fetched)

// a fetched construct -> an index.html plus one page per mesh
const pages = toConstructPages(fetched) // [{ filename, html }, ...]

toConstructHtml accepts { meshHref } to control where each recap card links. See the web reference.