Skip to content

Lurker Client Protocol & API — a guide for client authors

Audience: third-party client authors (native apps, TUIs, alternative web clients). Last verified against server code 2026-07-21 (main @ fefbeee, protocol version 1). The server source is authoritative; file:line references point into this repository.

Reference implementations: vue_client/ (first-party web, cookie auth) and lurker-ios (native, Bearer auth). Where this doc says "the web client does X," that is reference-client policy, not protocol — noted as such.


1. Architecture in one page

Lurker's server is a persistent IRC client (a bouncer with a database). Your client never speaks IRC. The server owns IRC parsing, TLS/SASL, reconnection, history storage, highlight matching, ignore filtering, and push delivery. Your client speaks two things:

  1. One WebSocket at /ws — all realtime traffic, including message history. Every frame in both directions is a JSON object.
  2. A REST surface under /api/* — auth, config, networks CRUD, settings, uploads, push registration, and other request/response management tasks.

The envelope discriminator differs by direction, and this asymmetry is load-bearing:

  • Server → client frames carry a top-level kind field (snapshot, backlog, irc, read-state, …).
  • Client → server messages carry a top-level type field (send, join, mark-read, …).

Concepts your client models: networks (IRC connections the user configured), buffers (channel / DM / server-log / system conversations), messages (one global id sequence), members (channel nicklists), and read state (server-authoritative unread/highlight counts).

Editions. A server runs as standalone (self-hosted) or node (a hosted lurker.chat cell behind the control plane at app.lurker.chat). Discover which — and the protocol version — before doing anything else:

GET /api/config            (no auth)
→ { "edition": "standalone" | "node",
    "protocolVersion": 1,
    "minProtocolVersion": 1 }

Node edition disables /api/api-tokens, /mcp, and /uploads/* static serving; standalone has no /api/node/*. The WS protocol itself is identical in both. Health check: GET /api/health{"status":"ok","time":"<ISO 8601>"} (no auth, no version).


2. Compatibility contract

Defined in server/protocol.ts. The deal is:

  • Additive-only evolution. New frame kinds, new event types, and new fields may appear at any time. Existing fields are never repurposed. protocolVersion bumps only for a change the additive rule cannot express.
  • Unknown is never fatal — and your client must honor its half. Ignore frame kinds you don't recognize, event types you don't recognize, and fields you don't recognize. The server does the same: an unknown verb gets a non-fatal {kind:'error', text:'unknown message type: …'} and the socket stays open (wsHub.ts:2900).
  • Announce your version on the upgrade: /ws?v=1. Omitting ?v means "treat me as current" — always send it so a future minProtocolVersion bump rejects you cleanly (HTTP 426 Upgrade Required) instead of feeding you frames you misparse. The snapshot frame also carries protocolVersion.

There is no capability negotiation beyond the version integer — no WS subprotocol, no feature flags on the socket.


3. Authentication

Two credentials open every door (REST and WS); both resolve to the same sessions row:

CredentialWho uses itHow
Signed cookie lurker_sessionBrowsersSet by the login endpoints; httpOnly, SameSite=Lax, 30-day
Authorization: Bearer <token>Native / TUI clientsToken from the mint endpoints below; sent on every REST call and on the WS upgrade

Browsers can't set headers on a WS upgrade, hence the cookie path. Native clients should use Bearer exclusively.

3.1 Self-hosted: mint a token

POST /api/auth/login/token          (no auth; failure-throttled)
{ "username": "...", "password": "..." }
→ 200 { "token": "...", "expiresAt": "<ISO8601>", "user": { "id", "username", "role" } }
→ 401 invalid credentials · 429 throttled (see §3.4)

The token is an opaque 32-byte base64url session token (routes/auth.ts:558, db/sessions.ts:17) — the same value the cookie would carry, returned in the body.

  • Lifetime: 30 days, fixed. No refresh token — re-login to renew. Expired tokens are deleted on lookup, plus an hourly purge.
  • Revoke: POST /api/auth/logout with the Bearer deletes the row (per-device revoke). Password reset does not revoke other devices' cell sessions on standalone; each session is its own row.
  • Store it in the platform keychain/keystore. The iOS app uses a Keychain generic-password item with after-first-unlock accessibility so background reconnects work (SessionStore.swift:29).
  • Mint is password-only. A passkey-only account can't mint a native token until it sets a password (PUT /api/auth/password). Surface that case: the mint endpoint just returns 401.

3.2 Hosted (app.lurker.chat): mint at the control plane

POST https://app.lurker.chat/_cp/auth/app/login     (no auth)
{ "email": "...", "password": "..." }
→ 200 { "token": "..." }

(lurker-ios Backend.swift:26-39.) The hosted token is a signed, chat-scoped claim: it works for all proxied chat traffic (REST + WS against app.lurker.chat) but is rejected on control-plane account/billing routes — account administration stays on the web. Revocation is global-only (password reset invalidates every session, via the session epoch); there is no per-device revoke on hosted. After minting, use the token exactly as in §3.1 — same header, same endpoints — the control plane proxies you to the right cell transparently.

3.3 Browser flows (for completeness)

WebAuthn/passkey and password login endpoints (/api/auth/setup*, /invite/*, /login/options|verify|password, /passkeys*) set the lurker_session cookie and are designed for the first-party web client; a native client doesn't need them. GET /api/auth/auth-methods{passkey:boolean} tells a login form what to offer. GET /api/auth/me{user:{id,username,role,is_paused}} validates a session.

3.4 Cross-cutting auth behavior

  • 401 semantics: any 401 from /api/* or a refused WS upgrade means dead session — clear the stored token and return to login. The server deliberately never uses 401 for downstream failures (upload provider errors are 502/400), so you can trust it.
  • Rate limiting: credential endpoints allow 10 failures / 15 min / IP → 429 + Retry-After; the whole /api/auth router is capped at 60 req/min/IP. Honor Retry-After.
  • Paused accounts (hosted billing): every authed non-GET REST call returns 403 {"error":"account paused"} (except logout and exports), and write-verbs on the WS return {kind:'error', text:'account paused'}. The {kind:'account-state', paused:bool} frame notifies live. Treat paused as read-only mode, not an error loop.
  • No CSRF tokens exist; browser security rests on SameSite=Lax + the CORS allowlist (CORS_ORIGIN env, credentials mode). Native clients are unaffected.
  • Global JSON body limit: 1 MB (app.ts:67).

4. The WebSocket

4.1 Upgrade

GET /ws?v=1&since=<highest-message-id-seen>
Authorization: Bearer <token>        (native; browsers ride the cookie)
  • Rejections are raw HTTP status lines before the upgrade completes (wsHub.ts:1694-1718), in order: 403 (browser Origin not same-origin or allowlisted — native clients send no Origin and pass), 426 (?v below minProtocolVersion), 401 (bad/missing credentials).
  • ?since is your resume cursor — the highest persisted message id your client has ever seen (see §4.4). Omit or 0 for a fresh connect.

4.2 Frame plumbing

  • JSON text frames only. Max inbound frame: 256 KiB (uploads go over REST).
  • Flood control: per-socket token bucket, capacity 120, refill 40/sec. Exhaustion → {kind:'error', text:'message rate exceeded'} then close 1008. Don't machine-gun verbs; batch where the protocol lets you.
  • Malformed JSON → {kind:'error', text:'invalid json'}; socket stays open.
  • Heartbeat is WS-level ping/pong, not JSON. The server pings every 30 s and terminates sockets that don't pong by the next sweep. Browser and most WS libraries auto-pong; if yours doesn't, implement it or you'll be dropped every ~60 s. There is no application-level ping message.
  • Delivery is at-most-once per socket, no per-frame acks. Reliability comes from the id cursor + reconnect gap-fill (§4.4), not from the transport.

4.3 Connect: the snapshot burst

On every successful connect the server immediately sends a burst of separate frames, synchronously, in this order (wsHub.ts sendSnapshotInner, 1893):

  1. {kind:'snapshot', protocolVersion, maxUploadBytes, networks, globalIgnores, cursor?} — full live state for every network (see §5.1). cursor (present only on a fresh connect) is the current global max message id: seed your resume cursor from it, because the shell backlogs that follow carry no rows and would otherwise leave your cursor at 0. maxUploadBytes is the largest upload this account may send right now — see §Uploads.
  2. {kind:'draft-snapshot', drafts} — saved per-buffer input drafts.
  3. {kind:'bookmark-ids-snapshot', ids:[…]} — bookmarked message ids.
  4. A backlog frame for the app-scoped system buffer (networkId:null, target:':system:').
  5. {kind:'contacts-snapshot', contacts:[…]} — the friends/contacts list.
  6. One backlog frame per open buffer on each connected network.
  7. Per offline network: a real backlog for its :server: log, shells for its channels/DMs.
  8. {kind:'backlog-complete'} — terminal marker, nothing else in it.

Render progressively from frame 1; don't wait for the end.

What frame 8 is for. Until it arrives, "I have no row for this buffer yet" and "there is no such buffer" are the same observation, and no amount of waiting separates them. After it arrives, absence is proof: a buffer key with no backlog frame is not open — for channels, DMs and :server: logs alike, on connected and disconnected networks alike. That matters if you navigate by key rather than by tapping a row that already exists (restoring the last-read buffer at launch, opening one from a notification tap): without it those screens sit on a spinner forever, because nothing ever says the row isn't coming (#635).

"Not open" is not "never existed", and the burst can't tell you which. A buffer the user closed from another client ships no frame either (it's dropped from the enumeration — wsHub.ts:609), yet it keeps its full persisted history and one open-buffer reopens it with that history intact. Both cases mean the same thing for what you render — it isn't in the buffer list — which is why §9.1 models closed as absent. They differ for what you destroy: don't purge local history, drafts, or a saved read position on the strength of a missing frame, because the server hasn't forgotten any of it.

Two things that look like they'd answer the same question and don't:

  • snapshot's per-network channels is not authoritative for buffer existence. It's empty outright for any network with no live connection (paused account, manually disconnected, never autoconnected) even though those networks still own persisted buffers, and even on a live connection it's read at the 'connected' instant — before auto-rejoin JOINs land. Judge membership from it and you'll decide a user left channels they're still in.
  • Don't probe with open-buffer. For a #channel with no row the server reads that as "join it" (§9.1), so a probe silently re-JOINs a channel the user deliberately left from another client.

An older server never sends frame 8 (it's additive, and protocolVersion does not move for it) — if you don't see one, keep whatever you do today rather than concluding anything. A truncated burst withholds it too: the server only sends it once the whole burst went out, so a snapshot that failed part-way through never claims a buffer is missing.

Shells vs. hydrated backlogs. On a fresh connect (since=0) channel/DM buffers arrive as shells: {kind:'backlog', …, events:[], hasMoreOlder:true} — "this buffer exists; fetch content when the user opens it." Hydrate a shell with either {type:'open-buffer'} (server replies with a populated backlog — the iOS approach) or {type:'history', mode:'latest'} (the web-client approach). Both are valid; pick one.

4.4 Reconnect and resume (?since)

Every persisted message has an id from one global monotonic sequence across all buffers (SQLite rowid — db/messages.ts:115). Track the highest id you have ever seen and present it as ?since on reconnect; the server then ships, per buffer, only events with id > since.

Rules that keep resume correct:

  • The system buffer has a separate id sequence. Never feed its ids into your cursor. Only events with networkId != null advance it. Getting this wrong corrupts resume for every other buffer (web: useSocket.ts:461; iOS: LurkerStore.swift:339).
  • Ephemeral events (§7.2) have no id; they never advance the cursor.
  • Per buffer, a resume gap is capped at 500 events. If the true gap is larger, the server instead sends the latest 200 with reset:true — meaning replace this buffer's contents wholesale; do not splice (there would be a hole). buildResumeSlice, wsHub.ts:771.
  • An in-band {type:'snapshot'} message re-runs the whole burst as a gap-fill from the server's tracked cursor for your socket — useful after long background/hidden periods without dropping the socket (the web client does this after >30 s hidden).
  • Reconnect policy is yours; references: web = flat 2 s retry; iOS = exponential 1→30 s, reset on first received frame, short-circuited by reachability changes. Signal "connected" on the first received frame, not on socket open — a refused upgrade looks like an open-then-close to some WS APIs (LurkerClient.swift:172).

5. Data model

5.1 Network snapshot blob

One per network inside kind:'snapshot' (ircConnection.snapshot(), ircConnection.ts:4315 + ircManager.ts:765):

{ networkId, state,                    // 'connecting'|'connected'|'reconnecting'|'disconnected'
  nick, userModes, lagMs,
  multilineLimits,
  away: { active, since, message, autoSet, backAt } | null,
  channels: [ { name, topic, modes,
                members: [ { nick, modes: [], away, user, host, account } ] } ],
  peerPresence: { "<lowercased nick>": { nick, state, stateAt, awayMessage } },
  pinned: [], collapsedNicklists: {}, channelNotify: {},
  ignoredMasks: [], nickNotes: [], relayBots: [] }

Offline networks appear with state:'disconnected', channels:[], peerPresence:{}. The snapshot does not include network display names/hosts — fetch GET /api/networks for the roster (the iOS app does this before opening the socket; it doubles as a token validity check).

Member modes are prefix-mode letters, highest first (q a o h v), not sigils (~ & @ % +). Map to sigils yourself for display.

5.2 Buffers

A buffer is one conversation: kind ∈ channel | dm | server | system. The server owns buffer existence and open/closed state (a real buffers table row per user/network/target). Clients never decide that a buffer exists — they materialize their local model in response to specific frames, and only those (§9.1). Identity is (networkId, case-folded target); two pseudo-buffers use sentinel targets:

  • :server: — per-network server log (networkId set, target literally ":server:"). Uncloseable. Catch-all for server-voice text (§7.3).
  • :system: — app-scoped Lurker log (networkId: null). Read-mostly; carries type:'system' events. Separate id space (§4.4).

Sentinel targets are exact-match, never case-folded. (The web client's :friends: is a purely client-side virtual view — not a wire concept; ignore it.)

Case folding: IRC targets are case-insensitive and servers echo inconsistently-cased names. Fold with ASCII toLowerCase for identity; keep the first/canonical casing for display. RFC 1459 casemapping (treating {}|^ as the lowercase of []\~) is deliberately not implemented anywhere in Lurker — match that, don't "fix" it unilaterally.

5.3 Messages (MessageEvent)

Common fields on every persisted event (db/messages.ts:31 + decorateMessage, wsHub.ts:430):

{ id, networkId, target, time,        // ISO 8601 — see the note below
  type,                               // see §7.2
  nick, text, kind,                   // kind = raw IRC command; see the ⚠ below
  self,                               // you sent it (any of your clients)
  userhost, alt, mirrored, dm,
  matched, matchedRuleId,             // highlight decoration
  fromIgnored, notifyAlways, notify,
  msgid? }                            // IRCv3 server message id, when supplied

plus type-specific extras (newNick, kicked, modes, members, …). Ephemeral event types (§7.2) carry no id.

time is IRCv3 server-time where the network offers it (the @time= tag), receive time otherwise. Far-future stamps (> ~2 min ahead) fall back to receive time. Rows persisted before this existed carry receive time. Because a bouncer upstream can replay old messages live, time is not guaranteed monotonic with respect to id — order and dedupe by id, always (§9.3).

msgid is the server-assigned IRCv3 message id (message-tags networks; own sends learn theirs via echo-message). Absent — not null — on rows from untagged networks and on optimistic self echoes. It is the future anchor for react/reply; today it is informational only.

notify is the server's delivery decision — the one flag to gate a live alert (toast, sound, native buzz) on. It is the union of the content signals (matched, dm, notifyAlways) with the user's ignore/mute verdict already applied. A NONOTIFY rule — a muted channel, network, DM, or sender (§6 add-ignore) — forces notify:false while the message is still delivered and still counts toward unread; only the alert is suppressed. A hide-level ignore also forces notify:false, but that message is additionally excluded from unread/highlight counts server-side (the from_ignored stamp) and hidden by your render filter — so don't count it. So a muted-channel highlight arrives as matched:true, notify:false — style it as a highlight in history, but do not raise a notification for it. Do not re-implement ignore matching client-side for this decision — the server owns it (it must: push fires when no client is attached, so the veto can't live only in a client). The raw signals stay on the wire beside notify so you can still pick the toast kind / sound per signal type. Note a NOHIGHLIGHT rule is display-only (it clears matched, not notify), so a de-highlighted DM still notifies — notify and the client-applied render/hide filter (from the snapshot's globalIgnores / ignoredMasks) are two different jobs: the server pre-resolves the notify verdict into this flag; you apply the hide verdict yourself.

Live-frame kind clobber. When an event arrives live it is wrapped as {...event, kind:'irc'} — the event's own kind field is overwritten by the envelope discriminator. The raw-IRC-command kind (privmsg, action, notice, …) survives only inside backlog/history events[]. Dispatch on type, never on an event's kind.

5.4 Read state

Server-authoritative, per buffer: {networkId, target, lastReadId, unread, highlights, highlightsCapped} — broadcast to all the user's sockets after every countable event and every mark-read. Never count unread locally (§9.4).


6. Client → server verbs

All messages are {type:'<verb>', …}. Any message carrying a networkId for a network you don't own gets {kind:'error', text:'unknown network'}. Unknown verbs are non-fatal (§2). Verbs marked ⏸ are rejected when the account is paused. Dispatch: handleClientMessage, wsHub.ts:2031.

Sending ⏸

typeFieldsNotes
sendnetworkId, target, text, clientId?PRIVMSG. Ack via send-result iff clientId present
actionnetworkId, target, text, clientId?CTCP ACTION (/me)
noticenetworkId, target, text, clientId?NOTICE
rawnetworkId, lineRaw IRC line — the escape hatch for /mode, /kick, /whois, unknown commands
ctcpnetworkId, target, ctcpType, args, issuingTargetCTCP request (/ping, /version at a user)

Ack contract: include a client-generated clientId on send/action/ notice and the server replies {kind:'send-result', clientId, ok, error?}. This confirms acceptance only — the message itself comes back as a normal irc echo with self:true and its real id (§9.3). The web client times acks out after 8 s (client policy). On networks that ACK echo-message upstream, that self:true frame arrives only after the IRC server reflects the send back (one upstream round trip, carrying the real msgid + server time); elsewhere it is emitted immediately from the server's optimistic local copy.

Channels & buffers ⏸

typeFieldsNotes
joinnetworkId, channel, key?Request only — the buffer appears on channel-joined (§9.1)
partnetworkId, channel, reason?Buffer survives, parted
open-buffernetworkId, targetReopen/create: replies backlog + buffer-opened; JOINs if an unjoined channel; mints an empty DM row for a bare nick
close-buffernetworkId, target, reason?Closes (PARTs a joined channel, untracks a DM peer). :server: refuses

View state (persisted server-side, fanned out to your other devices)

typeFields
mark-readnetworkId, target, messageId — MAX-clamped server-side, idempotent. System buffer: networkId: null, target: ':system:' (send an explicit null, don't omit)
mark-all-read
clear-buffer / unclear-buffernetworkId, target
pin-buffer / unpin-buffernetworkId, target
reorder-pinsnetworkId, targets:[…]
set-nicklist-collapsednetworkId, target, collapsed
set-channel-notify-alwaysnetworkId, target, notifyAlways
draft-set / draft-clearnetworkId, target, body?
input-history-addnetworkId, target, text
set-bookmark / unset-bookmarkmessageId
set-nick-notenetworkId, nick, note
set-relay-botnetworkId, nick, marked, pattern
set-contact / delete-contactcontactId, displayName, notifyOnline, targets / contactId
add-ignore / remove-ignorenetworkId (null = global), rule/mask / id/mask. rule = {mask (null or '*' = anyone), channels?, pattern?, patternKind: substr|full|regex, levels? (default ALL), isExcept?, expiresAt?} (ignoreRuleInput.ts). Channel/network muting is expressed here — a rule with no mask scoped to a channel — not via a dedicated verb

Presence & status

typeFieldsNotes
presencevisible: boolPer-socket, resets to false on every new socket — re-assert on connect (§9.5)
typingnetworkId, target, stateSends +typing TAGMSG
away ⏸ / backmessage / —User-scoped: hits every network
probe-presencenetworkId, nickSilent WHOIS; answer arrives as a peer-presence event

Sync & fetch

typeFieldsReply
snapshotRe-runs the snapshot burst as a gap-fill (§4.4)
historynetworkId, target, mode: before|after|around|latest, limit (1–500), token?, before?/afterId?/anchorId?{kind:'history'} (§8)
searchquery, networkId?, target?, nick?, nicks?, before?, limit?, token?{kind:'search-result'}
list-channelsnetworkIdKicks off /LIST; progress via chanlist-state
chanlist-searchnetworkId, query, sortBy, sortDir, offset, limit{kind:'chanlist-result'}

E2E (RPE2E, per-channel opt-in)

e2e ⏸ (networkId, target, args), e2e-export (networkId) → {kind:'e2eExport'}, e2e-import (networkId, json) → {kind:'e2eImport'}. Status lines surface as ephemeral type:'e2e' events. Niche — safe to skip in a v1 client.


7. Server → client frames

7.1 Frame kinds

kindPayloadWhen
snapshotprotocolVersion, maxUploadBytes, networks[], globalIgnores[], cursor?Connect burst / gap-fill
draft-snapshot / bookmark-ids-snapshot / contacts-snapshotdrafts / ids[] / contacts[]Connect burst
backlog-completeLast frame of the burst; absence is proof (§4.3)
backlognetworkId, target, events[], reset?, hasMoreOlder, joined, lastReadId, unread, highlights, highlightsCapped, clearedBeforeId, clearedAt, speakers?, inputHistory?Burst, open-buffer reply, resume gap
ircA decorated MessageEvent (§5.3) with kind clobbered to 'irc'Every live IRC-side event
historynetworkId, target, mode, token, events[], speakers, hasMoreOlder, hasMoreNewer, hasMore, before/afterId/anchorId/anchorMissing (per mode)Reply to history
read-statesee §5.4After every countable event / mark-read
send-resultclientId, ok, error?Ack for send/action/notice
buffer-opened / buffer-closed / buffer-reopenednetworkId, targetBuffer lifecycle (§9.1)
buffer-clearednetworkId, target, clearedBeforeId, clearedAt/clear marker
pins-changednetworkId, pinned[]Authoritative pin order
nicklist-collapsed-changed / channel-notify-changednetworkId, target, …View-state sync
draft-updated / input-history-added / bookmark-updated / nick-note-updated / relay-bot-updated / contact-updated / contact-deleted / ignore-list-updatedvariousMulti-device view-state fan-out
settingschanges, maxUploadBytes? (only when the upload cap was touched)Server-side settings changed
highlight-rules-changedRe-fetch highlight rules
account-statepaused: boolHosted pause/resume
chanlist-state / chanlist-result/LIST cache meta / result pageChannel browser
e2eExport / e2eImportE2E key material / import resultReplies, this socket only
dcc-transferfull transfer row (snake_case)DCC state changes
upload-progresstoken, phase, destination, percentDuring REST upload (correlate via progressToken)
exportjobExport job progress
errortextNon-fatal; also the reply to unknown verbs

7.2 irc event types (the inner type field)

Also the type of rows inside backlog/history events[]. P = persisted (has id, advances the cursor); E = ephemeral (no id).

typeP/EExtra fields / meaning
messagePPRIVMSG (kind:'privmsg' in stored rows)
actionP/me
noticePmirrored:true on the :server: copy of a notice to a closed/absent DM
joinPaccount?
part / quitPtext = reason
kickPkicked, text
nickPnewNick
own-nickEyour nick changed — nick is the new one
modePtext, modes[]
usermodeEyour user modes, whole string
topicPa topic change (renders as a line)
channel-topicERPL_TOPIC on join — set state, render nothing
channel-modesEfull channel mode string
channel-joinedEyou are in the channel — the materialization signal (§9.1)
channel-partedEyou left/were removed — mark parted, keep history
join-errorEjoin failed — text, reason; do not create a buffer
namesEmembers[…] — full nicklist replace
member-updateEmember{…} — single-nick patch (away/account/host changes)
inviteE/Pyou were invited (channel, from) / op-visibility variant
stateEnetwork state + nick on connect — drive the connection indicator
motdPMOTD and all otherwise-unclassified server text (§7.3)
errorPserver error text; unknownCommand? for 421s
systemP*system-buffer line; severity in level: info|warn|error, not in type
away-stateEyour own away state per network
peer-presenceEnick, state ∈ online|offline|away|back, stateAt, awayMessage, cameOnline
typingEnick, state ∈ active|paused|done
lagElagMs
ctcpECTCP request/reply status text
chghostPnewIdent, newHost — render only; the nicklist patch rides member-update
e2eERPE2E status, level + text
chanlist-start/progress/endE/LIST refresh progress

*system rows persist in their own table with their own id sequence — see §4.4.

7.3 Where server text lands

Events of type motd, error, e2e, ctcp may arrive with no target. Route them to that network's :server: buffer. motd is deliberately the catch-all for all "server voice" text that has no better home — don't build a taxonomy on top of it. system events (with networkId:null) belong to :system:.


8. History & backlog merging

All history flows over the WS. Request {type:'history', mode, …}; one reply {kind:'history'}. limit clamps to 1–500. History is DB-backed and connection-independent — offline networks still serve it.

modeRequest keysSemanticsMerge
before (default)before (exclusive id)Older page (scroll up)Prepend, dedupe by id
afterafterIdNewer page (scroll down while detached)Append, dedupe by id
aroundanchorIdJump-to-message window (max 2×limit+1); anchorMissing if goneReplace slice; buffer is now detached from live
latestNewest slice; hydrates a shell; includes inputHistoryReplace; buffer is live again

hasMoreOlder / hasMoreNewer gate the pagers (hasMore is a legacy alias of hasMoreOlder). Echo the request token discipline the web client uses if you pipeline requests: keep a monotonically increasing token and drop any reply whose token you've superseded.

Jump-to-message detaches the buffer (Discord/Slack convention): after around, live events for that buffer should not be spliced into the visible slice (track them separately or refetch); latest reattaches.

Merge rules that protect you from data loss:

  1. backlog with reset:true → replace the buffer's contents wholesale (resume gap overflowed; splicing leaves a hole).
  2. backlog without a reset field, or the system buffer's backlog (which hardcodes reset:false but means "replace") → replace. Only networkId != null and reset === false means gap-append. (iOS FrameParser.swift:104-111; web detects the non-overlap case instead.)
  3. Never un-hydrate: a later shell (events:[]) for a buffer you already populated must not wipe it.
  4. On any replace, keep held live events newer than the slice tail — a message can land mid-hydrate.
  5. Dedupe everything by id against what you hold; drop legacy away/back rows if you encounter them in old history.
  6. The web client caps its in-memory ring at 500 events/buffer and pages the rest — policy, not protocol, but a sane default.

9. Rules your client must get right

The tribal-knowledge section. Every one of these was a real bug once.

9.1 Buffer materialization

The server decides existence; your client mirrors it in response to exactly these signals:

  • A channel buffer materializes on channel-joined — never on the join request. {type:'join'} is intent; record it as pending (the web client keeps a 10 s timeout that surfaces "no response joining"). join-error cancels the intent and creates nothing — a 470 forward means the channel you asked for never existed; the channel you were forwarded to announces itself with its own channel-joined.
  • A DM materializes on an incoming persisted message/action, or locally when the user initiates one (send + activate, or open-buffer on a bare nick — the server persists an empty DM row, which survives reloads).
  • buffer-closed → drop the buffer from your model entirely (messages, drafts, membership). Closed = absent.
  • buffer-reopened needs no handler — the message that caused the reopen arrives as a normal irc event and materializes the buffer via the DM rule.
  • channel-parted → resolve, never materialize: mark parted, clear members, keep the buffer and history. If you have no such buffer, ignore it.
  • Never materialize from ambient signals: typing, member-update, and read-state for unknown buffers must resolve-or-drop. (mark-all-read fans out read-state for closed buffers too — resurrecting them in the sidebar was bug #319; typing-tag DM creation was #292.)
  • NOTICEs never create or reopen a DM — a notice to a closed/absent DM arrives on :server: with mirrored:true. Already handled server-side; just don't special-case it.
  • Navigating to a buffer by key (launch restore, notification tap) is the one case where you need to ask "does this exist?" rather than mirror a signal. Wait for backlog-complete; if the key still has no backlog frame after it, that buffer isn't open — say so instead of spinning. It may be closed rather than gone, so render it as absent but don't destroy anything local over it. Don't guess from snapshot and don't probe with open-buffer — see §4.3 for why both are traps.

9.2 Identity & case

Fold targets with ASCII lowercase for identity; sentinels (:server:, :system:) exact-match; first-seen casing is display-canonical (§5.2).

9.3 Sending: no optimistic rendering

Do not locally append sent messages. Send {type:'send', clientId}; the authoritative row echoes back as an irc event with self:true and the real id — that's when it renders. send-result only tells you accepted/failed (drive a spinner or an error toast from it). This is what makes multi-device echo, ids, and ordering trivially correct. Consequences:

  • A dead socket at send time = keep the input text, tell the user; nothing was sent.
  • Your own echoes count for dedupe and cursor like any other event; skip self-events when building tab-completion "recent speakers".

9.4 Read state is server-authoritative

Render read-state verbatim; never count locally. Send mark-read with the id the user has seen — the server MAX-clamps, so re-sending stale ids is safe. Mark on focus-in and on live messages while focused (focus-out marking loses the tab-close race). The unread divider is client policy: snapshot lastReadId when the buffer becomes active and pin it until switch-away. App badge = Σ highlights across buffers; recompute on every read-state (a push notification can only revise the OS badge, your client must correct it when the user actually reads).

9.5 Presence is per-socket and explicit

Every new socket starts visible:false. Assert {type:'presence', visible:true} when your UI is actually in front of the user, false when it leaves — and re-assert after every reconnect. Presence is what gates push (no push while any client is visible) and auto-away (no visible client for away.auto.delay_seconds → server sets away). An open socket is deliberately not presence: a backgrounded phone keeps its socket and must still get push. On mobile, flush presence:false before suspension if the platform allows; the server's heartbeat reaper (~60 s) is the fallback.

9.6 Ordering & dedupe

Per buffer, apply a persisted event only if its id is greater than the newest you hold; run side effects (nicklist mutations, topic set, notification, unread sounds) only when the event was fresh. Replays happen by design (resume overlap, backlog/live races) — the topic-revert and double-join-line bugs both came from mutating state below a missing dedupe check. Membership side effects ride the same events you render: join→add member, part/quit→ remove, kick→remove kicked, nick→rename (chghost renders only; its nicklist patch arrives separately as member-update).


10. HTTP API reference

Everything requireAuth unless noted. Errors are {"error": "<message>"} (some add key/code/status). Exact multipart field names matter.

Networks — /api/networks

Method & pathBody / notes
GET /{networks:[…]}. Passwords never returned — has_password / has_sasl_password booleans instead
POST /{name*, host*, port, tls, nick*, username, realname, server_password, autoconnect, sasl_account, sasl_password, default_channel, connect_commands, trusted_certificates}201 {network}, connects immediately. 403 if host blocked by admin allowlist
PATCH /:id · DELETE /:idPartial update / delete
POST /reorder{ids:[…]} (409 on set mismatch, returns authoritative order)
POST /:id/connect · /disconnect · /reconnectdisconnect takes {reason?}
POST /:id/join · /part{channel*, key?} / {channel*, reason?}; 409 if not connected

GET /api/network-presets{presets, allowUserDefined} for the add-network form.

Settings & personalization

EndpointNotes
GET /api/settings/bootstrap{registry, values} — the registry is self-describing (types, defaults, enums); build your settings UI from it rather than hardcoding keys
PATCH /api/settings{changes:{key:value,…}}{values}; other devices get the settings frame
DELETE /api/settings/:keyReset to default
/api/highlight-rulesCRUD: GET /, POST /, PATCH /:id, DELETE /:id
GET /api/highlightsPaginated highlight feed: ?limit (≤200), before, networkId, q, nick (repeatable), target{items, nextBefore}
GET /api/bookmarks?limit, before{items, nextBefore}
POST /api/drafts/flushBeacon-style: raw text body containing JSON {drafts:[{networkId,target,body}]}204. For page-unload flush; live clients use the draft-set verb

Uploads — /api/uploads

EndpointNotes
POST /multipart/form-data, file field image; optional uploaderId, progressToken (≤64 chars — progress arrives as upload-progress WS frames). → {id, url, mime, can_delete, thumbnail_url?}. 413 over cap, 415 rejected type, 502 provider error (never 401)
GET /?before, limit, q, kind{items, providers, maxUploadBytes}
GET /:id/thumbBinary thumbnail
DELETE /:id409 if not deletable

/api/uploaders manages upload destinations (list/select/create/update/delete; secrets write-only). Standalone serves local files publicly at GET /uploads/:key (no auth, sandboxed CSP). Paste the returned url into a message — the server does the rest.

Size cap. maxUploadBytes — on the snapshot frame and on GET /api/uploads — is the largest file this account may send, and the number to compress media against. Do not hardcode a cap. It is the smallest of three ceilings: the 200 MB hard limit, the instance's declared transport limit (LURKER_MAX_UPLOAD_MB — what a CDN or reverse proxy in front of Lurker will pass), and the per-user or operator-baked uploader cap. Only the first is universal, so a self-hosted instance and a CDN-fronted one legitimately differ by a lot.

It already has the multipart envelope subtracted, so a file at exactly maxUploadBytes still fits inside the request body once the boundaries, part headers, and uploaderId / progressToken fields are added. Size the file to it; don't budget for the envelope yourself.

Treat it as advisory, not a contract: it is resolved for the account's default uploader, so a per-upload uploaderId override with a tighter policy cap, or an operator changing the limit mid-session, is still settled by a 413 carrying the real number. It is refreshed on reconnect, and re-sent on the settings frame whenever the user changes their own cap — so a client that reads it from both frames never compresses against a stale number.

Imports are capped separately. POST /api/imports has its own, much larger limit (500 MB) and is not bound by the 200 MB upload ceiling — so maxUploadBytes is the wrong number to check an archive against. Read maxImportBytes from GET /api/exports/preview instead. Only the instance's transport limit is shared between the two, which does mean lowering LURKER_MAX_UPLOAD_MB lowers both. Over-limit archives get a 413 with code: "archive_too_large"; there is no way to compress an archive to fit, so check before uploading.

DCC — /api/dcc (403 unless enabled for the account)

GET /?limit list · POST /:id/accept|reject|cancel. Live updates via dcc-transfer frames; file bytes move over IRC, not HTTP.

Export / import

GET /api/exports/preview · POST /api/exports ({include_messages}, allowed while paused, → 202 {job}, progress via export frames) · GET /api/exports/:id/download (.lurk archive, Range-capable) · POST /api/imports (multipart, field archive, ≤500 MB; 409 account_not_empty). A mobile/TUI client can skip all of this.

Out of scope for third-party clients

  • /api/admin/* — admin panel (users, invites, presence, instance uploaders/ networks). Admin-gated; build against it only if you're making an admin tool.
  • /api/api-tokens + /mcp (standalone only) — a separate Bearer namespace for MCP/automation. Those tokens cannot open the WS; don't confuse them with session tokens.
  • /api/node/* (node edition) — control-plane internal, fleet-secret gated.

11. Push notifications

GET /api/push/config{publicKey, transports} where transports ⊆ ['webpush','apns','fcm'], filtered to what this server can actually deliver. This is the source of truth — check it before asking the OS for notification permission.

EndpointNotes
POST /api/push/devices{token, transport: 'apns'|'fcm'}201. Token validated by shape only. 503 if the transport isn't configured server-side
DELETE /api/push/devices{token}call before POST /api/auth/logout, while the session still authenticates
POST/GET/DELETE /api/push/subscriptionsWeb Push (VAPID) subscription CRUD
POST /api/push/heartbeat{endpoint} — Web Push liveness

⚠ Native push is effectively first-party-only today

APNs/FCM delivery requires the server to hold push credentials for the specific app a device token belongs to — the .p8 signing key + bundle ID for iOS, the Firebase service account for Android. The hosted lurker.chat cells carry the official Lurker apps' credentials; therefore:

  • Official clients + app.lurker.chat: push works.
  • Official clients + self-hosted: no native push — the instance doesn't (and can't) have the official signing keys. GET /api/push/config on a self-hosted box reports ['webpush'] at most.
  • Third-party clients: no native push anywhere. This is structural, not a policy gate: registration isn't checked against client identity, so on hosted your POST /api/push/devices may return 201 — and delivery will then silently fail, because the server signs with the official app's credentials and APNs/FCM reject tokens belonging to a different app. Don't burn hours debugging this; it cannot work.

A notification relay for self-hosters (a Lurker-operated proxy the official apps could receive self-hosted pushes through — IRCCloud/Bitwarden style, or something UnifiedPush-shaped that could also serve third-party clients) is a design idea only. It is not built and not a commitment; do not architect against it.

What third-party clients can do today: Web Push works on any instance for browser-based clients; a TUI doesn't need push at all (it only matters when no client is attached — and presence gating already suppresses push while your client is visible). A native third-party app should treat "no push" as its baseline and lean on the ?since= resume for fast catch-up.

Notification payload detail (for official-app parity): networkId and target ride at the payload top level beside aps/data for tap-routing, and pushes revise the OS badge — your client recomputes the true badge from read-state (§9.4).


12. A minimal viable client

What the iOS app actually ships with (FrameParser.parseWs, LurkerStore.reduce) — a useful floor for a v1:

  • Receive frames (7): snapshot, backlog, history, irc, read-state, send-result, error — everything else safely ignored.
  • irc types rendered: message, action, notice, error, system, join, part, quit, nick, kick, mode, topic, motd, invite (plus channel-topic for state). Unknown → drop.
  • Send verbs (8): presence, send, history (before/latest), mark-read, mark-all-read, join, open-buffer, close-buffer.
  • REST (4): POST /api/auth/login/token (or the CP login), GET /api/networks, POST /api/auth/logout, and optionally GET /api/push/config.

Known first-party gaps at this tier (fine to share): nicklist only as of last snapshot (no live names/member-update patching), no typing/peer-presence, no slash commands. Slash commands are parsed client-side — the server does not interpret / in send text. Either implement a command table (translate to typed verbs; fall back to {type:'raw'} for the rest — see MessageInput.vue:2670 for the reference table) or expose UI verbs directly like iOS does. /ns//cs style credential commands should go over raw without local echo.

Suggested build order: config check → token mint → GET /api/networks → socket

  • snapshot burst → render buffers/shells → hydrate on open → send with echo rendering → mark-read → resume cursor → reconnect policy → the §9 rules as you hit them.

13. Source-of-truth map

AreaFiles
WS upgrade, auth, snapshot burst, verb dispatch, fan-outserver/services/wsHub.ts
IRC event production, network snapshot blobserver/services/ircConnection.ts, server/services/ircManager.ts
Protocol version & compat rulesserver/protocol.ts, server/routes/config.ts
Message shape & id sequenceserver/db/messages.ts
Session auth (cookie + Bearer)server/middleware/auth.ts, server/routes/auth.ts, server/db/sessions.ts
REST routersserver/routes/*.ts (mounted in server/app.ts)
Shared portable datashared/settingsRegistry.ts, shared/urlPattern.ts
Reference web clientvue_client/src/composables/useSocket.ts, vue_client/src/stores/buffers.ts, vue_client/src/components/MessageInput.vue
Reference native clientlurker-ios/LurkerKit (LurkerClient.swift, LurkerStore.swift, FrameParser.swift)

Released under the MPL-2.0 License.