λ

Igropyr

A distributed backend framework where crashes self-heal, code hot-swaps, faults speak a protocol, and continuations drive conversations.

Pure Chez Scheme · Erlang-style actors · libuv event loop · MIT

$ npm i igropyr
01 · Fault tolerance

Let It Crash

HTTP requests run in a supervised worker pool. Handlers don't defend—they crash, and the system recovers. (WebSocket sessions own their own processes, so a crash kills only that connection, leaving the pool untouched.)

Crashes self-heal

  • A crashed worker is replaced instantly. The task is seamlessly retried on a fresh worker, up to 3 times, before the client ever sees an error.
  • A worker stuck for over 30s—even in a deadlocked CPU loop—is ruthlessly killed and replaced. Preemptive scheduling guarantees that no single handler can freeze the server.
  • A half-sent, slow-drip request parks only its own reader process and is quietly reaped by a timeout. Other connections never notice.

Write the happy path. The supervisor owns the sad one.

(app-get app "/crash"
  (lambda (req res)
    ;; the worker dies; the supervisor retries on a
    ;; fresh worker -- the pool refills itself
    (raise 'handler-crashed)))

(app-get app "/stuck"
  (lambda (req res)
    ;; a hot loop cannot freeze the system: preemption
    ;; keeps serving, the ticker kills this worker
    (let loop ((n 0)) (loop (+ n 1)))))
02 · Live systems

Hot code swapping

Swap the entire handler—or just patch a single route—on a live server. The TCP listener, open connections, and the worker pool remain completely untouched. In-flight requests gracefully drain on the old code, while new traffic instantly hits the new logic.

Deploy without a restart

Routes live in a mutable registry behind the worker pool. Re-registering a path patches it atomically for the very next request; http-swap! replaces the entire top-level handler with the exact same guarantee.

Combined with graceful shutdown (http-shutdown! safely drains in-flight work) and SO_REUSEPORT multi-process listening, zero-downtime operation isn't an engineering project—it's the default.

(app-get app "/version"
  (lambda (req res) (send-text! res "v1")))

;; hit /upgrade on the LIVE server:
(app-get app "/upgrade"
  (lambda (req res)
    (app-get app "/version"          ; re-register =
      (lambda (req res)                ; hot replace
        (send-text! res "v2 (hot swapped)")))
    (send-text! res "upgraded")))
03 · The remote retry ring

Faults speak a protocol

When retries are exhausted or a stuck worker is killed, Igropyr doesn't just throw a black-box 500 error—it tells the client exactly what happened, on a connection that stays open.

Killed first, told after

The on-failure hook returns a structured JSON fault only after the stuck worker is physically dead. When the client hears stuck, it comes with an absolute guarantee: there is no execution left in flight. The state is definite.

  • crash — retries exhausted; safely resubmit with changed parameters, or compensate.
  • stuck — killed mid-flight; safely resubmit carrying state, or roll back.

Keep-alive survives the fault. The client resubmits on the very same connection and gets a fresh retry round. Dial down the stuck-ms limit, and a user who once stared at a spinner for 30 seconds now transparently cycles through several informed retries in the exact same time. Failures become invisible at the UI.

(app-listen app 8080
  `((stuck-ms . 3000)          ; fail fast
    (check-ms . 1000)
    (on-failure . ,(make-fault-handler))))

;; the client receives, connection kept alive:
;;   {"fault":"crash","attempts":4,"retryable":true}
;;   {"fault":"stuck","elapsed-ms":3012,...}
;; unset? the plain 500 remains. zero breakage.
04 · Web programming with continuations

Conversations are continuations

A multi-request workflow—a checkout wizard, a booking, a fund transfer—runs as one single green process. Its local lexical bindings are the conversation state. This allows it to hold resources that a disconnected session store could never serialize: like a live, open database transaction spanning across multiple network roundtrips.

Control flow is program text

"The user is at the confirm step" literally means the process is parked at that exact line of code. A sequence of events that the code cannot express simply cannot happen—there is no external state machine to get wrong, and no replay attack to defend against.

The gone guarantee

The transaction declares its point of no return through the commit! primitive, giving the framework a razor-sharp boundary to judge any death.

  • Before commit!: A dead process means a dropped connection, which means the database itself automatically rolled back. The framework answers gone—absolute physical proof that nothing committed. This is the one status you may safely retry on.
  • After commit! (or an unknown kill/missing record): The framework answers unknown. It refuses to guess. Reconcile; never resubmit.

Combined with the fault protocols above, the client always knows the definite server state. Just as importantly, it is told plainly—and honestly—when the state cannot be known. The complete remote transaction ring is finally closed.

(conversation-start!
  (lambda (req suspend! commit!)
    (let ((tx (begin-tx!)))       ; live, across requests
      (guard (e (#t (rollback! tx) (raise e)))
        (let ((req2 (suspend! confirm-page)))
          (commit! (lambda () (commit-tx! tx)))
          done))))
  req)

(conversation-resume! id token req)
;; => (values reply next-token) | 'stale | 'gone
;; the token names the reply you are answering;
;; repeating one replays its answer, once.
;; commit through commit!: a failure after it is
;; 'unknown, never the retryable 'gone.
05 · Scheme talks to Scheme

Communicate in S-expressions

When the client is Scheme too, requests and replies are pure s-expressions. There is no codec to design, agree upon, or debug. (igropyr sexpr) acts as the safe boundary parser, while app-rpc dispatches exactly one datum per message—seamlessly across HTTP, WebSocket, or SSE.

No codec on the wire

Exact ratios and bignums cross the network intact. There is no lossy JSON floating-point approximation anywhere in the stack. A call like (rpc "/rpc" '(add 1 2 1/2)) comes back as (ok 7/2)—the mathematical ratio perfectly preserved.

The WebAssembly peer

The natural partner to this backend is Goeteia, a Scheme compiler running natively in WebAssembly. Its browser-side (web rpc), (web ws), and (web sse) modules speak this exact same wire format, turning the browser into a first-class Scheme runtime.

This very site is written in pure Scheme and compiled down to bare HTML and CSS. That honeycomb fire effect above? It is compiled and rendered in real time, directly in your browser, by Goeteia.

;; Igropyr: one s-expression per message
(app-rpc app "/rpc"
  `((add      . ,(lambda (args) (apply + args)))
    (get-user . ,(lambda (args) (find-user (car args))))))

;; a Scheme browser -- Goeteia -- calls it. no JSON, no codec:
(rpc "/rpc" '(add 1 2 1/2))

;; the Igropyr server returns
(ok 7/2) ;; -- exact ratio intact
06 · From node to hive

Write once, run distributed

Nodes discover each other and wire up a full, true mesh—no central coordinator, and no fragile registry to babysit. Links self-heal, and work fluidly spreads across every live member.

Self-expanding distributed cluster

Point cluster-start at a discovery strategy, and it keeps the topology honest: it actively dials any member it isn't linked to yet, and mercilessly drops anyone that leaves. The static strategy uses a fixed list; with redis, nodes heartbeat themselves into a transient set. If a node stops beating, it simply falls out of the mesh. There is no central bookkeeping to drift out of sync.

Secure, name-based routing

Underneath lies a pure node-to-node distribution layer. A mutual HMAC-SHA256 handshake strictly gates who may join. Once inside, rsend and rcall reach registered processes on remote machines purely by name. monitor-node watches members come and go, while the links themselves stubbornly reconnect through network blips.

Distributed execution pools

(igropyr dpool) rides on top of this mesh. Submit a task, and it lands on an available live node. If that node suffers a physical death mid-execution, the mesh notices, and the work instantly reappears elsewhere. You get a guaranteed at-least-once execution primitive, seamlessly stretched across the entire cluster.

(node-start! 'web-1 secret 8888 "0.0.0.0")

;; discover peers via redis; nodes heartbeat
;; themselves in and expire on their own
(cluster-start
  `((name . "render-farm")
    (discover . (redis ,conn "10.0.0.1" 8888))))

;; fan work across every live member; a node
;; dying mid-task -> the task reruns elsewhere
(define pool (dpool-start '(web-1 web-2 web-3) 'render))
(dpool-await pool
  (dpool-submit pool #(resize "x.png" 800)))
Foundations

What it stands on

λ

Pure Chez Scheme

Every line is Scheme — R6RS libraries in .sc, no C shim. libuv, zlib and the crypto for MySQL auth are reached through Chez's FFI directly. Whole-program compilation folds the framework and your app into one optimized binary.

Erlang-style actors

Green processes with spawn / send / receive, link and monitor, a process registry, gen-server and PubSub. One OS thread, preemptive scheduling, pure message passing — no shared state, no locks.

Async on libuv

One event loop feeds thousands of parked processes. DNS, file reads and database round-trips park the calling process, never the thread. Non-blocking HTTP/WebSocket clients and Redis, MySQL and PostgreSQL drivers included.

150k+
req/s, keep-alive, M4 Pro
0
failed requests under ab -c 500
≤35s
full recovery from a stuck pool
1
OS thread
What comes with it
Core / framework split, like Node and Expressthe core exposes one entry point, (http-listen port (lambda (req res) ...)); the bundled (igropyr express) layer (create-app, app-get, send-json!, ...) is optional, and alternative frameworks can be built on the same core
Green processesthousands of lightweight processes scheduled over one OS thread; continuation-based context switching with preemption, so even a CPU-spinning handler cannot freeze the system
Pure message passingspawn / send / receive / link / monitor; no shared state between processes
Fault tolerant by defaulta fixed worker pool behind a supervisor: crashed workers are replaced and the task retried (at most 3 times, then the client gets a 500); workers stuck for more than 30 s are killed and replaced; a slow or half-sent request only ever blocks its own reader process
Failure hook (remote retry ring)when retries are exhausted or a stuck worker is killed (killed first, so no execution is in flight), an optional on-failure handler answers a structured JSON fault instead of the plain 500, on the same keep-alive connection — the client resubmits (changed parameters, carried state) and gets a fresh retry round; unset, the plain 500 remains
Conversations (process-per-dialogue)a multi-request dialogue runs as one green process holding live state — even an open database transaction — across rounds; suspend! answers and parks, conversation-resume! continues — carrying a token that names the reply it answers, so a double click or a retried request replays that answer instead of taking a step nobody asked for; the transaction commits through commit!, so a death that left the flow before it is the rollback guarantee — a later resume gets gone and may be retried — while one after it is unknown, which may not
Hot code swappingreplace the handler (or individual routes) on a live server: the listener, open connections and worker pool stay up, in-flight requests finish on the old code
WebSocketRFC 6455 upgrade on the same port; each socket is its own green process, so server push is just a message send
Streaming responses & SSEchunked response body via res-begin!/res-write!/res-end!; Server-Sent Events helpers on top
OTP building blocksgen-server (call/cast/info), a process registry (register/whereis), and topic PubSub with automatic cleanup of dead subscribers
JSONa safe recursive-descent parser (no read; full escape and surrogate handling) and writer
S-expression RPCwhen the peer is also Scheme there is no codec: (igropyr sexpr) is a safe whitelisted parser (no read, depth-limited), and app-rpc / send-sexpr! / ws-send-sexpr! / sse-send-sexpr! carry one datum per message — exact ratios and bignums cross intact. The browser end is Goeteia's (web rpc/ws/sse)
Forms & cookiesreq-form parses urlencoded and multipart bodies (file uploads included); req-cookie / set-cookie!
Middleware suitecookie sessions (gen-server store, CSPRNG sids), CORS with preflight, security headers, and an access logger
Chunked transfer-encodingTransfer-Encoding: chunked request bodies are decoded transparently
Non-blocking Redis, MySQL and PostgreSQL clientspure Scheme, same event loop; callers park their green process while the OS thread keeps serving; both SQL drivers come with a self-healing connection pool
Non-blocking HTTP & WebSocket clientsoutbound http-get / http-post and ws-connect, both with async DNS (libuv thread pool) and the same park-the-caller model
Static file serving & streaminghot files come from an in-memory cache — a hashtable lookup, no disk read and no stat syscall (mtime re-checked at most once a second). A cache miss reads once on libuv's thread pool, so a cold read never blocks the scheduler; files over 1 MiB stream in bounded chunks with backpressure (constant memory, no GC traffic), never read whole
gzip compressionresponses negotiated via Accept-Encoding; static files cache their compressed form
Ops-readyrate limiting, a global error handler, and a Prometheus /metrics endpoint
Runtime introspection & graceful shutdownhttp-stats (live connection/request/pool counters), http-shutdown! (drain in-flight requests, refuse new connections)
Multi-process scalingSO_REUSEPORT bind option for kernel-balanced multi-process listening on Linux (pair with pm2 or systemd)
HTTP/1.1 keep-alive & pipeliningpersistent connections by default on 1.1; each connection's reader process loops over successive requests
Fast~150 k req/s with keep-alive at 100 connections, and ~32 k req/s at 500 concurrent connections (ab -n 50000 -c 500, zero failed requests), on an Apple M4 Pro
Acknowledgements

Built on the shoulders of others

Igropyr is built on Chez Scheme — the fastest Scheme compiler, with a first-class FFI that reaches libuv directly. With deep gratitude for Kent Dybvig's life work, and to Cisco for open-sourcing it.

The primary inspirations: Node.js is the event-loop server on libuv, and the lean core / optional-framework split that Node and Express made the norm. The actor model, the supervisor, and Let It Crash come from Erlang/OTP; Swish — a Chez Scheme system built on those ideas — was the concrete blueprint for the scheduler, the receive macro, and the supervisor. The conversation model is the actor-native take on web programming with continuations — a great idea from the Scheme and functional-programming community.