I/O

<!-- audited: 2026-09-23 -->

All I/O in Elle is async — reads and writes yield to the scheduler. User code runs inside the async scheduler automatically.

I/O backend

On Linux, Elle uses io_uring for all I/O: file reads, writes, TCP, timers, subprocess pipes. Operations are submitted to the kernel's submission queue and completed without syscalls or threads — the kernel handles multiplexing directly. A single-threaded event loop drains any ready completions, then blocks on the completion queue (waiting for at least one completion) and resumes the waiting fiber.

On macOS, Elle uses a thread-pool backend that provides the same abstraction. Blocking I/O operations run on background threads; the event loop collects results and resumes fibers identically. User code sees no difference — the same port/open, port/read-line, ev/spawn API works on both platforms.

Both backends are syscall-free from the fiber's perspective: the fiber yields :io, the scheduler submits the operation, and the fiber resumes with the result. No threads are created per-operation on Linux; on macOS, the thread pool is shared across all fibers.

Whatever cannot lift to io_uring — getaddrinfo, an arbitrary Task closure, blocking stdin reads, and everything on the thread-pool platform — runs on a single shared thread pool whose every worker reports through one completion channel. On the thread-pool platform the scheduler's blocking wait is a recv() on that channel. On Linux it blocks on one io_uring_enter instead, and a bridge eventfd carries the channel's completions into it: a worker raises the eventfd after publishing, and a standing poll on the ring turns that edge into a completion. Either way the scheduler has exactly one blocking primitive, so a completion published by a worker can never be missed while the scheduler is asleep.

Backend teardown

An io_uring submission queue entry references memory the kernel reaches asynchronously: a BufferPool slot for read-all, the fiber's own region buffer for read, read-line and read-exact, and the payload's region pages for write. The kernel may complete the operation, and write or read that memory, at any point up until its completion is reaped. So a backend must never be torn down while an operation is still in flight: freeing that memory with the kernel still holding a pointer lands the eventual write in freed heap (manifesting as a malloc(): unsorted double linked list corrupted abort).

Dropping an async backend therefore first brings the ring to a quiescent state — it cancels every pending io_uring operation and drains the resulting completions, so no kernel-owned buffer outlives the backend. This matters when user code submits work it never waits for, e.g. (io/submit backend req) with no following (io/wait backend …): the operation is in flight when the backend value goes out of scope.

A thread-pool worker reads into the fiber's buffer and writes from the payload's pages the same way, so the drop also stops every pool operation that carries a stop pipe and waits for its completion. The stdin worker reads into a buffer of its own and hands it over through the channel, so it needs no wait. Where a stream operation's bytes live holds the argument.

A backend the program never lets go of — a top-level (io/backend :async), or the scheduler's own — is still there when the heap that carries it is torn down. The heap brings each one to a quiescent state before it frees its regions, so the drain reads values that are still allocated and the operations still in flight let go of what they hold while there is something to let go of. Without that order the backend's own drop would run inside the teardown sweep, after the sweep had freed part of what it holds.

Cancelling an operation

ev/timeout cancels an operation on every call — the body's or the timer's, whichever lost — so cancellation runs constantly rather than at the edges. Two things come back when it does, on either backend:

• The worker. A thread-pool operation runs on a worker thread. A

cancelled operation reports completion like any other, so its worker goes back to the pool for the next operation; only the result is thrown away. (ev/report):workers counts the operations out right now.

• The descriptor. A cancelled read stops rather than going on

reading, and so does a cancelled write whose peer stopped taking bytes. Whatever arrives next belongs to whoever reads the port next. Here the peer is a shell that waits 0.3 s before it writes:

lisp (let [peer (subprocess/exec "/bin/sh" ["-c" "sleep 0.3; printf late"]) out (get peer :stdout)] (assert (nil? (ev/timeout 0.1 (fn [] (port/read out 64)))) "the deadline wins, and ev/timeout answers nil") (assert (= (bytes "late") (port/read out 64)) "the next read still sees the peer's bytes") (subprocess/wait peer))

And a port that goes away while an operation still runs — closed, or released with the regions of the fiber that opened it — keeps its descriptor number until that operation ends, so the number cannot be handed to a new port while a worker holds it.

io-cancel-releases.lisp pins both. See an operation in flight for how a cancelled operation is ended, and descriptors and workers for what it gives back.

How many operations run at once

However many the OS allows. The thread-pool backend runs each operation on a worker thread and starts one whenever no worker is free, so the ceiling is RLIMIT_NPROC, kernel.threads-max and the memory for the stacks; when the OS refuses a thread, port/read and friends signal that refusal rather than the runtime pre-empting it with a smaller number of its own. io_uring runs its operations in the kernel and has no such ceiling.

A worker that finishes an operation waits for the next one instead of exiting, and retires after ten idle seconds. So a program that keeps asking for I/O pays for a thread once rather than per operation, and one that stops asking stops paying.

io-keepalive is that wait, in seconds. A scheduler reads it when it makes its backend, so bind it around the ev/run whose I/O it should govern; 0 turns reuse off, and every operation then starts and ends a thread of its own.

(assert (= :done (parameterize ((*io-keepalive* 0))
                    (ev/run (fn [] (ev/sleep 0) :done))))
        "bind *io-keepalive* around the ev/run whose backend it governs")

(io/workers backend) reports how many operations a backend has out — the workers busy right now, not the ones waiting for work — and ev/report carries the running scheduler's count as :workers.

Ports

Ports are bidirectional file descriptors. Open with port/open, close with port/close.

Port operations

(with-temp-dir dir
  (let [path (path/join dir "lines.txt")]
    (let [w (port/open path :write)]
      (port/write w "one\ntwo\n")
      (port/close w))
    (let [r (port/open path :read)]
      (assert (= "one" (port/read-line r)) "read-line drops the newline")
      (assert (= 4 (port/tell r)) "tell counts what was consumed, not the kernel offset")
      (assert (= "two" (port/read-line r)))
      (assert (nil? (port/read-line r)) "nil at EOF")
      (assert (= 0 (port/seek r 0)) "seek answers the new position")
      (assert (= "one" (port/read-line r)) "and reads from there")
      (assert (= "two\n" (port/read-all r)) "read-all takes the rest")
      (port/close r)
      (port/close r))))                  # closing twice is harmless

port/write writes every byte

port/write returns the length of the data you gave it. The caller never loops on the return value.

One write(2) transfers only what fits in the fd's send buffer at that moment. On a socket that is often far less than the payload: a 4 KiB send buffer accepts about 21 KB of a 200 KB write, and a default one accepts a few megabytes of an 8 MB write. The backend therefore resubmits from the byte after the last one accepted, and completes the operation only when the whole payload is gone. port/read is the deliberate opposite — it returns "up to n bytes" per POSIX, and port/read-exact is its all-or-nothing sibling.

If the fd fails part-way through, port/write raises the error rather than returning a short count. An unknown prefix of the payload reached the peer in that case, the same guarantee write(2)-loop helpers give elsewhere.

The pinning tests are port-shortwrite.lisp and port-shortread-framing.lisp for the read direction.

A read that overshoots keeps the rest for the same port

port/read-line stops at the newline and port/read-exact stops at the Nth grapheme, but the kernel read behind each of them takes a whole block. The backend holds the extra bytes and serves the next read on that port from them before it goes back to the kernel, so no byte is lost between two reads and port/tell reports the logical position rather than the kernel offset.

The remainder belongs to the port that produced it, not to its descriptor number. A closed descriptor's number goes straight back to the OS — and a port dropped without port/close still closes its descriptor — so the next port/open can be handed that number. It starts with an empty remainder, whichever port held the number before it. The pinning test is io.lisp § "a recycled descriptor number carries no remainder".

What a read reserves before it runs is not a bound on what it answers with. port/read-line reserves 64 KiB, which covers every real protocol line; a line longer than that is answered in pieces, and reading on gives the next piece until the newline arrives. No byte is dropped to make an answer fit — the backend has already taken those bytes from the kernel, so there would be nothing left to read them again. port-longline.lisp pins it, and port_longline_threadpool (modes.rs) pins it on the other backend.

On a text port port/read-exact counts grapheme clusters, and a cluster has no upper bound in bytes: one emoji built from four people and three joiners is 25 bytes and one grapheme. So the byte length of n clusters is not known until the bytes arrive. The backend reads in chunks, holds what it has, and sizes the result to what it turns out to be — the count in the request bounds the answer in clusters and says nothing about its bytes. A read-exact that follows an over-reading read-line is the same story: the held remainder joins the bytes this read produces, the first n clusters of the join are the answer, and the rest goes back to the remainder for the next read on that port. port-text-framing.lisp pins all three, and port_text_framing_threadpool (modes.rs) pins them on the other backend.

:timeout bounds each operation

Every port call takes an optional :timeout in milliseconds, and it bounds each kernel operation rather than the whole call.

Most calls are a single operation, so the two readings agree. They part company on the calls that loop — port/write until the payload is gone, port/read-exact until its count, port/read-all until EOF, and port/read-line until a newline. There, a peer that has stopped trips the deadline, while a peer that is merely slow keeps making progress and the call finishes however long that takes.

(defn listener-port [listener]
  "The port number a listener bound to an ephemeral port received."
  (let [path (port/path listener)]
    (parse-int (slice path (+ 1 (string/find path ":"))))))

(defn with-quiet-peer [body]
  "Run `body` against a peer that accepts the connection and then neither
   reads nor writes. A small send buffer makes it stall the writer quickly."
  (ev/run (fn []
            (let [listener (tcp/listen "127.0.0.1" 0)]
              (ev/spawn (fn []
                          (let [conn (tcp/accept listener)]
                            (ev/sleep 2)
                            (port/close conn))))
              (let [conn (tcp/connect "127.0.0.1" (listener-port listener)
                                      :sndbuf 4096 :timeout 5000)]
                (defer (begin (port/close conn) (port/close listener))
                  (body conn)))))))

(defn timed-out? [thunk]
  "True when `thunk` signals the :timeout error."
  (let [[ok? err] (protect (thunk))]
    (and (not ok?) (= (get err :error) :timeout))))

# The peer is connected and sends nothing, so the read gives up at its deadline.
(assert (with-quiet-peer (fn [conn]
                           (timed-out? (fn [] (port/read-line conn :timeout 200)))))
        "a peer that sends nothing must trip the read's deadline")

# The peer never reads, so the send buffer fills and the write gives up too.
(assert (with-quiet-peer
          (fn [conn]
            (timed-out? (fn []
                          (port/write conn (bytes (string/repeat "x" 2000000))
                                      :timeout 200)))))
        "a peer that never reads must trip the write's deadline")

A peer that is merely slow is the opposite case, and it is what the per-operation reading buys. Every gap here stays inside the deadline while the whole call runs well past it:

(ev/run (fn []
          (let* [listener (tcp/listen "127.0.0.1" 0)
                 chunk 4096
                 chunks 20
                 gap 0.05
                 deadline 500]
            (ev/spawn (fn []
                        (let [conn (tcp/accept listener)]
                          (repeat chunks
                                  (port/write conn (bytes (string/repeat "z" chunk)))
                                  (ev/sleep gap))
                          (port/close conn))))
            (let* [conn (tcp/connect "127.0.0.1" (listener-port listener)
                                     :timeout 5000)
                   want (* chunk chunks)
                   started (clock/monotonic)
                   got (port/read-exact conn want :timeout deadline)
                   elapsed (- (clock/monotonic) started)]
              (port/close conn)
              (port/close listener)
              (assert (= (length got) want)
                      "a slow peer still delivers every byte")
              # The read returns once the last chunk lands, after `chunks - 1`
              # gaps — so this bound sits between the deadline and that total,
              # and the gap sits an order of magnitude under the deadline. Tie
              # either margin closer and a loaded scheduler's overshoot on one
              # `ev/sleep` trips the deadline this example exists to survive.
              (assert (> elapsed 0.5)
                      "and the call outlives its own :timeout while doing it")))))

Both readings stop a hang; only the per-operation one keeps a slow transfer working. port/read is unaffected either way: it is a single "up to n bytes" operation.

The bound covers every kind of port. A socket peer that stops reading, a child process that never reads its stdin, and a fifo nobody opens for reading all stall the same way, and :timeout returns from all three.

# The child never reads its stdin, so the pipe buffer fills and the rest of
# the payload has nowhere to go.
(ev/run (fn []
          (let [child (subprocess/exec "sleep" ["30"])]
            (assert (timed-out? (fn []
                                  (port/write (get child :stdin)
                                              (bytes (string/repeat "x" 1000000))
                                              :timeout 200)))
                    "a child that never reads must trip the write's deadline")
            (subprocess/kill child :sigterm)
            (subprocess/wait child))))

The pinning tests are port-write-timeout.lisp and port-read-timeout.lisp, both run on each backend, each covering a socket peer and a pipe peer.

The calls that wait for a peer

tcp/accept, unix/accept, tcp/connect, unix/connect and udp/recv-from take the same :timeout, and they need it most: each waits on a peer that may never appear. A listener nobody calls, a datagram socket nobody sends to, and a connect to an address that drops the packet all wait alike.

(ev/run (fn []
          (let [listener (tcp/listen "127.0.0.1" 0)]
            (assert (timed-out? (fn [] (tcp/accept listener :timeout 200)))
                    "an accept nobody calls must trip its own deadline")
            (port/close listener))))

ev/timeout and io/cancel end these calls too, on either backend. The pinning tests are net-wait-timeout.lisp for the deadline and the a_cancelled_pool_ tests in netcancel.rs for the cancellation.

The calls that wait for something other than a peer

Four more calls wait on something that may never happen, and each takes the same two endings — its own :timeout where it has one, and ev/timeout or io/cancel from outside.

ev/poll-fd answers an expired wait with 0 rather than signalling, which is what lets a caller poll in a loop — wayland.lisp polls with a 33 ms bound on every iteration.

A cancelled subprocess/wait is the one that gives something back besides the fiber: it may have reaped the child on its way out, and a reap takes the exit status from the kernel for good. subprocess.md owns what becomes of that status.

port/open is the one with a direction to it. POSIX blocks an open(2) on a fifo until the other end is open, and Elle keeps that for the write side: the open waits for a reader, and :timeout bounds the wait. On the read side the port comes back at once and the first port/read is what waits for a writer, which is where a reader's :timeout applies anyway.

Each of these is pinned in io-cancel-releases.lisp, which measures what a cancelled operation gives back, and in park.rs.

Streams from ports

A stream is a fiber. Each of these closes the port when the stream ends.

(with-temp-dir dir
  (let [path (path/join dir "stream.txt")
        w (port/writer (port/open path :write))]
    (fiber/resume w)                     # run it to its first wait
    (each s ["one\n" "two\n" nil] (fiber/resume w s))   # nil closes the port
    (assert (= (list "one" "two") (stream/collect (port/lines (port/open path :read))))
            "the lines come back without their newlines")))

Closing stdin

(port/close stdin) is supported and does what you'd expect:

• Any in-flight (port/read-line stdin) / (port/read …) /

(port/read-all stdin) is cancelled. The waiting fiber resumes with an :io-error whose message is stdin closed.

• The dedicated stdin worker thread (which sits on read(2) against

fd 0) is signalled via an internal self-pipe, returns from its syscall, drains any further pending requests as cancelled, and exits cleanly. No leaked OS thread.

• Subsequent stdin reads error from the port is closed check

in AsyncBackend::submit.

• The OS file descriptor for stdin is not itself closed (the

stdio ports never owned it). This matches the existing stdout / stderr close semantics.

Output

(print "no newline")
(println "with newline")
(println "count: " 42)         # multiple args concatenated
(eprint "to stderr")
(eprintln "error: bad input")
(pp {:a [1 2 3]})              # pretty-print data structures

All output functions are async — they yield to the scheduler. stdout and stderr are dynamic parameters that can be rebound.

Subprocesses

subprocess/exec spawns a child and answers a subprocess — the value its streams, pid and exit status are read from, and the value subprocess/wait and subprocess/kill take. Its stdio ports are ordinary ports, so everything above applies to them: the same reads and writes, the same :timeout, the same cancellation.

See subprocess.md for the type, its reads, and what a kill answers.

Temporary files

file/mktempdir creates a uniquely-named directory under the platform temp root and returns its path. The root is the runtime's native temp location — TMPDIR on Unix (the per-user folder on macOS), %TEMP% on Windows — so scripts never hardcode /tmp; point TMPDIR at a tmpfs such as /dev/shm to keep scratch I/O in RAM. Uniqueness is made in the runtime (pid + counter, retried on collision), so concurrent processes cannot race each other to the same name the way fixed scratch filenames do.

with-temp-dir scopes one to a body and deletes it on the way out — recursively, error or not. file/delete-dir-all is the underlying recursive delete (file/delete-dir only removes empty directories).

(with-temp-dir dir
  (file/write (path/join dir "scratch.txt") "data")
  (assert (= (file/read (path/join dir "scratch.txt")) "data")))

System args and environment

# sys/args returns args after the source file
(def args (sys/args))

# Environment
(sys/env)              # => struct of all env vars
(sys/env "HOME")       # => single var, or nil

See also

• subprocess.md — the subprocess type, its reads, kill and wait
• behaviors.md — supervised subprocesses, GenServer, actors
• concurrency.md — ev/spawn, ev/join, parallel I/O
• fibers — fiber-based async model
• strings.md — string operations