Guile Knots


Overview

Guile Knots is a library providing tools and patterns for programming with Guile Fibers. Guile Knots provides higher level building blocks for writing programs using Guile Fibers, including managing code that can’t run in a thread used by fibers. Also included is a web server implementation using Fibers, which while being similar to the web server provided by Fibers, can provide some benefits in specific circumstances.

Table of Contents


1 API

The following is the list of modules provided by this library.


1.1 (knots)

1.1.1 Procedures

Procedure: call-with-default-io-waiters thunk

Run THUNK with Guile’s default blocking I/O waiters active.

This is useful when restoring the default Guile I/O waiters from within a context (like Fibers) where different I/O waiters are used, for example when creating a new thread from a fiber.

Procedure: call-with-knots-stack-wrapping thunk

Run THUNK, and if it raises an exception, wrap that exception in a &knots-exception capturing the current stack and re-raise the combined exception.

The stack is captured non-unwindingly, so the raise-site frames are preserved for later display by print-backtrace-and-exception/knots. The wrapped exception is re-raised in the same dynamic context, so callers are expected to install an outer with-exception-handler (typically with #:unwind? #t) to handle it.

This is the primitive used by the fiber and thread entry points in Knots. Users writing their own fiber or thread wrappers should use this procedure rather than reimplementing the stack capture.

(with-exception-handler
    (lambda (exn)
      (print-backtrace-and-exception/knots exn))
  (lambda ()
    (call-with-knots-stack-wrapping
      (lambda ()
        (do-work))))
  #:unwind? #t)
Procedure: call-with-sigint thunk cvar

Run THUNK with a SIGINT handler that signals the Fibers condition CVAR. Restores the previous handler when THUNK returns.

Typical usage is to pass a condition variable to this procedure and wait on CVAR in a fiber to implement clean shutdown on Ctrl-C:

(let ((quit-cvar (make-condition)))
  (call-with-sigint
    (lambda () (wait quit-cvar))
    quit-cvar))
Procedure: call-with-temporary-thread thunk

Run THUNK in a temporary thread and return its result to the calling fiber.

Procedure: display/knots obj OPT: port

Write OBJ to PORT (default: current output port) as a UTF-8 byte sequence via put-bytevector.

When used with ports without buffering, this should be safer than display.

Procedure: format/knots port s . args

Like format but should be safer when used with a port without buffering.

Procedure: knots-exception-stack obj

Return the stack from a &knots-exception.

Procedure: knots-exception? obj

Return #t if OBJ is a &knots-exception.

Procedure: make-knots-exception a

Construct a &knots-exception with the given stack.

Procedure: print-backtrace-and-exception/knots exn KEY: #:port #:debug?

Print the backtrace and exception information from EXN to PORT.

It’s important to use print-backtrace-and-exception/knots for displaying backtraces involving functionality from Guile Knots, since the stack involved is potentially split across several fibers. The stacks involved are attached to the exception, and this procedure extracts this information out and assembles a backtrace including all the code involved.

The recommended pattern is to pair this with call-with-knots-stack-wrapping, which captures the raise-site stack onto the exception itself. The outer handler can then unwind freely:

(with-exception-handler
    (lambda (exn)
      (print-backtrace-and-exception/knots exn))
  (lambda ()
    (call-with-knots-stack-wrapping
      (lambda ()
        (do-things))))
  #:unwind? #t)
Procedure: simple-format/knots port s . args

Like simple-format but should be safer when used with a port without buffering.

Procedure: spawn-fiber/knots thunk OPT: scheduler KEY: #:parallel?

Spawn a fiber to run THUNK, with knots exception handling.

Accepts the same optional SCHEDULER and #:parallel? arguments as spawn-fiber.

Procedure: wait-when-system-clock-behind KEY: #:minimum-time #:check-interval

Block until the system clock reads at least MINIMUM-TIME (a Unix timestamp, defaulting to 978400800 — Jan 02 2001 02:00:00).

Useful at startup in environments (virtual machines, embedded systems) where the clock may start at or near the Unix epoch. Prints a warning to the current error port every CHECK-INTERVAL seconds (default 20) while waiting.

1.1.2 Record Types

Record type: &knots-exception

This record type has the following fields:

  • stack

1.2 (knots actors)

1.2.1 Procedures

Procedure: handle-channel-request channel handler KEY: #:exception-value

Undocumented procedure.

Procedure: make-channel-request channel args

Undocumented procedure.


1.3 (knots backtraces)

1.3.1 Procedures

Procedure: call-with-knots-stack-wrapping thunk

Run THUNK, and if it raises an exception, wrap that exception in a &knots-exception capturing the current stack and re-raise the combined exception.

The stack is captured non-unwindingly, so the raise-site frames are preserved for later display by print-backtrace-and-exception/knots. The wrapped exception is re-raised in the same dynamic context, so callers are expected to install an outer with-exception-handler (typically with #:unwind? #t) to handle it.

This is the primitive used by the fiber and thread entry points in Knots. Users writing their own fiber or thread wrappers should use this procedure rather than reimplementing the stack capture.

(with-exception-handler
    (lambda (exn)
      (print-backtrace-and-exception/knots exn))
  (lambda ()
    (call-with-knots-stack-wrapping
      (lambda ()
        (do-work))))
  #:unwind? #t)
Procedure: exception-frames/knots exn

Return a flat vector of <frame> objects representing the filtered backtrace associated with EXN, as derived from the &knots-exception wrappers attached to it.

The frames are ordered innermost-first like stack->vector). The set frames match what print-backtrace-and-exception/knots would display.

If EXN contains no &knots-exception wrappers, the returned vector is empty since this procedure doesn’t capture the current stack. Use this in combination with call-with-knots-stack-wrapping.

Procedure: knots-exception-stack obj

Return the stack from a &knots-exception.

Procedure: knots-exception? obj

Return #t if OBJ is a &knots-exception.

Procedure: make-knots-exception a

Construct a &knots-exception with the given stack.

Procedure: print-backtrace-and-exception/knots exn KEY: #:port #:debug?

Print the backtrace and exception information from EXN to PORT.

It’s important to use print-backtrace-and-exception/knots for displaying backtraces involving functionality from Guile Knots, since the stack involved is potentially split across several fibers. The stacks involved are attached to the exception, and this procedure extracts this information out and assembles a backtrace including all the code involved.

The recommended pattern is to pair this with call-with-knots-stack-wrapping, which captures the raise-site stack onto the exception itself. The outer handler can then unwind freely:

(with-exception-handler
    (lambda (exn)
      (print-backtrace-and-exception/knots exn))
  (lambda ()
    (call-with-knots-stack-wrapping
      (lambda ()
        (do-things))))
  #:unwind? #t)

1.3.2 Record Types

Record type: &knots-exception

This record type has the following fields:

  • stack

1.4 (knots non-blocking)

1.4.1 Procedures

Procedure: non-blocking-open-socket-for-uri uri KEY: #:verify-certificate?

Open a socket for URI and return it as a non-blocking port.

For HTTPS URIs the TLS handshake is completed while the socket is still blocking (required because Guile’s TLS wrapper does not support non-blocking handshakes), then the underlying socket is made non-blocking. For plain HTTP the socket is made non-blocking immediately.

#:verify-certificate? controls TLS certificate verification and defaults to #t.

Procedure: non-blocking-port port

Make PORT non-blocking and return it.


1.5 (knots notification)

1.5.1 Macros

Macro: notification? x

Return #t if OBJ is a <notification>.

1.5.2 Procedures

Procedure: make-notification

Return a new notification.

Waiting on a notification will trigger when the notification is notified. If a notification is notified when there is no waiter, the waiter will wake immediately when it next waits. Multiple notifications during this period will only lead to one wakeup.

Procedure: notify! notification OPT: scheduler

Signal NOTIFICATION. Returns #t if this call took responsibility for arranging a wakeup, or #f if this has already happened.

SCHEDULER is the fibers scheduler on which to deliver the wakeup; it defaults to (current-scheduler), so notify! must either be called from a fiber or be passed a scheduler explicitly.

Procedure: wait-for-notification notification KEY: #:timeout

Block until NOTIFICATION is signalled and return #t.

If TIMEOUT is a number, wait at most that many seconds and return #f if no notify arrives in time. A timed-out wait does not cancel an in-flight notify: it may still be delivered to a later waiter.

Procedure: wait-for-notification-operation notification

Return an operation that completes with #t when NOTIFICATION is signalled. Use this to compose a wait with other operations via choice-operation.


1.6 (knots parallelism)

1.6.1 Macros

Macro: fibers-let x

Let, but run each binding in a fiber in parallel.

Macro: fibers-parallel x

Run each expression in parallel. If any expression raises an exception, this will be raised after all exceptions have finished.

Macro: parallelism-limiter? x

Return #t if OBJ is a <parallelism-limiter>.

Macro: with-parallelism-limiter x

Evaluate EXP ... while holding a slot from PARALLELISM-LIMITER. Syntactic sugar around call-with-parallelism-limiter.

1.6.2 Procedures

Procedure: call-with-parallelism-limiter parallelism-limiter thunk

Acquire a slot from PARALLELISM-LIMITER, call THUNK, release the slot, and return the values from THUNK. Blocks if no slot is currently available.

Procedure: destroy-parallelism-limiter parallelism-limiter

Destroy PARALLELISM-LIMITER, releasing its underlying resource pool.

Procedure: fiberize proc KEY: #:parallelism #:input-channel #:process-channel

Convert PROC into a procedure backed by #:parallelism (default: 1) background fibers. Returns a wrapper that sends its arguments to one of the fibers and blocks until the result is returned.

#:input-channel is the channel that callers write requests to; defaults to a fresh channel. #:process-channel is the channel the fibers read from; defaults to #:input-channel. Setting them differently allows external parties to bypass the wrapper and write directly to process-channel.

Procedure: fibers-batch-for-each proc parallelism-limit . lists

Call PROC on LISTS, running up to PARALLELISM-LIMIT fibers in parallel.

Procedure: fibers-batch-map proc parallelism-limit . lists

Map PROC over LISTS in parallel, with a PARALLELISM-LIMIT. If any of the invocations of PROC raise an exception, this will be raised once all of the calls to PROC have finished.

Procedure: fibers-for-each proc . lists

Call PROC on LISTS, running up to 20 fibers in parallel.

Procedure: fibers-map proc . lists

Map PROC over LISTS in parallel, running up to 20 fibers in PARALLEL. If any of the invocations of PROC raise an exception, this will be raised once all of the calls to PROC have finished.

Procedure: fibers-map-with-progress proc lists KEY: #:report

Map PROC over LISTS, calling #:REPORT if specified after each invocation of PROC finishes. REPORT is passed the results for each element of LISTS, or #f if no result has been received yet.

Procedure: make-parallelism-limiter limit KEY: #:name #:scheduler

Return a parallelism limiter that allows at most LIMIT concurrent fibers to execute within with-parallelism-limiter at the same time. Further fibers block until a slot becomes free.

#:name is a string used in log messages. Defaults to "unnamed".


1.7 (knots promise)

1.7.1 Macros

Macro: fibers-promise? x

Return #t if OBJ is a <fibers-promise>.

1.7.2 Procedures

Procedure: fibers-delay thunk

Return a new fiber-aware promise that will evaluate THUNK when first forced. THUNK is not called until fibers-force is called on the promise.

Procedure: fibers-delay/eager thunk

Return a new fiber-aware promise and immediately begin evaluating THUNK in a new fiber. Exceptions during eager evaluation are silently discarded; they will be re-raised when fibers-force is called.

Procedure: fibers-force fp

Force the fiber-aware promise FP, returning its values.

The first call evaluates the promise’s thunk. Concurrent callers block on a condition variable until evaluation finishes, then receive the same result. If the thunk raises an exception, the exception is stored and re-raised for all callers.

Procedure: fibers-promise-reset fp

Reset the fiber-aware promise FP so that the next call to fibers-force re-evaluates its thunk.

Procedure: fibers-promise-result-available? fp

Return #t if the fiber-aware promise FP has been evaluated (successfully or with an exception) and #f if evaluation has not yet started or is still in progress.


1.8 (knots queue)

1.8.1 Procedures

Procedure: spawn-queueing-fiber dest-channel

Spawn a fiber that serialises items onto DEST-CHANNEL in FIFO order. Returns a new input channel.

Multiple producers can put items on the returned channel concurrently. The fiber buffers them locally and forwards them to DEST-CHANNEL one at a time, preserving arrival order.


1.9 (knots resource-pool)

1.9.1 Macros

Macro: resource-pool-channel x

Return the channel used by the resource pool.

Macro: resource-pool-configuration x

Return the configuration alist of the resource pool.

Macro: resource-pool-name x

Return the name of the resource pool.

Macro: resource-pool? x

Return #t if OBJ is a <resource-pool>.

Macro: with-resource-from-pool x

Evaluate EXP ... with RESOURCE bound to a resource checked out from POOL. Syntactic sugar around call-with-resource-from-pool.

1.9.2 Parameters

Parameter: resource-pool-default-timeout-handler

Default value:

#f

1.9.3 Procedures

Procedure: call-with-resource-from-pool pool proc KEY: #:timeout #:timeout-handler #:max-waiters #:channel #:destroy-resource-on-exception? #:delay-logger #:duration-logger

Call PROC with a resource from POOL, blocking until a resource becomes available. Return the resource once PROC has returned.

#:delay-logger is called as (delay-logger seconds) with the time spent waiting for a resource to become available. Defaults to the pool’s #:delay-logger if not specified.

#:duration-logger is called as (duration-logger seconds) after PROC completes, whether it returned normally or raised an exception. Defaults to the pool’s #:duration-logger if not specified.

Procedure: destroy-resource-pool pool

Destroy POOL, preventing any new checkouts. Blocks until all checked-out resources have been returned, running the pool’s #:destructor on each. Any fibers waiting for a resource receive &resource-pool-destroyed.

Procedure: make-fixed-size-resource-pool resources-list-or-vector KEY: #:delay-logger #:duration-logger #:scheduler #:name #:default-checkout-timeout #:default-max-waiters

Create a resource pool from RESOURCES-LIST-OR-VECTOR, a list or vector of pre-existing resource values.

Use with-resource-from-pool or call-with-resource-from-pool to borrow a resource and return it automatically when done.

Optional keyword arguments:

#:name

A optional string used in log messages. Defaults to "unnamed".

#:default-checkout-timeout

Default checkout timeout when requesting a resource from the pool, unset by default.

#:default-max-waiters

Maximum number of fibers that may queue waiting for a resource. When this limit is exceeded, &resource-pool-too-many-waiters is raised when a resource is requested. Defaults to #f (no limit).

#:delay-logger

Called as (delay-logger seconds) with the time spent waiting for a resource to become available. Defaults to #f (no logging).

#:duration-logger

Called as (duration-logger seconds) after the proc passed to call-with-resource-from-pool completes, whether it returned normally or raised an exception. Can be overridden per-call via the #:duration-logger keyword argument to call-with-resource-from-pool. Defaults to #f (no logging).

#:scheduler

The Fibers scheduler to use for the pool’s internal fiber. Defaults to the current scheduler.

Procedure: make-resource-pool return-new-resource max-size KEY: #:min-size #:idle-seconds #:delay-logger #:duration-logger #:destructor #:lifetime #:scheduler #:name #:add-resources-parallelism #:default-checkout-timeout #:default-max-waiters #:resource-creation-retry-delay #:resource-destruction-retry-delay

Create a dynamic resource pool. RETURN-NEW-RESOURCE is a thunk called to create each new resource value. MAX-SIZE is the maximum number of resources the pool will hold simultaneously.

Resources are created on demand when a checkout is requested and the pool is not yet at MAX-SIZE. Use with-resource-from-pool or call-with-resource-from-pool to request a resource and return it automatically when done.

Optional keyword arguments:

#:min-size

Minimum number of resources to keep alive even when idle. Defaults to 0.

#:idle-seconds

Seconds a resource may remain unused before being destroyed, provided the pool is above #:min-size. Defaults to #f (never expire idle resources).

#:lifetime

Maximum number of checkouts a single resource will serve before being destroyed and replaced by a fresh one. Defaults to #f (no limit).

#:destructor

A procedure called as (destructor resource) when a resource is removed from the pool. Defaults to #f.

#:add-resources-parallelism

Maximum number of concurrent calls to RETURN-NEW-RESOURCE when the pool needs to grow. Allowing resources to be created in parallel can result in more resources being created than can fit inside the pool, if this happens, the surplus resources are destroyed. Defaults to 1.

#:name

A string used in log messages. Defaults to "unnamed".

#:default-checkout-timeout

Default checkout timeout when requesting a resource from the pool, unset by default.

#:default-max-waiters

Maximum number of fibers that may queue waiting for a resource. When this limit is exceeded, &resource-pool-too-many-waiters is raised when a resource is requested. Defaults to #f (no limit).

#:resource-creation-retry-delay

Seconds to sleep before retrying when resource creation fails. Defaults to 1.

#:resource-destruction-retry-delay

Seconds to sleep before retrying when the resource destructor fails. Defaults to 5.

#:delay-logger

Called as (delay-logger seconds) with the time spent waiting for a resource to become available. Defaults to #f (no logging).

#:duration-logger

Called as (duration-logger seconds) after the proc passed to call-with-resource-from-pool completes, whether it returned normally or raised an exception. Can be overridden per-call via the #:duration-logger keyword argument to call-with-resource-from-pool. Defaults to #f (no logging).

#:scheduler

The Fibers scheduler to use for the pool’s internal fiber. Defaults to the current scheduler.

Procedure: make-resource-pool-destroy-resource-exception

Construct a &resource-pool-destroy-resource exception.

Procedure: resource-pool-delay-logger resource-pool

Undocumented procedure.

Procedure: resource-pool-destroy-resource-exception? obj

Return #t if OBJ is a &resource-pool-destroy-resource exception.

Procedure: resource-pool-destroyed-error-pool obj

Return the pool from a &resource-pool-destroyed exception.

Procedure: resource-pool-destroyed-error? obj

Return #t if OBJ is a &resource-pool-destroyed exception.

Procedure: resource-pool-duration-logger resource-pool

Undocumented procedure.

Procedure: resource-pool-stats pool KEY: #:timeout

Return an alist of statistics for POOL with the following keys:

resources

Total number of resources currently held by the pool.

available

Number of resources not currently checked out.

waiters

Number of fibers currently queued waiting for a resource.

checkout-failure-count

Cumulative number of checkouts where an exception was raised inside the proc.

Blocks waiting for the pool fiber to respond. #:timeout is the number of seconds to wait; defaults to 5. Raises &resource-pool-timeout if the pool does not respond in time.

Procedure: resource-pool-timeout-error-pool obj

Return the pool from a &resource-pool-timeout exception.

Procedure: resource-pool-timeout-error? obj

Return #t if OBJ is a &resource-pool-timeout exception.

Procedure: resource-pool-too-many-waiters-error-pool obj

Return the pool from a &resource-pool-too-many-waiters exception.

Procedure: resource-pool-too-many-waiters-error-waiters-count obj

Return the waiters count from a &resource-pool-too-many-waiters exception.

Procedure: resource-pool-too-many-waiters-error? obj

Return #t if OBJ is a &resource-pool-too-many-waiters exception.

1.9.4 Record Types

Record type: &resource-pool-destroy-resource

This record type has the following fields:

Record type: &resource-pool-destroyed

This record type has the following fields:

  • pool
Record type: &resource-pool-timeout

This record type has the following fields:

  • pool
Record type: &resource-pool-too-many-waiters

This record type has the following fields:

  • pool
  • waiters-count

1.10 (knots sort)

1.10.1 Procedures

Procedure: fibers-sort! items less KEY: #:parallelism

Sort ITEMS destructively using LESS as the comparison procedure, using a parallel merge sort. Returns the sorted list.

Splits ITEMS into chunks, sorts each in an eager fiber-promise in parallel, then merges pairs of sorted chunks in parallel until one sorted list remains.

#:parallelism sets the number of initial chunks. Defaults to the current fibers parallelism.


1.11 (knots thread-pool)

1.11.1 Macros

Macro: fixed-size-thread-pool-channel x

Return the channel of the fixed-size thread pool.

Macro: fixed-size-thread-pool-current-procedures x

Return the current procedures vector of the fixed-size thread pool.

Macro: fixed-size-thread-pool? x

Return #t if OBJ is a <fixed-size-thread-pool>.

Macro: thread-pool-resource-pool x

Return the underlying resource pool of the thread pool.

Macro: thread-pool? x

Return #t if OBJ is a <thread-pool>.

1.11.2 Procedures

Procedure: call-with-thread thread-pool proc KEY: #:delay-logger #:duration-logger #:checkout-timeout #:channel #:destroy-thread-on-exception? #:max-waiters

Run PROC in THREAD-POOL and return its values, blocking until complete. If called from within a thread that already belongs to THREAD-POOL, PROC is called directly in that thread.

Optional keyword arguments:

#:checkout-timeout

Seconds to wait for a free thread before raising &thread-pool-timeout-error. Defaults to the pool’s #:default-checkout-timeout.

#:max-waiters

Maximum number of fibers that may queue waiting for a thread (for dynamic pools). Defaults to the pool’s #:default-max-waiters.

#:destroy-thread-on-exception?

When #t, destroy the thread after PROC raises an exception. Equivalent to per-call #:expire-on-exception?. Defaults to #f.

#:delay-logger

Called as (delay-logger seconds) with the time spent waiting for a thread to become available. Defaults to the pool’s #:delay-logger if not specified.

#:duration-logger

Called as (duration-logger seconds) after PROC completes (whether or not it raised an exception). Defaults to the pool’s #:duration-logger if not specified.

#:channel

Override the channel used to communicate with the thread.

Procedure: destroy-thread-pool pool

Destroy POOL, stopping all of its threads and calling the destructor if specified. This procedure will block until the destruction is complete.

Procedure: make-fixed-size-thread-pool size KEY: #:thread-initializer #:thread-destructor #:delay-logger #:duration-logger #:thread-lifetime #:expire-on-exception? #:name #:use-default-io-waiters? #:default-checkout-timeout #:retry-delay

Create a pool of SIZE threads started immediately. Use call-with-thread to run a procedure in one of the threads.

Optional keyword arguments:

#:thread-initializer

A thunk called once when each thread starts. Its return value is passed as extra arguments to every procedure run in that thread. Defaults to #f (no extra arguments).

#:thread-destructor

A procedure called with the value returned by #:thread-initializer when a thread exits. Defaults to #f.

#:thread-lifetime

Maximum number of procedures a thread will run before restarting (and re-running #:thread-initializer). Defaults to #f (no limit).

#:expire-on-exception?

When #t, replace a thread after any unhandled exception. Defaults to #f.

#:use-default-io-waiters?

When #t (the default), each thread uses blocking I/O waiters so that port reads and writes block the thread rather than trying to suspend a fiber.

#:name

String used in thread names and log messages. Defaults to "unnamed".

#:default-checkout-timeout

Seconds to wait for a free thread slot before raising &thread-pool-timeout-error. Defaults to #f (wait forever).

#:retry-delay

Seconds to sleep before retrying when the thread initializer or destructor raises an exception. Defaults to 1.

#:delay-logger

Called as (delay-logger seconds) with the time spent waiting for a thread to become available.

#:duration-logger

Called as (duration-logger seconds) after each procedure completes, whether it returned normally or raised an exception.

Procedure: make-thread-pool max-size KEY: #:min-size #:scheduler #:thread-initializer #:thread-destructor #:delay-logger #:duration-logger #:thread-lifetime #:expire-on-exception? #:name #:use-default-io-waiters? #:default-checkout-timeout #:default-max-waiters

Create a dynamic thread pool with up to MAX-SIZE threads. Use call-with-thread to run a procedure in one of the threads.

Unlike make-fixed-size-thread-pool, threads are created on demand and may be reclaimed when idle (controlled by #:min-size and the resource pool’s idle management).

Accepts the same #:thread-initializer, #:thread-destructor, #:thread-lifetime, #:expire-on-exception?, #:use-default-io-waiters?, #:name, #:default-checkout-timeout, #:delay-logger, and #:duration-logger arguments as make-fixed-size-thread-pool, plus:

#:min-size

Minimum number of threads to keep alive. Defaults to MAX-SIZE (i.e.: the pool is pre-filled and never shrinks).

#:scheduler

Fibers scheduler for the pool’s internal resource pool fiber. Defaults to the current scheduler.

#:default-max-waiters

Maximum number of fibers that may queue waiting for a thread. Raises &thread-pool-timeout-error when exceeded. Defaults to #f (no limit).

Procedure: set-thread-name name

Set the name of the calling thread to NAME. NAME is truncated to 15 bytes.

Procedure: thread-name

Return the name of the calling thread as a string.

Procedure: thread-pool-arguments-parameter pool

Return the arguments parameter for POOL, dispatching on pool type.

Procedure: thread-pool-default-checkout-timeout pool

Return the default checkout timeout for POOL.

Procedure: thread-pool-delay-logger pool

Return the delay logger for POOL, dispatching on pool type.

Procedure: thread-pool-duration-logger pool

Return the duration logger for POOL, dispatching on pool type.

Procedure: thread-pool-timeout-error-pool obj

Return the pool from a &thread-pool-timeout-error exception.

Procedure: thread-pool-timeout-error? obj

Return #t if OBJ is a &thread-pool-timeout-error exception.

1.11.3 Record Types

Record type: &thread-pool-timeout-error

This record type has the following fields:

  • pool

1.12 (knots timeout)

1.12.1 Procedures

Procedure: port-read-timeout-error? obj

Return #t if OBJ is a &port-read-timeout-error.

Procedure: port-timeout-error? obj

Return #t if OBJ is a &port-timeout-error.

Procedure: port-write-timeout-error? obj

Return #t if OBJ is a &port-write-timeout-error.

Procedure: wait-until-port-readable-operation port

Make an operation that will succeed when PORT is readable.

Procedure: wait-until-port-writable-operation port

Make an operation that will succeed when PORT is writable.

Procedure: with-fibers-timeout thunk KEY: #:timeout #:on-timeout

Run THUNK in a new fiber and return its values, waiting TIMEOUT seconds for it to finish. If THUNK does not complete within TIMEOUT seconds, the ON-TIMEOUT procedure is called and with-fibers-timeout returns the result of ON-TIMEOUT instead.

If THUNK raises an exception it is re-raised in the calling fiber.

Procedure: with-port-timeouts thunk KEY: #:timeout #:read-timeout #:write-timeout #:poll-interval-ms

Run THUNK with per-operation I/O timeouts on all ports. If any read or write blocks for longer than the given number of seconds, an exception is raised.

#:timeout sets both read and write timeouts. #:read-timeout and #:write-timeout specify the timeout for reads and writes respectively. All three default to #f (no timeout).

#:poll-interval-ms sets the poll timeout interval in milliseconds used when running without fibers. Defaults to 200.

This procedure works both with fibers, and without fibers by using the poll system call with a timeout.

On read timeout, raises &port-read-timeout-error. On write timeout, raises &port-write-timeout-error. Both carry the thunk and port fields from &port-timeout-error.

1.12.2 Record Types

Record type: &port-read-timeout-error

This record type has the following fields:

  • thunk
  • port
Record type: &port-timeout-error

This record type has the following fields:

  • thunk
  • port
Record type: &port-write-timeout-error

This record type has the following fields:

  • thunk
  • port

1.13 (knots web-server)

1.13.1 Macros

Macro: active-request-fd x

Undocumented macro.

Macro: active-request-id x

Undocumented macro.

Macro: active-request-method x

Undocumented macro.

Macro: active-request-peer x

Undocumented macro.

Macro: active-request-start-time x

Undocumented macro.

Macro: active-request-uri x

Undocumented macro.

Macro: active-request? x

Undocumented macro.

Macro: web-server-port x

Return the port number of the web server.

Macro: web-server-socket x

Return the socket of the web server.

Macro: web-server? x

Return #t if OBJ is a <web-server>.

1.13.2 Procedures

Procedure: default-accept-exception-hook exn

Default hook for exceptions raised in the accept loop. Logs the exception and sleeps for 0.1 seconds before the loop retries, so persistent failures such as EMFILE do not become a tight loop.

Procedure: default-write-response-exception-handler exn request

Default handler for exceptions raised while writing an HTTP response. Logs the error for REQUEST to the current error port.

Procedure: format-active-request req OPT: port

Return a single human-readable line describing REQ.

Procedure: format-active-requests server OPT: port

Undocumented procedure.

Procedure: make-chunked-input-port/knots port KEY: #:keep-alive?

Undocumented procedure.

Procedure: make-chunked-output-port/knots port KEY: #:keep-alive? #:buffering

Returns a new port which translates non-encoded data into a HTTP chunked transfer encoded data and writes this to PORT. Data written to this port is buffered until the port is flushed, at which point it is all sent as one chunk. The port will otherwise be flushed every BUFFERING bytes, which defaults to 1200. Take care to close the port when done, as it will output the remaining data, and encode the final zero chunk. When the port is closed it will also close PORT, unless KEEP-ALIVE? is true.

port-position on the returned port reports the number of bytes written. The count must be read before close-port, as port-position errors on a closed port.

Procedure: read-request-body/knots r OPT: body-port

Read and return the full body of request R as a bytevector. Handles chunked transfer encoding.

Procedure: request-body-ended-prematurely-error? obj

Return #t if OBJ is a &request-body-ended-prematurely exception.

Procedure: run-knots-web-server handler KEY: #:host #:family #:addr #:port #:ipv6-v6only? #:socket #:read-request-exception-handler #:write-response-exception-handler #:connection-idle-timeout #:connection-buffer-size #:accept-exception-hook #:post-request-hook #:connection-accepted-hook #:connection-closed-hook #:listen-backlog #:call-handler-with-body-port? #:active-request-tracking?

Run the knots web server, returning a <web-server> record.

HANDLER should be a procedure that takes two arguments — the HTTP request and an input port for reading its body — and returns two values, the response and response body. The response and body will be run through ‘sanitize-response’ before being sent back to the client.

For example, here is a simple "Hello, World!" server:

 (define (handler request body-port)
   (values '((content-type . (text/plain)))
           "Hello, World!"))
 (run-knots-web-server handler #:port 8080)

This listens on the IPv6 loopback address, reachable at http://[::1]:8080/.

The following keyword arguments are accepted:

host

Numeric address string used to derive addr. When #f (the default), the server binds to the loopback address of family. When given, family is auto-detected from the address (a : indicates IPv6) unless family is also supplied.

family

Address family for the listening socket. Defaults to AF_INET6 (so the server listens on IPv6 only by default). With IPV6_V6ONLY set on the socket, an additional AF_INET server can be run on the same port to handle IPv4 clients.

addr

The address to bind to. Defaults to (inet-pton family host) when host is set, otherwise to the loopback address of family (IN6ADDR_LOOPBACK for AF_INET6, INADDR_LOOPBACK for AF_INET). Ignored if socket is supplied.

port

TCP port to listen on. Defaults to 8080. Ignored if socket is supplied.

ipv6-v6only?

Sets IPV6_V6ONLY on an AF_INET6 listening socket. Defaults to #f, so a server bound to :: also accepts IPv4 clients via v4-mapped addresses. Set to #t when running a separate AF_INET server on the same port. Ignored if socket is supplied.

socket

A pre-created listening socket to use instead of letting the server build one from family, addr, and port. Useful for inheriting sockets from a supervisor or for binding with options the default helper does not set.

read-request-exception-handler

Procedure of one argument (the exception) called when reading a request fails. When the handler returns, the connection is closed and a 400 response is sent. Defaults to a handler that logs the exception to (current-error-port).

write-response-exception-handler

Procedure of two arguments (the exception and the request) called when writing a response fails. Defaults to a handler that logs the exception and closes the client port.

connection-idle-timeout

If a number, close idle keep-alive connections that go this many seconds without sending data. Defaults to #f (no timeout).

connection-buffer-size

Buffer size in bytes used for both the client socket and chunked output. Defaults to 1024.

accept-exception-hook

Procedure of one argument (the exception) called when the accept loop raises. Defaults to default-accept-exception-hook, which logs the exception and sleeps for 0.1 seconds before the loop retries, so persistent failures such as EMFILE do not become a tight loop. Pass #f to disable both the log and the backoff, or supply a custom hook.

post-request-hook

If non-#f, a procedure called once for every request the server attempts to handle, including requests that fail to parse, fail in the handler, or fail while writing the response.

request is the parsed request, or #f if the request could not be parsed (in which case the server has synthesised a 400 response). response is the response sent to the client; for a request that raised in the handler the server synthesises a 500, which is what the hook sees.

Times are values from get-internal-real-time. read-request-time is taken just after the request line and headers were read; response-start-time is taken just before the server begins writing the response, so the gap between the two captures the time spent computing the response (handler plus sanitize-response). response-end-time is non-#f only when the response (including any body) was written and flushed in full; metric consumers can use this to distinguish delivered responses from truncated or undelivered ones.

response-body-bytes-written is the number of decoded body bytes written to the client (chunked framing overhead is not counted), or #f when the count is unknown — for example when the body procedure raised mid-write or when write-response itself failed. HEAD responses and responses without a body report 0.

request-body-bytes-read is the number of decoded request body bytes received from the client. In the default mode (call-handler-with-body-port? #t) the server drains any unread body before this is reported, so the count matches what the client sent. In legacy mode it reflects what the handler chose to read. Reports 0 when the request had no body or could not be parsed.

For forwards compatibility, it’s recommended to use #:allow-other-keys for the post-request-hook.

Exceptions raised by the hook itself are caught and logged.

connection-accepted-hook

If non-#f, a procedure called when a new client connection has been accepted, as (connection-accepted-hook sockaddr).

connection-closed-hook

If non-#f, a procedure called just before a client connection is closed, as

(connection-closed-hook sockaddr
                        #:start-time T1
                        #:end-time   T2)

where the times are values from get-internal-real-time taken when the connection was first accepted and just before it was closed. Hooks should declare #:allow-other-keys for forwards compatibility.

Exceptions raised by either connection hook are caught and logged.

listen-backlog

Backlog passed to listen. Defaults to 1024; a large value avoids dropped SYNs under bursty load.

call-handler-with-body-port?

When #t (the default), HANDLER is called with two arguments, as described above. When #f, HANDLER is instead called with just the request as a single argument and is responsible for reading the body itself via ‘read-request-body/knots’. This option is provided backwards compatibility and will eventually be removed.

active-request-tracking?

When true, the server tracks in-flight requests so they can be inspected via server-active-requests and format-active-requests. Defaults to #f.

Procedure: sanitize-response request response body

"Sanitize" the given response and body, making them appropriate for the given request.

As a convenience to web handler authors, RESPONSE may be given as an alist of headers, in which case it is used to construct a default response. Ensures that the response version corresponds to the request version. If BODY is a string, encodes the string to a bytevector, in an encoding appropriate for RESPONSE. Adds a ‘content-length’ and ‘content-type’ header, as necessary.

If BODY is a procedure, it is called with a port as an argument, and the output collected as a bytevector. In the future we might try to instead use a compressing, chunk-encoded port, and call this procedure later, in the write-client procedure. Authors are advised not to rely on the procedure being called at any particular time.

Procedure: server-active-requests server

Undocumented procedure.

1.13.3 Record Types

Record type: &request-body-ended-prematurely

This record type has the following fields:

  • bytes-read

1.14 (knots web)

1.14.1 Procedures

Procedure: call-with-cached-connection cache proc KEY: #:close-connection-on-exception?

Check out a connection port from CACHE and call (proc port), returning the result. The port is returned to the cache when PROC returns, or closed on exception if CLOSE-CONNECTION-ON-EXCEPTION? is true (the default).

Procedure: call-with-connection-cache uri max-cached-connections proc KEY: #:verify-certificate?

Create a connection cache for URI with up to MAX-CACHED-CONNECTIONS, call (proc cache), then destroy the cache and return the values returned by PROC.

Procedure: http-fold-requests connection-cache proc seed requests KEY: #:batch-size

Fold PROC over HTTP request/response pairs using CONNECTION-CACHE for connections. PROC is called as (proc request response body-port accumulator) and its return value becomes the new accumulator. Requests are sent in batches of up to BATCH-SIZE before responses are read (HTTP pipelining).

When the server closes the connection mid-batch the remaining requests are retried on a fresh connection from the cache.

Procedure: make-connection-cache uri max-cached-connections KEY: #:verify-certificate? #:idle-seconds #:connect-timeout #:name #:default-checkout-timeout #:default-max-waiters #:lifetime #:scheduler

Create a resource pool of up to MAX-CACHED-CONNECTIONS connections to URI.

Optional keyword arguments:

#:verify-certificate?

Whether to verify TLS certificates. Defaults to #t.

#:idle-seconds

Seconds a connection may remain unused before being closed. Defaults to 600.

#:connect-timeout

Seconds to wait when establishing a new connection. Defaults to 30.

#:name

A string used in log messages. Defaults to "unnamed".

#:default-checkout-timeout

Default timeout in seconds when waiting for a connection from the cache. Defaults to #f (wait indefinitely).

#:default-max-waiters

Maximum number of fibers that may queue waiting for a connection. Defaults to #f (no limit).

#:lifetime

Maximum number of checkouts a single connection will serve before being closed and replaced. Defaults to #f (no limit).

#:scheduler

The Fibers scheduler to use for the pool’s internal fiber. Defaults to the current scheduler.


Appendix A Version History

Version 0.1
  • Initial release.

Appendix B Copying Information

Copyright © 2024, 2025, 2026 Christopher Baines <mail@cbaines.net>

This library is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.


Concept Index


Data Type Index


Procedure Index

Jump to:   A   C   D   E   F   H   K   M   N   P   R   S   T   W  

A
active-request-fdknots_web-server
active-request-idknots_web-server
active-request-methodknots_web-server
active-request-peerknots_web-server
active-request-start-timeknots_web-server
active-request-uriknots_web-server
active-request?knots_web-server

C
call-with-cached-connectionknots_web
call-with-connection-cacheknots_web
call-with-default-io-waitersknots
call-with-knots-stack-wrappingknots
call-with-knots-stack-wrappingknots_backtraces
call-with-parallelism-limiterknots_parallelism
call-with-resource-from-poolknots_resource-pool
call-with-sigintknots
call-with-temporary-threadknots
call-with-threadknots_thread-pool

D
default-accept-exception-hookknots_web-server
default-write-response-exception-handlerknots_web-server
destroy-parallelism-limiterknots_parallelism
destroy-resource-poolknots_resource-pool
destroy-thread-poolknots_thread-pool
display/knotsknots

E
exception-frames/knotsknots_backtraces

F
fiberizeknots_parallelism
fibers-batch-for-eachknots_parallelism
fibers-batch-mapknots_parallelism
fibers-delayknots_promise
fibers-delay/eagerknots_promise
fibers-for-eachknots_parallelism
fibers-forceknots_promise
fibers-letknots_parallelism
fibers-mapknots_parallelism
fibers-map-with-progressknots_parallelism
fibers-parallelknots_parallelism
fibers-promise-resetknots_promise
fibers-promise-result-available?knots_promise
fibers-promise?knots_promise
fibers-sort!knots_sort
fixed-size-thread-pool-channelknots_thread-pool
fixed-size-thread-pool-current-proceduresknots_thread-pool
fixed-size-thread-pool?knots_thread-pool
format-active-requestknots_web-server
format-active-requestsknots_web-server
format/knotsknots

H
handle-channel-requestknots_actors
http-fold-requestsknots_web

K
knots-exception-stackknots
knots-exception-stackknots_backtraces
knots-exception?knots
knots-exception?knots_backtraces

M
make-channel-requestknots_actors
make-chunked-input-port/knotsknots_web-server
make-chunked-output-port/knotsknots_web-server
make-connection-cacheknots_web
make-fixed-size-resource-poolknots_resource-pool
make-fixed-size-thread-poolknots_thread-pool
make-knots-exceptionknots
make-knots-exceptionknots_backtraces
make-notificationknots_notification
make-parallelism-limiterknots_parallelism
make-resource-poolknots_resource-pool
make-resource-pool-destroy-resource-exceptionknots_resource-pool
make-thread-poolknots_thread-pool

N
non-blocking-open-socket-for-uriknots_non-blocking
non-blocking-portknots_non-blocking
notification?knots_notification
notify!knots_notification

P
parallelism-limiter?knots_parallelism
port-read-timeout-error?knots_timeout
port-timeout-error?knots_timeout
port-write-timeout-error?knots_timeout
print-backtrace-and-exception/knotsknots
print-backtrace-and-exception/knotsknots_backtraces

R
read-request-body/knotsknots_web-server
request-body-ended-prematurely-error?knots_web-server
resource-pool-channelknots_resource-pool
resource-pool-configurationknots_resource-pool
resource-pool-delay-loggerknots_resource-pool
resource-pool-destroy-resource-exception?knots_resource-pool
resource-pool-destroyed-error-poolknots_resource-pool
resource-pool-destroyed-error?knots_resource-pool
resource-pool-duration-loggerknots_resource-pool
resource-pool-nameknots_resource-pool
resource-pool-statsknots_resource-pool
resource-pool-timeout-error-poolknots_resource-pool
resource-pool-timeout-error?knots_resource-pool
resource-pool-too-many-waiters-error-poolknots_resource-pool
resource-pool-too-many-waiters-error-waiters-countknots_resource-pool
resource-pool-too-many-waiters-error?knots_resource-pool
resource-pool?knots_resource-pool
run-knots-web-serverknots_web-server

S
sanitize-responseknots_web-server
server-active-requestsknots_web-server
set-thread-nameknots_thread-pool
simple-format/knotsknots
spawn-fiber/knotsknots
spawn-queueing-fiberknots_queue

T
thread-nameknots_thread-pool
thread-pool-arguments-parameterknots_thread-pool
thread-pool-default-checkout-timeoutknots_thread-pool
thread-pool-delay-loggerknots_thread-pool
thread-pool-duration-loggerknots_thread-pool
thread-pool-resource-poolknots_thread-pool
thread-pool-timeout-error-poolknots_thread-pool
thread-pool-timeout-error?knots_thread-pool
thread-pool?knots_thread-pool

W
wait-for-notificationknots_notification
wait-for-notification-operationknots_notification
wait-until-port-readable-operationknots_timeout
wait-until-port-writable-operationknots_timeout
wait-when-system-clock-behindknots
web-server-portknots_web-server
web-server-socketknots_web-server
web-server?knots_web-server
with-fibers-timeoutknots_timeout
with-parallelism-limiterknots_parallelism
with-port-timeoutsknots_timeout
with-resource-from-poolknots_resource-pool


Variable Index