Guile Knots is a library providing tools and patterns for programming with Guile Fibers. Guile Knots provides higher level building blocks for writing programs using Guile Fibers, including managing code that can’t run in a thread used by fibers. Also included is a web server implementation using Fibers, which while being similar to the web server provided by Fibers, can provide some benefits in specific circumstances.
The following is the list of modules provided by this library.
Run THUNK with Guile’s default blocking I/O waiters active.
This is useful when restoring the default Guile I/O waiters from within a context (like Fibers) where different I/O waiters are used, for example when creating a new thread from a fiber.
Run THUNK with a SIGINT handler that signals the Fibers condition CVAR. Restores the previous handler when THUNK returns.
Typical usage is to pass a condition variable to this procedure and wait on CVAR in a fiber to implement clean shutdown on Ctrl-C:
(let ((quit-cvar (make-condition)))
(call-with-sigint
(lambda () (wait quit-cvar))
quit-cvar))
Run THUNK in a temporary thread and return its result to the calling fiber.
Write OBJ to PORT (default: current output port) as a UTF-8 byte
sequence via put-bytevector.
When used with ports without buffering, this should be safer than display.
Like format but should be safer when used with a port without
buffering.
Return the stack from a &knots-exception.
Return #t if OBJ is a &knots-exception.
Construct a &knots-exception with the given stack.
Print the backtrace and exception information from EXN to PORT. This
procedure captures the stack, so should be run before the stack is
unwound, so using with-exception-handler without #:unwind?
#t, the exception may need to then be re-raised and handled in an outer
exception handler.
(with-exception-handler
(lambda (exn)
;; Recover from the exception
#f)
(lambda ()
(with-exception-handler
(lambda (exn)
(print-backtrace-and-exception/knots exn)
(raise-exception exn))
(lambda ()
(do-things))))
#:unwind? #t)
It’s important to use print-backtrace-and-exception/knots for
displaying backtraces involving functionality from Guile Knots, since
the stack involved is potentially split across several fibers. The
stacks involved are attached to the exception, and this procedure
extracts this information out and assembles a backtrace including all
the code involved.
Like simple-format but should be safer when used with a port
without buffering.
Spawn a fiber to run THUNK, with knots exception handling.
Accepts the same optional SCHEDULER and #:parallel? arguments as
spawn-fiber.
Block until the system clock reads at least 2001-01-02.
Useful at startup in environments (virtual machines, embedded systems) where the clock may start at or near the Unix epoch. Prints a warning to the current error port every 20 seconds while waiting.
Return the stack from a &knots-exception.
Return #t if OBJ is a &knots-exception.
Construct a &knots-exception with the given stack.
Print the backtrace and exception information from EXN to PORT. This
procedure captures the stack, so should be run before the stack is
unwound, so using with-exception-handler without #:unwind?
#t, the exception may need to then be re-raised and handled in an outer
exception handler.
(with-exception-handler
(lambda (exn)
;; Recover from the exception
#f)
(lambda ()
(with-exception-handler
(lambda (exn)
(print-backtrace-and-exception/knots exn)
(raise-exception exn))
(lambda ()
(do-things))))
#:unwind? #t)
It’s important to use print-backtrace-and-exception/knots for
displaying backtraces involving functionality from Guile Knots, since
the stack involved is potentially split across several fibers. The
stacks involved are attached to the exception, and this procedure
extracts this information out and assembles a backtrace including all
the code involved.
Open a socket for URI and return it as a non-blocking port.
For HTTPS URIs the TLS handshake is completed while the socket is still blocking (required because Guile’s TLS wrapper does not support non-blocking handshakes), then the underlying socket is made non-blocking. For plain HTTP the socket is made non-blocking immediately.
#:verify-certificate? controls TLS certificate verification and
defaults to #t.
Make PORT non-blocking and return it.
Let, but run each binding in a fiber in parallel.
Run each expression in parallel. If any expression raises an exception, this will be raised after all exceptions have finished.
Return #t if OBJ is a <parallelism-limiter>.
Evaluate EXP ... while holding a slot from PARALLELISM-LIMITER.
Syntactic sugar around call-with-parallelism-limiter.
Acquire a slot from PARALLELISM-LIMITER, call THUNK, release the slot, and return the values from THUNK. Blocks if no slot is currently available.
Destroy PARALLELISM-LIMITER, releasing its underlying resource pool.
Convert PROC into a procedure backed by #:parallelism (default:
1) background fibers. Returns a wrapper that sends its arguments to one
of the fibers and blocks until the result is returned.
#:input-channel is the channel that callers write requests to;
defaults to a fresh channel. #:process-channel is the channel
the fibers read from; defaults to #:input-channel. Setting them
differently allows external parties to bypass the wrapper and write
directly to process-channel.
Call PROC on LISTS, running up to PARALLELISM-LIMIT fibers in parallel.
Map PROC over LISTS in parallel, with a PARALLELISM-LIMIT. If any of the invocations of PROC raise an exception, this will be raised once all of the calls to PROC have finished.
Call PROC on LISTS, running up to 20 fibers in parallel.
Map PROC over LISTS in parallel, running up to 20 fibers in PARALLEL. If any of the invocations of PROC raise an exception, this will be raised once all of the calls to PROC have finished.
Map PROC over LISTS, calling #:REPORT if specified after each invocation of PROC finishes. REPORT is passed the results for each element of LISTS, or #f if no result has been received yet.
Return a parallelism limiter that allows at most LIMIT concurrent fibers
to execute within with-parallelism-limiter at the same time.
Further fibers block until a slot becomes free.
#:name is a string used in log messages. Defaults to
"unnamed".
Return a new fiber-aware promise that will evaluate THUNK when first
forced. THUNK is not called until fibers-force is called on the
promise.
Return a new fiber-aware promise and immediately begin evaluating THUNK
in a new fiber. Exceptions during eager evaluation are silently
discarded; they will be re-raised when fibers-force is called.
Force the fiber-aware promise FP, returning its values.
The first call evaluates the promise’s thunk. Concurrent callers block on a condition variable until evaluation finishes, then receive the same result. If the thunk raises an exception, the exception is stored and re-raised for all callers.
Reset the fiber-aware promise FP so that the next call to
fibers-force re-evaluates its thunk.
Return #t if the fiber-aware promise FP has been evaluated
(successfully or with an exception) and #f if evaluation has not
yet started or is still in progress.
Spawn a fiber that serialises items onto DEST-CHANNEL in FIFO order. Returns a new input channel.
Multiple producers can put items on the returned channel concurrently. The fiber buffers them locally and forwards them to DEST-CHANNEL one at a time, preserving arrival order.
Return the channel used by the resource pool.
Return the configuration alist of the resource pool.
Return the name of the resource pool.
Return #t if OBJ is a <resource-pool>.
Evaluate EXP ... with RESOURCE bound to a resource checked out from
POOL. Syntactic sugar around call-with-resource-from-pool.
Call PROC with a resource from POOL, blocking until a resource becomes available. Return the resource once PROC has returned.
#:delay-logger is called as (delay-logger seconds) with
the time spent waiting for a resource to become available. Defaults to
the pool’s #:delay-logger if not specified.
#:duration-logger is called as (duration-logger seconds)
after PROC completes, whether it returned normally or raised an
exception. Defaults to the pool’s #:duration-logger if not
specified.
Destroy POOL, preventing any new checkouts. Blocks until all
checked-out resources have been returned, running the pool’s
#:destructor on each. Any fibers waiting for a resource receive
&resource-pool-destroyed.
Create a resource pool from RESOURCES-LIST-OR-VECTOR, a list or vector of pre-existing resource values.
Use with-resource-from-pool or
call-with-resource-from-pool to borrow a resource and return it
automatically when done.
Optional keyword arguments:
#:nameA optional string used in log messages. Defaults to "unnamed".
#:default-checkout-timeoutDefault checkout timeout when requesting a resource from the pool, unset by default.
#:default-max-waitersMaximum number of fibers that may queue waiting for a resource. When
this limit is exceeded, &resource-pool-too-many-waiters is raised
when a resource is requested. Defaults to #f (no limit).
#:delay-loggerCalled as (delay-logger seconds) with the time spent waiting for
a resource to become available. Defaults to #f (no logging).
#:duration-loggerCalled as (duration-logger seconds) after the proc passed to
call-with-resource-from-pool completes, whether it returned
normally or raised an exception. Can be overridden per-call via the
#:duration-logger keyword argument to
call-with-resource-from-pool. Defaults to #f (no
logging).
#:schedulerThe Fibers scheduler to use for the pool’s internal fiber. Defaults to the current scheduler.
Create a dynamic resource pool. RETURN-NEW-RESOURCE is a thunk called to create each new resource value. MAX-SIZE is the maximum number of resources the pool will hold simultaneously.
Resources are created on demand when a checkout is requested and the
pool is not yet at MAX-SIZE. Use with-resource-from-pool or
call-with-resource-from-pool to request a resource and return it
automatically when done.
Optional keyword arguments:
#:min-sizeMinimum number of resources to keep alive even when idle. Defaults to
0.
#:idle-secondsSeconds a resource may remain unused before being destroyed, provided
the pool is above #:min-size. Defaults to #f (never
expire idle resources).
#:lifetimeMaximum number of checkouts a single resource will serve before being
destroyed and replaced by a fresh one. Defaults to #f (no
limit).
#:destructorA procedure called as (destructor resource) when a resource is
removed from the pool. Defaults to #f.
#:add-resources-parallelismMaximum number of concurrent calls to RETURN-NEW-RESOURCE when the pool
needs to grow. Allowing resources to be created in parallel can result
in more resources being created than can fit inside the pool, if this
happens, the surplus resources are destroyed. Defaults to 1.
#:nameA string used in log messages. Defaults to "unnamed".
#:default-checkout-timeoutDefault checkout timeout when requesting a resource from the pool, unset by default.
#:default-max-waitersMaximum number of fibers that may queue waiting for a resource. When
this limit is exceeded, &resource-pool-too-many-waiters is raised
when a resource is requested. Defaults to #f (no limit).
#:delay-loggerCalled as (delay-logger seconds) with the time spent waiting for
a resource to become available. Defaults to #f (no logging).
#:duration-loggerCalled as (duration-logger seconds) after the proc passed to
call-with-resource-from-pool completes, whether it returned
normally or raised an exception. Can be overridden per-call via the
#:duration-logger keyword argument to
call-with-resource-from-pool. Defaults to #f (no
logging).
#:schedulerThe Fibers scheduler to use for the pool’s internal fiber. Defaults to the current scheduler.
Construct a &resource-pool-destroy-resource exception.
Undocumented procedure.
Return #t if OBJ is a &resource-pool-destroy-resource
exception.
Return the pool from a &resource-pool-destroyed exception.
Return #t if OBJ is a &resource-pool-destroyed exception.
Undocumented procedure.
Return an alist of statistics for POOL with the following keys:
resourcesTotal number of resources currently held by the pool.
availableNumber of resources not currently checked out.
waitersNumber of fibers currently queued waiting for a resource.
checkout-failure-countCumulative number of checkouts where an exception was raised inside the proc.
Blocks waiting for the pool fiber to respond. #:timeout is the
number of seconds to wait; defaults to 5. Raises
&resource-pool-timeout if the pool does not respond in time.
Return the pool from a &resource-pool-timeout exception.
Return #t if OBJ is a &resource-pool-timeout exception.
Return the pool from a &resource-pool-too-many-waiters exception.
Return the waiters count from a &resource-pool-too-many-waiters
exception.
Return #t if OBJ is a &resource-pool-too-many-waiters
exception.
This record type has the following fields:
This record type has the following fields:
poolThis record type has the following fields:
poolThis record type has the following fields:
poolwaiters-countSort ITEMS destructively using LESS as the comparison procedure, using a parallel merge sort. Returns the sorted list.
Splits ITEMS into chunks, sorts each in an eager fiber-promise in parallel, then merges pairs of sorted chunks in parallel until one sorted list remains.
#:parallelism sets the number of initial chunks. Defaults to the
current fibers parallelism.
Return the channel of the fixed-size thread pool.
Return the current procedures vector of the fixed-size thread pool.
Return #t if OBJ is a <fixed-size-thread-pool>.
Return the underlying resource pool of the thread pool.
Return #t if OBJ is a <thread-pool>.
Run PROC in THREAD-POOL and return its values, blocking until complete. If called from within a thread that already belongs to THREAD-POOL, PROC is called directly in that thread.
Optional keyword arguments:
#:checkout-timeoutSeconds to wait for a free thread before raising
&thread-pool-timeout-error. Defaults to the pool’s
#:default-checkout-timeout.
#:max-waitersMaximum number of fibers that may queue waiting for a thread (for
dynamic pools). Defaults to the pool’s #:default-max-waiters.
#:destroy-thread-on-exception?When #t, destroy the thread after PROC raises an exception.
Equivalent to per-call #:expire-on-exception?. Defaults to
#f.
#:delay-loggerCalled as (delay-logger seconds) with the time spent waiting for
a thread to become available. Defaults to the pool’s
#:delay-logger if not specified.
#:duration-loggerCalled as (duration-logger seconds) after PROC completes (whether
or not it raised an exception). Defaults to the pool’s
#:duration-logger if not specified.
#:channelOverride the channel used to communicate with the thread.
Destroy POOL, stopping all of its threads and calling the destructor if specified. This procedure will block until the destruction is complete.
Create a pool of SIZE threads started immediately. Use
call-with-thread to run a procedure in one of the threads.
Optional keyword arguments:
#:thread-initializerA thunk called once when each thread starts. Its return value is passed
as extra arguments to every procedure run in that thread. Defaults to
#f (no extra arguments).
#:thread-destructorA procedure called with the value returned by
#:thread-initializer when a thread exits. Defaults to #f.
#:thread-lifetimeMaximum number of procedures a thread will run before restarting (and
re-running #:thread-initializer). Defaults to #f (no
limit).
#:expire-on-exception?When #t, replace a thread after any unhandled exception. Defaults
to #f.
#:use-default-io-waiters?When #t (the default), each thread uses blocking I/O waiters so
that port reads and writes block the thread rather than trying to
suspend a fiber.
#:nameString used in thread names and log messages. Defaults to
"unnamed".
#:default-checkout-timeoutSeconds to wait for a free thread slot before raising
&thread-pool-timeout-error. Defaults to #f (wait
forever).
#:delay-loggerCalled as (delay-logger seconds) with the time spent waiting for
a thread to become available.
#:duration-loggerCalled as (duration-logger seconds) after each procedure
completes, whether it returned normally or raised an exception.
Create a dynamic thread pool with up to MAX-SIZE threads. Use
call-with-thread to run a procedure in one of the threads.
Unlike make-fixed-size-thread-pool, threads are created on demand
and may be reclaimed when idle (controlled by #:min-size and the
resource pool’s idle management).
Accepts the same #:thread-initializer,
#:thread-destructor, #:thread-lifetime,
#:expire-on-exception?, #:use-default-io-waiters?,
#:name, #:default-checkout-timeout, #:delay-logger,
and #:duration-logger arguments as
make-fixed-size-thread-pool, plus:
#:min-sizeMinimum number of threads to keep alive. Defaults to MAX-SIZE (i.e.: the pool is pre-filled and never shrinks).
#:schedulerFibers scheduler for the pool’s internal resource pool fiber. Defaults to the current scheduler.
#:default-max-waitersMaximum number of fibers that may queue waiting for a thread. Raises
&thread-pool-timeout-error when exceeded. Defaults to #f
(no limit).
Set the name of the calling thread to NAME. NAME is truncated to 15 bytes.
Return the name of the calling thread as a string.
Return the arguments parameter for POOL, dispatching on pool type.
Return the default checkout timeout for POOL.
Return the delay logger for POOL, dispatching on pool type.
Return the duration logger for POOL, dispatching on pool type.
Return the pool from a &thread-pool-timeout-error exception.
Return #t if OBJ is a &thread-pool-timeout-error
exception.
Return #t if OBJ is a &port-read-timeout-error.
Return #t if OBJ is a &port-timeout-error.
Return #t if OBJ is a &port-write-timeout-error.
Make an operation that will succeed when PORT is readable.
Make an operation that will succeed when PORT is writable.
Run THUNK in a new fiber and return its values, waiting TIMEOUT seconds for it to finish. If THUNK does not complete within TIMEOUT seconds, the ON-TIMEOUT procedure is called and with-fibers-timeout returns the result of ON-TIMEOUT instead.
If THUNK raises an exception it is re-raised in the calling fiber.
Run THUNK with per-operation I/O timeouts on all ports. If any read or write blocks for longer than the given number of seconds, an exception is raised.
#:timeout sets both read and write timeouts.
#:read-timeout and #:write-timeout specify the timeout for
reads and writes respectively. All three default to #f (no
timeout).
This procedure works both with fibers, and without fibers by using the poll system call with a timeout.
On read timeout, raises &port-read-timeout-error. On write
timeout, raises &port-write-timeout-error. Both carry the
thunk and port fields from &port-timeout-error.
Return the port number of the web server.
Return the socket of the web server.
Return #t if OBJ is a <web-server>.
Default handler for exceptions raised while writing an HTTP response. Logs the error for REQUEST to the current error port.
Returns a new port which translates non-encoded data into a HTTP chunked transfer encoded data and writes this to PORT. Data written to this port is buffered until the port is flushed, at which point it is all sent as one chunk. The port will otherwise be flushed every BUFFERING bytes, which defaults to 1200. Take care to close the port when done, as it will output the remaining data, and encode the final zero chunk. When the port is closed it will also close PORT, unless KEEP-ALIVE? is true.
Read and return the full body of request R as a bytevector. Handles chunked transfer encoding.
Return #t if OBJ is a &request-body-ended-prematurely
exception.
Return an input port for reading the body of request REQUEST. Handles chunked transfer encoding.
Run the knots web server.
HANDLER should be a procedure that takes one argument, the HTTP request and returns two values, the response and response body.
For example, here is a simple "Hello, World!" server:
(define (handler request)
(let ((body (read-request-body request)))
(values '((content-type . (text/plain)))
"Hello, World!")))
(run-knots-web-server handler)
The response and body will be run through ‘sanitize-response’ before sending back to the client.
"Sanitize" the given response and body, making them appropriate for the given request.
As a convenience to web handler authors, RESPONSE may be given as an alist of headers, in which case it is used to construct a default response. Ensures that the response version corresponds to the request version. If BODY is a string, encodes the string to a bytevector, in an encoding appropriate for RESPONSE. Adds a ‘content-length’ and ‘content-type’ header, as necessary.
If BODY is a procedure, it is called with a port as an argument, and the output collected as a bytevector. In the future we might try to instead use a compressing, chunk-encoded port, and call this procedure later, in the write-client procedure. Authors are advised not to rely on the procedure being called at any particular time.
Check out a connection port from CACHE and call (proc port),
returning the result. The port is returned to the cache when PROC
returns, or closed on exception if CLOSE-CONNECTION-ON-EXCEPTION? is
true (the default).
Create a connection cache for URI with up to MAX-CACHED-CONNECTIONS,
call (proc cache), then destroy the cache and return the values
returned by PROC.
Fold PROC over HTTP request/response pairs using CONNECTION-CACHE for
connections. PROC is called as (proc request response body-port
accumulator) and its return value becomes the new accumulator. Requests
are sent in batches of up to BATCH-SIZE before responses are read (HTTP
pipelining).
When the server closes the connection mid-batch the remaining requests are retried on a fresh connection from the cache.
Create a resource pool of up to MAX-CACHED-CONNECTIONS to URI.
Copyright © 2024, 2025 Christopher Baines <mail@cbaines.net>
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
| Jump to: | C D F H K M N P R S T W |
|---|
| Jump to: | C D F H K M N P R S T W |
|---|
| Index Entry | Section | |
|---|---|---|
| R | ||
resource-pool-default-timeout-handler | knots_resource-pool | |