I/O in Trio

Sockets and networking

The trio.socket module provides trio’s basic networking API.

trio.socket‘s top-level exports

Generally, trio.socket‘s API mirrors that of the standard library socket module. Most constants (like SOL_SOCKET) and simple utilities (like inet_aton()) are simply re-exported unchanged. But there are also some differences:

All functions that return sockets (e.g. socket.socket(), socket.socketpair(), ...) are modified to return trio sockets instead. In addition, there is a new function to directly convert a standard library socket into a trio socket:

trio.socket.from_stdlib_socket(sock)

Convert a standard library socket.socket() into a trio socket.

The following functions have identical interfaces to their standard library version, but are now async functions, so you need to use await to call them:

Trio intentionally DOES NOT include some obsolete, redundant, or broken features:

Socket objects

class trio.socket.SocketType

Trio socket objects are overall very similar to the standard library socket objects, with a few important differences:

Async all the things: Most obviously, everything is made “trio-style”: blocking methods become async methods, and the following attributes are not supported:

  • setblocking(): trio sockets always act like blocking sockets; if you need to read/write from multiple sockets at once, then create multiple tasks.
  • settimeout(): see Cancellation and timeouts instead.
  • makefile(): Python’s file-like API is synchronous, so it can’t be implemented on top of an async socket.

No implicit name resolution: In the standard library socket API, there are number of methods that take network addresses as arguments. When given a numeric address this is fine:

# OK
sock.bind(("127.0.0.1", 80))
sock.connect(("2607:f8b0:4000:80f::200e", 80))

But in the standard library, these methods also accept hostnames, and in this case implicitly trigger a DNS lookup to find the IP address:

# Might block!
sock.bind(("localhost", 80))
sock.connect(("google.com", 80))

This is problematic because DNS lookups are a blocking operation.

For simplicity, trio forbids such usages: hostnames must be “pre-resolved” to numeric addresses before they are passed to socket methods like bind() or connect(). In most cases this can be easily accomplished by calling either resolve_local_address() or resolve_remote_address().

await resolve_local_address(address)

Resolve the given address into a numeric address suitable for passing to bind().

This performs the same address resolution that the standard library bind() call would do, taking into account the current socket’s settings (e.g. if this is an IPv6 socket then it returns IPv6 addresses). In particular, a hostname of None is mapped to the wildcard address.

await resolve_remote_address(address)

Resolve the given address into a numeric address suitable for passing to connect() or similar.

This performs the same address resolution that the standard library connect() call would do, taking into account the current socket’s settings (e.g. if this is an IPv6 socket then it returns IPv6 addresses). In particular, a hostname of None is mapped to the localhost address.

Modern defaults: And finally, we took the opportunity to update the defaults for several socket options that were stuck in the 1980s. You can always use setsockopt() to change these back, but for trio sockets:

  1. Everywhere except Windows, SO_REUSEADDR is enabled by default. This is almost always what you want, but if you’re in one of the rare cases where this is undesireable then you can always disable SO_REUSEADDR manually:

    sock.setsockopt(trio.socket.SOL_SOCKET, trio.socket.SO_REUSEADDR, False)
    

    On Windows, SO_EXCLUSIVEADDR is enabled by default. Unfortunately, this means that if you stop and restart a server you may have trouble reacquiring listen ports (i.e., it acts like Unix without SO_REUSEADDR). To get the Unix-style SO_REUSEADDR semantics on Windows, you can disable SO_EXCLUSIVEADDR:

    sock.setsockopt(trio.socket.SOL_SOCKET, trio.socket.SO_EXCLUSIVEADDR, False)
    

    but be warned that this may leave your application vulnerable to port hijacking attacks.

  2. TCP_NODELAY is enabled by default.

  3. IPV6_V6ONLY is disabled, i.e., by default on dual-stack hosts a AF_INET6 socket is able to communicate with both IPv4 and IPv6 peers, where the IPv4 peers appear to be in the “IPv4-mapped” portion of IPv6 address space. To make an IPv6-only socket, use something like:

    sock = trio.socket.socket(trio.socket.AF_INET6)
    sock.setsockopt(trio.socket.IPPROTO_IPV6, trio.socket.IPV6_V6ONLY, True)
    

    This makes trio applications behave more consistently across different environments.

  4. On platforms where it’s supported (recent Linux and recent MacOS), TCP_NOTSENT_LOWAT is enabled with a reasonable buffer size (currently 16 KiB).

See issue #72 for discussion of these defaults.

The following methods are similar, but not identical, to the equivalents in socket.socket():

bind(address)

Bind this socket to the given address.

Unlike the stdlib connect(), this method requires a pre-resolved address. See resolve_local_address().

await connect(address)

Connect the socket to a remote address.

Similar to socket.socket.connect(), except async and requiring a pre-resolved address. See resolve_remote_address().

Warning

Due to limitations of the underlying operating system APIs, it is not always possible to properly cancel a connection attempt once it has begun. If connect() is cancelled, and is unable to abort the connection attempt, then it will:

  1. forcibly close the socket to prevent accidental re-use
  2. raise Cancelled.

tl;dr: if connect() is cancelled then you should throw away that socket and make a new one.

await sendall(data, flags=0)

Send the data to the socket, blocking until all of it has been accepted by the operating system.

flags are passed on to send.

If an error occurs or the operation is cancelled, then the resulting exception will have a .partial_result attribute with a .bytes_sent attribute containing the number of bytes sent.

sendfile()

Not implemented yet!

The following methods are not provided:

  • send(): This method has confusing semantics hidden under a friendly name, and makes it too easy to create subtle bugs. Use sendall() instead.

The following methods are identical to their equivalents in socket.socket(), except async, and the ones that take address arguments require pre-resolved addresses:

All methods and attributes not mentioned above are identical to their equivalents in socket.socket():

The abstract Stream API

(this is currently more of a sketch than something actually useful, see issue #73)

class trio.AsyncResource
abstractmethod forceful_close()

Force an immediate close of this resource.

This will never block, but (depending on the resource in question) it might be a “rude” shutdown.

abstractmethod await graceful_close()

Close this resource, gracefully.

This may block in order to perform a “graceful” shutdown (for example, sending a message alerting the other side of a connection that it is about to close). But, if cancelled, then it still must close the underlying resource.

Default implementation is to perform a forceful_close() and then execute a checkpoint.

class trio.SendStream
abstractmethod await sendall(data)
abstractmethod await wait_maybe_writable()
can_send_eof
abstractmethod await send_eof()
class trio.RecvStream
abstractmethod await recv(max_bytes)
class trio.Stream
staticmethod staple(send_stream, recv_stream)

Async disk I/O

Not implemented yet!

Subprocesses

Not implemented yet!

Signals

with trio.catch_signals(signals) as batched_signal_aiter

A context manager for catching signals.

Entering this context manager starts listening for the given signals and returns an async iterator; exiting the context manager stops listening.

The async iterator blocks until at least one signal has arrived, and then yields a set containing all of the signals that were received since the last iteration. (This is generally similar to how UnboundedQueue works, but since Unix semantics are that identical signals can/should be coalesced, here we use a set for storage instead of a list.)

Note that if you leave the with block while the iterator has unextracted signals still pending inside it, then they will be re-delivered using Python’s regular signal handling logic. This avoids a race condition when signals arrives just before we exit the with block.

Parameters:signals – a set of signals to listen for.
Raises:RuntimeError – if you try to use this anywhere except Python’s main thread. (This is a Python limitation.)

Example

A common convention for Unix daemon is that they should reload their configuration when they receive a SIGHUP. Here’s a sketch of what that might look like using catch_signals():

with trio.catch_signals({signal.SIGHUP}) as batched_signal_aiter:
    async for batch in batched_signal_aiter:
        # We're only listening for one signal, so the batch is always
        # {signal.SIGHUP}, but if we were listening to more signals
        # then it could vary.
        for signum in batch:
            assert signum == signal.SIGHUP
            reload_configuration()