The socket is one of the most crucial primitives for systems communication. It is the endpoint on which an application can send and/or receive information, either between processes on the same system, or to any other network-connected system. Tens or perhaps even hundreds were used behind the scenes for the simple task of getting this blog post to reach your screen. They are a fundamental aspect of how applications and systems communicate with each other.

In Linux, the syscall which creates a socket (and in fact is part of the POSIX standard) accepts three parameters:

• Domain
• Type
• Protocol

Each of these warrants some deeper exploration, as the choice of parameters here can wildly change the behavior of the resulting socket.

Domains

The “domain” you select for your socket has dramatic impact on the kind of infrastructure (both software and physical) ends up being used “behind” the socket. A handful of perhaps the most familiar options includes:

• AF_INET - IPv4
• AF_INET6 - IPv6
• AF_UNIX - Unix Domain Sockets (UDS) (i.e. a local/IPC socket, usually available via some named path on the filesystem)

Each of these isn’t just some parameter that slightly changes behavior, but rather a “pointer” to a full implementation, each of which has its own rules and decision-making.

I learned my lesson from my IP freebind blog post and am keeping in mind that while it can be tempting to view v4 and v6 as two sides of the same coin, they are indeed totally separate implementations.

While the supported list in Linux is much longer than the few I provided above, the vast majority of the use cases I encounter in my day to day map to one of these.

Types

In addition to the domain, when creating a socket you must also specify its “type”. The socket type changes some of the behavior of the socket itself.

Concepts

To understand socket types, we must first understand a few concepts which are often mentioned in their descriptions. They’re a bit abstract; the specific and tangible way these concepts manifest themselves (and the constraints they operate within) might depend on other factors, such as the combination of domain and type for a given socket.

• portability - though this blog post will focus on the usage of sockets in Linux, it’s useful to know what is or is not also available on other operating systems. Some socket options, types, etc are part of the POSIX standard and so are available on other operating systems, but others are Linux-specific.

• preserving message boundaries - When you open a socket, at some point you’ll have to send some data into it. But how will the receiver know when it has received all the data? This is a task known as “framing”. Some application-level protocols have their own framing mechanisms but some simply expect that each chunk of data contains the full message.

  Some socket types work best when the application has its own notion of framing, and can take liberties with the transmitted data by chunking it up into smaller portions (“byte stream”) when/if needed to fulfill other considerations like reliability, but also to operate within other constraints like MTU or internal system buffers/limits. When the application cannot handle this, a socket type which “preserves message boundaries” is needed instead. This means that any time the socket has data passed to it, it will be kept as-is. This could mean that the message is unexpectedly truncated or even dropped entirely if too large.

  Think about it like packing luggage for air travel - you can either pack everything into one huge bag, or break things up into a few smaller bags. You might prefer to consolidate into a single bag to save money, but then you run the risk of that bag being too big/heavy and then you’re really out of luck. It all depends on your budget, and the nature of the contents of your luggage.

• reliability - Again, in the context of socket types, a bit of an abstraction, because what actually provides the reliability? Well, this is where “it depends” comes in. For instance, the way reliability is provided between AF_UNIX is very different from AF_INET, AF_BLUETOOTH etc. But in general, if you select a type which says it requires reliability, something will provide this. The most common example here is AF_INET or AF_INET6 with SOCK_STREAM - this defaults to using TCP over the network, which has well-understood reliability features.

• connection-oriented - When two networked peers communicate, they may send one message, they may send multiple. Sometimes each message is meant to be self-sufficient - the canonical example being DNS over UDP, where a single request is made, and a single reply is expected. No “connection” or even “session” is used here, it’s just a single back and forth. Other times, several messages may be passed back and forth, all within the same “session context”. One of the biggest considerations in this case is how each peer will know when the other side is “done” sending messages?

  A socket type which is not connection-oriented, such as SOCK_DGRAM, does not offer any way to determine this - each message is received and sent in isolation, and if a peer wishes to inform the other side it is complete, there needs to be some kind of application-specific signaling for doing this.

  Connection-oriented socket types, on the other hand, make this a lot easier to do at the systems level. In the same way that socket types which preserve message boundaries implicitly provide insight into when a particular “message” is complete, connection-oriented socket types have built-in mechanisms for signaling that the “session” or “connection” is closed, and the peer is done sending messages.

  The ergonomics on connection-oriented sockets differ from connectionless in the syscalls required as well, as we’ll see in upcoming examples.

Linux Socket Types

Below is a list of the socket types which are practically supported in Linux and are most commonly used. Their names (and the descriptions from socket(2)) are as follows:

• SOCK_STREAM: “Provides sequenced, reliable, two-way, connection-based byte streams. An out-of-band data transmission mechanism may be supported.” – socket(2)

  As mentioned above, SOCK_STREAM is commonly paired with AF_INET/AF_INET6 for using TCP over the network. Obviously this means that the application can enjoy the reliability features provided by TCP. It also can rely on systems-level mechanisms for determining when the connection to the peer is closed, either because of a problem or because the peer is done sending messages. When calling recv() or similar, the length of the received data is returned. Conventionally, if the value returned is 0, this signifies the end of the connection.

  The application isn’t totally off the hook for other things, however. SOCK_STREAM does not preserve message boundaries - and it certainly couldn’t in this example, as TCP regularly chops up payloads to accommodate MTU, sliding window sizes, etc. This means the application must have its own mechanism for determining when it has received a complete message - “framing”. A common example is HTTP1 (HTTP2/HTTP3 have different framing techniques), which uses things like CRLF delimiters and the Content-Length header so the receiver knows when a full request or response has been completely received. Other applications will need to employ similar mechanisms when message boundaries aren’t guaranteed to be preserved.

• SOCK_DGRAM: “Supports datagrams (connectionless, unreliable messages of a fixed maximum length).” – socket(2)

  This is a common choice for applications like DNS over UDP. AF_INET/AF_INET6 + SOCK_DGRAM defaults to UDP over the network, and unlike SOCK_STREAM, message boundaries are preserved. This is ideal for applications like UDP where we simply need to just get the whole message in one datagram and send it to the other side, so neither side has to do its own framing to determine when a message is complete.

  However, preserving message boundaries does come with its own potential challenges. Over the network, MTU is always a concern - care must be taken to ensure messages don’t get too big, else they might get dropped. DNS for instance may truncate a response it knows to be too big, and set the TC bit so the receiver can try again over TCP. Even AF_UNIX usage doesn’t totally bypass these problems, as maximum datagram size or internal buffers limitations can have a similar effect.

  Also unlike SOCK_STREAM, no reliability or connection-awareness is provided. Some applications don’t care about either of these - for instance, streaming audio/video typically handles session establishment via a totally separate protocol, often over a connection-oriented protocol, and the data is sent unreliably and separately to this. However, other protocols like QUIC have their own reliability mechanisms.

• SOCK_SEQPACKET: “Provides a sequenced, reliable, two-way connection-based data transmission path for datagrams of fixed maximum length; a consumer is required to read an entire packet with each input system call.” – socket(2)

  SOCK_SEQPACKET is one that I wasn’t always familiar with until I got deeper into systems programming. One reason is that unlike the other two types we’ve mentioned, this one is Linux specific. If you want to build a portable network application for many operating systems, you likely won’t want to rely on his. Another is that even on Linux, you can’t use it for all cases. At least in my experience, the main combination in which this is useful is with AF_UNIX - the combination of which gets you preserved message boundaries but also the implicit reliability of inter-process communication, and without overhead of network protocols like TCP or UDP.

  In other words, if you know you’re on Linux, and you just want to have two applications talk to each other on the same machine, SOCK_SEQPACKET might be a good fit. It’s connection-oriented (built in EOF detection just like SOCK_STREAM - recv() returns 0 when closed) but also preserves message boundaries so you don’t need a framing protocol. Obviously there are other ways of doing inter-process communication, but I find this to be a great one to keep in your back pocket if you need something relatively simple.

  One note is that the implicit reliability of using a local system socket doesn’t mean you avoid the potential pitfalls of preserving message boundaries on the local machine. SO_RCVBUF and SO_SNDBUF are socket options which can be set on a SOCK_SEQPACKET socket which can impact the allowed size of messages being received or sent. Exceeding this can result in unexpected blocking, dropped messages, or returned errors (this is still better arguably than SOCK_DGRAM which has a lot more scenarios where messages are simply dropped silently).

There’s also SOCK_RAW, but this one is a bit of an outlier. It definitely has its place, but it requires elevated permissions, and requires that the application take a much more low-level role. So I’ll cover this in a separate post.

As I go through this list, I think about the tradeoffs between these and the following venn diagram emerges in my mind (there is no shared bit between the three of them, only the edges between adjacent two). It really helps to quickly remind me of the tradeoffs being made between each when I consider which I want for my given use case.

Protocol

Each combination of <domain> + <type> has its own set of compatible protocols (or perhaps just one) which can satisfy the requirements of that combination. One of these is the default for that combination, which we can automatically choose by providing 0 as the third parameter for socket(). This is an overwhelmingly common thing to do when working with sockets - overriding the default protocol for a given socket is pretty atypical.

For instance, when we used SOCK_DGRAM with AF_NET this resulted in UDP being used as the default. But other protocols may be supported by that domain if we choose them, and it also doesn’t mean that other domains will have that same default. UDP wouldn’t make much sense for AF_UNIX + SOCK_DGRAM, nor would TCP for AF_UNIX + SOCK_STREAM, especially considering the reliability considerations for AF_UNIX is very different from AF_INET.

If a chosen protocol is not supported, the kernel will return EPROTONOSUPPORT in response to the call to socket().

Practical Examples in Python

Python has first-class support for working with sockets at a low level (meaning, little/no abstraction over the basic syscalls). It’s often my go-to for experimentation, particularly with some of the more niche combinations of domain, type, socket options, etc.

AF_INET + SOCK_STREAM

Server implementation below:

 1import socket
 2
 3# In python, socket.socket() has only domain and type as required parameters.
 4# "protocol" defaults to 0, which is fine because that's what we want anyways.
 5# This combination will result in TCP/IP
 6s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 7
 8# bind() tells the kernel which IP and port to use for this socket
 9s.bind(("127.0.0.1", 8123))
10
11# listen() sets the socket in "server mode"
12s.listen()
13while True:
14
15    # This will block until the client connects to the IP and port above
16    (conn, address) = s.accept()
17    chunk_size = 4
18    read_len = 0
19    print(f"Accepted connection from {address}")
20
21    # This loop will read until there's no more data left
22    # (note that since we're using SOCK_STREAM the data may
23    # arrive in any number of "chunks" for any number of reasons)
24    while True:
25        data = conn.recv(chunk_size)
26        if not data:
27            break
28        read_len += len(data)
29    print(f"Received {read_len} bytes")

The client is a bit simpler:

 1import socket
 2
 3# Important to use the same domain/type as our server, otherwise
 4# we won't be speaking the same protocol (in this case TCP/IP)
 5s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 6
 7# SOCK_STREAM is a connection-oriented type, so we must call connect()
 8# to trigger the necessary connection setup (in this case TCP 3WHS)
 9s.connect(("127.0.0.1", 8123))
10s.send(b"Hello!")
11
12# Clean up this connection gracefully so that server doesn't
13# get RST from us later when it closes on its side
14s.close()

Running the server first, and then the client provides the resulting output on the server:

mierdin@t-bug:~/socket-examples $ python3 sock_stream_server.py 
Accepted connection from ('127.0.0.1', 32956)
Received 6 bytes

We can use tcpdump to explore how this actually hits the “wire”:

mierdin@t-bug:~ $ sudo tcpdump -i lo port 8123
17:22:23.205820 lo    In  IP localhost.56384 > localhost.8123: Flags [S], seq 3429665826, win 65495, options [mss 65495,sackOK,TS val 2994432284 ecr 0,nop,wscale 7], length 0
17:22:23.205836 lo    In  IP localhost.8123 > localhost.56384: Flags [S.], seq 3677951100, ack 3429665827, win 65483, options [mss 65495,sackOK,TS val 2994432284 ecr 2994432284,nop,wscale 7], length 0
17:22:23.205850 lo    In  IP localhost.56384 > localhost.8123: Flags [.], ack 1, win 512, options [nop,nop,TS val 2994432284 ecr 2994432284], length 0
17:22:23.205884 lo    In  IP localhost.56384 > localhost.8123: Flags [P.], seq 1:7, ack 1, win 512, options [nop,nop,TS val 2994432284 ecr 2994432284], length 6
17:22:23.205890 lo    In  IP localhost.8123 > localhost.56384: Flags [.], ack 7, win 512, options [nop,nop,TS val 2994432284 ecr 2994432284], length 0
17:22:23.205904 lo    In  IP localhost.56384 > localhost.8123: Flags [F.], seq 7, ack 1, win 512, options [nop,nop,TS val 2994432284 ecr 2994432284], length 0
17:22:23.206007 lo    In  IP localhost.8123 > localhost.58412: Flags [F.], seq 1, ack 8, win 512, options [nop,nop,TS val 2994432284 ecr 2994426418], length 0
17:22:23.206021 lo    In  IP localhost.58412 > localhost.8123: Flags [.], ack 2, win 512, options [nop,nop,TS val 2994432284 ecr 2994432284], length 0
17:22:23.251344 lo    In  IP localhost.8123 > localhost.56384: Flags [.], ack 8, win 512, options [nop,nop,TS val 2994432330 ecr 2994432284], length 0

AF_INET + SOCK_DGRAM

This is considerably simpler because as SOCK_DGRAM isn’t a connection-oriented type, there is no listen() or connect() necessary.

Server:

 1import socket
 2
 3# Same domain as before but now using SOCK_DGRAM for the type.
 4# This combination will result in UDP over IP.
 5s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 6
 7# Can re-use same IP and port number from previous example;
 8# different protocols so there is no port conflict here.
 9s.bind(("127.0.0.1", 8123))
10
11while True:
12    # SOCK_DGRAM is not a connection-oriented type, so there is no "listen()"
13    # like there was for SOCK_STREAM. There is no streaming, we just wait
14    # to receive a datagram.
15    payload = s.recv(10)
16    payload_len = len(payload)
17    print(f"Received {payload_len} bytes")

Client:

1import socket
2s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
3
4# Again, no connect(). Just send the payload to the IP and port specified
5s.sendto(b"Hello!", ("127.0.0.1", 8123))

And tcpdump shows in this case just the single UDP datagram.

mierdin@t-bug:~ $ sudo tcpdump -i any port 8123
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
17:38:00.536231 lo    In  IP localhost.38397 > localhost.8123: UDP, length 6

AF_UNIX + SOCK_SEQPACKET

Remember that SOCK_SEQPACKET is not supported by AF_INET/AF_INET6. This can be trivially reproduced by trying it:

mierdin@t-bug:~/socket-examples $ python3
Python 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.socket(socket.AF_INET, socket.SOCK_SEQPACKET)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/socket.py", line 232, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 94] Socket type not supported

So we’re going to switch to AF_UNIX for this. As mentioned earlier, this allows us to use Unix Domain Sockets (UDS). I plan to explore these in greater detail in a future blog post, but for now, suffice it to say that these allow us to bypass most of the traditional network stack in Linux which would require protocols like TCP and UDP, and communicate much more simply between applications on the same machine, typically using some named path on the filesystem as an endpoint. So instead of listening on an IP and port, we’d listen on something like /tmp/myunixsocket.sock.

AF_UNIX can be used with all three of the types we’ve explored here, but for brevity we’ll just use SOCK_SEQPACKET.

Server:

 1import socket
 2import os
 3
 4s = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET)
 5socket_path = "/tmp/myunixsocket.sock"
 6
 7# When we call `bind()` with the path above, the OS will actually create this path for us.
 8# However, it will not be cleaned up automatically. This can be handled any number of ways,
 9# but in this case we can just ensure the file is removed when this server starts, just
10# before the call to `bind()`
11try :
12    os.remove(socket_path)
13except OSError :
14    pass
15
16# We still need to bind() but note that we're passing a filesystem path
17# here, not an IP/port (latter not appropriate for this type)
18s.bind(socket_path)
19
20# SOCK_SEQPACKET is connection-oriented, so again this places this socket into "server mode"
21s.listen()
22
23while True:
24
25    # Again this will block until our client connects
26    conn, addr = s.accept()
27
28    # Unlike SOCK_STREAM we will not get data streamed to us `SOCK_SEQPACKET` preserves message boundaries.
29    # So we need to ensure we call `recv()` with enough space to receive any message the client might send.
30    # Otherwise it might get truncated.
31    payload = conn.recv(4096)
32    payload_len = len(payload)
33    print(f"Received {payload_len} bytes")

Client:

 1import socket
 2
 3# Unlike AF_INET, where choosing the wrong protocol might result in things
 4# just "not working", AF_UNIX being a local endpoint for both client and
 5# server means the kernel will know if there's a mismatch.
 6#
 7# For instance, if we choose `SOCK_STREAM` here and our server's socket
 8# type is `SOCK_SEQPACKET, we'll get:
 9#
10#   OSError: [Errno 91] Protocol wrong type for socket
11s = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET)
12
13# SOCK_SEQPACKET is a connection-oriented type, so we must call connect()
14s.connect('/tmp/myunixsocket.sock')
15
16# Now we can send!
17s.send(b"Hello!")
18
19# Like with TCP, calling close() here closes the connection client-side, but
20# some applications may conventionally close on the server side, and both cases
21# should probably be handled gracefully for production use cases.

Capturing AF_UNIX traffic has been a little trickier than simply using your favorite packet capture tool like tcpdump, but similar tools have emerged in recent years that get pretty close. I’ll cover this and more in a future blog post.

Conclusion

If you’re looking for foundational Linux networking primitives, you could do worse than to learn more about how sockets work, how applications use them, and how they intersect with other concepts in Linux networking.

I hope this blog post, while extremely introductory, was useful. I aim to use this as a launching pad to dive into many more practical specifics on sockets on Linux in future blog posts.