SO_LINGER on Non-Blocking Sockets

Posted by Nybek on Apr 29, 2015

Introduction

This post is a follow-up to Cross-Platform Testing of SO_LINGER. Here we look at the effects of setting SO_LINGER on non-blocking sockets. We are using the same test platforms as before.

Our main two observations are:

  • With one inconseqential exception, the results for setting SO_LINGER to {on, 0} on non-blocking sockets are the same as for blocking sockets.

  • Setting SO_LINGER to {on, N} where N > 0 on a non-blocking socket on Linux is a particularly bad idea. Even though the the socket is in non-blocking mode, this call will block.

Table of Contents

Results for SO_LINGER set to {on, 0}

The results here are the same as for blocking sockets with one small exception. On the BSDs and Darwin, setting SO_LINGER on a Listening Socket to {on, 0} still does not have the desired effect, but instead of blocking, the call to close() will return immediately without setting EWOULDBLOCK.

The result table looks exactly the same as for blocking sockets:

Effect of SO_LINGER set to {on, 0}
Abort on close() Abort on close() after shutdown()
Listening Socket Connected Socket Listening Socket Connected Socket
BSD No Yes No Yes
Darwin No Yes No Yes
Illumos Yes Yes Yes Yes
Linux Yes Yes Yes Yes
Windows (Cygwin) Yes Yes No No
Windows (Native) Yes Yes No No

Results for SO_LINGER set to {on, 20}

As we did with blocking sockets, let's look at the relevant passage from UNIX Network Programming (page 203), only this time we will consider it from the point of view of non-blocking sockets:

If l_onoff is nonzero and l_linger is nonzero, then the kernel will linger when the socket is closed (p.472 of TCPv2). That is, if there is any data still remaining in the socket send buffer, the process is put to sleep until either: (i) all the data is sent and acknowledged by the peer TCP, or (ii) the linger time expires. If the socket has been set to nonblocking (Chapter 16), it will not wait for the close to complete, even if the linger time is nonzero. When using this feature of the SO_LINGER option, it is important for the application to check the return value from close, because if the linger time expires before the remaining data is sent and acknowledged, close returns EWOULDBLOCK and any remaining data in the send buffer is discarded.

Although not very clear, we have distilled the passage down to the following two expected behaviours for non-blocking sockets:

  • Don't block on close
  • Abort on linger timeout: "if the linger time expires before the remaining data is sent and acknowledged... any remaining data in the send buffer is discarded". We also assume that an RST is then sent to the peer, though the above passage does not actually mention this.

The passage above doesn't say whether or not EWOULDBLOCK is returned when close() is called and there is still data in the socket send buffer. However, we will list the behaviour we observed.

The results table has the same layout as we used for blocking sockets except for the last column which considers which considers whether EWOULDBLOCK is returned on close() rather than on linger timeout:

Effect of SO_LINGER set to {on, 20}
Block on close() Time spent blocking (secs) Abort on linger timeout Return EWOULDBLOCK on close()
BSD No 0 No Yes
Darwin No 0 No No
Illumos No 0 No No
Linux Yes 20 No No
Windows (Cygwin) Yes * Until FIN is ACK'd * No No
Windows (Native) No 0 Yes ** Yes **

* On Windows (Cygwin), if close() is preceded by a call to shutdown() 'Block on close()' becomes "No" and 'Time spent blocking' becomes "0".

** On Windows (Native), if close() is preceded by a call to shutdown(), 'Abort on linger timeout' becomes "No" and 'Return EWOULDBLOCK on linger timeout' becomes "No".

Two points are worth noting about the results:

  1. The result line for Linux above should alert you that setting SO_LINGER to {on, N} where N > 0 on a non-blocking socket is an exceedingly bad idea. Although the socket is in non-blocking mode, the call to shutdown will block until the payload is fully dispatched or the linger timeout occurs.

  2. As was the case with blocking sockets, calling shutdown() before close() renders the SO_LINGER setting ineffective on Windows (native and under Cygwin).

Other Observations

As was the case with blocking sockets, any attempts to set SO_LINGER on a Connected Socket on Illumos (OpenSolaris) after a call to shutdown() will fail with EINVAL.

Download Code and Result Data

The code for the command line tools used in the tests is available on GitHub: linger-tools

The test result captures, including tcpdump traces, are available here: nonblock-linger-test-results.tar.gz

References

Stevens, W.R., Fenner, B., and Rudoff, A.M. 2004. UNIX Network Programming, Volume 1 (3rd edition): The Sockets Networking API . Addison-Wesley, Boston, Massachusetts.