5

I have created a SSL socket (server side) and place the socket into a select() queue. The select() properly returns when the socket "is ready" to read.

I then recv(1024) bytes. Under some circumstances this will get all the data, in others it may not.

However, if there is still data in the socket buffer (because I didn't recv() it all), and I pass that same socket into select() again, it will not be returned as being "ready" to read even though I know there is data there.

I suppose my question is really to confirm what "ready to be read" really means from select()'s perspective and what the best way to handle this would be. Continuing to recv() until EWOULDBLOCK seems sort of hack-ish given that I'm using select().

Am I thinking about this incorrectly? I realize I could use a larger recv buffer, but there is always the possibility that there would be more to read than recv can pull -- so what is the "right" way to handle this coming out of a select()?

Thanks in advance.

EDIT: As noted in the comments, I neglected to mention that this is an SSL server and apparently select() behaves differently when using wrapped sockets.

RLI123
  • 487
  • 3
  • 13
  • Take a look at this http://stackoverflow.com/questions/7749341/very-basic-python-client-socket-example and the example server code. You should use listen() and accept() to your server side socket, making the socket available for subsequent calls. – Hannu Oct 31 '16 at 16:44
  • @Hannu, my case is well after the listen()/accept() phase. The question you pointed me to doesn't have any of the elements of my issue. Thanks anyway. – RLI123 Oct 31 '16 at 16:49
  • https://pymotw.com/2/select/ does this explain it more thoroughly? Basically "readable" means different things if you are talking about your master socket you are listening to, or a socket created by accept()ing a connection to your master socket. In the latter case, it means there is data to be read, in the first case it means there is a pending connection you should or could accept(). – Hannu Oct 31 '16 at 16:53
  • @Hannu - yes, but if you read my OP, there is still "data to be read" because I left it there, yet select() is not returning the socket as being ready to read. That is the entire point of my question which, unless I'm missing something, is not addressed in the links you've provided. – RLI123 Oct 31 '16 at 16:55
  • Do you use listen and accept? If not, then it (probably) is as expected, as select for the server socket returns "there is a connection attempt to be accepted". You only make one connection from your client, and subsequent select() would not return anything. You can only use select() to determine if there is data to be read if you have created the socket by accept()ing a connection to the server socket. If you already do this, then I am sorry for wasting your time. If you don't do this, you can't use select to determine if there is data. Only to detect a connection attempt. – Hannu Oct 31 '16 at 17:14
  • With a normal TCP socket `select` will show if data are ready for `recv`, even if you just read part of the available data. It will be different with SSL sockets though. – Steffen Ullrich Oct 31 '16 at 17:15
  • @Steffen Ultich - that's it!! This is an ssl connection so select() is behaving differently than I expected. I'll have to research the correct way to process this difference for SSL recv calls. – RLI123 Oct 31 '16 at 17:18
  • 1
    It might have helped if you included into your question that this is a SSL socket. Maybe add it now. – Steffen Ullrich Oct 31 '16 at 17:20
  • I will. Although had I known that was relevant to the issue, I probably wouldn't have needed to ask the question in the first place. Thanks for setting me in the right path. – RLI123 Oct 31 '16 at 17:23
  • And please ignore everything from me as I didn't think about SSL. Good if you managed to solve your problem! – Hannu Oct 31 '16 at 17:25
  • You may want to read [ask] and [mcve]. – boardrider Oct 31 '16 at 17:57
  • Actually, although I added a note to my question, I actually did say it was an SSL socket (first line). The answer to my question was precisely what @Staffen Ullrich stated -- that there is a fundamental difference in how select() works between regular and ssl sockets. I'm not sure what additional information I could have provided that would have made a difference -- but thank you so much for pointing me to the FAQ's anyway. – RLI123 Oct 31 '16 at 18:26

1 Answers1

7

select cares only about the readiness of a socket from the view of the OS kernel. That is checking if a socket is ready for recv with select only checks if there are data in the socket buffer. But with SSL it is different since user space buffering is involved.

Even if you read only a few bytes from a SSL socket it needs to read the full SSL record which contains the encrypted data, decrypt the full record and then it can return the few bytes you've requested. The rest of the data will be buffered in user space for the next read. But, the full SSL record is removed from the OS socket buffer which means that select might not show you that there are still data available.

There are two ways to work around this. One way is to use the pending method to find out of there are still data buffered in user space. The other is to always recv in huge chunks so that no data will be buffered in user space. Since the maximum size of a SSL record is 16k and each recv handles only a single SSL record (implementation detail in openssl SSL_read) it would work to always call recv with a size of at least 16384.

Steffen Ullrich
  • 114,247
  • 10
  • 131
  • 172