trying to troubleshoot LWS_CALLBACK_WS_CLIENT_DROP_PROTOCOL #3185
Comments
|
This problem is recreatable against a few different servers:
But:
Any tips on how to troubleshoot this further? I am sure I am missing something obvious, but I cant figure out from the logs exactly what lws doesn't like that is causing it to drop the connection |
|
still looking for help on this one.... If I set a breakpoint where the 'drop protocol' is triggered and examine the wsi, this is what it shows in case this provides any insight to someone that can help.. |
|
bit more info..it is bombing out in ops-ws.c line 1233, I am not sure exactly what "we are spinning means" -- any help here?? Even if my code does nothing but log a message on each incoming binary frame (not even process them) I still get this event/error for the specific server scenarios I mentioned above (and not for the others). |
|
OK I am getting further in understanding this problem. First of all, I mischaracterized the problem above -- this problem has entirely to do with the use of SSL over a websocket (wss). I experience it every time using wss as an lws client, and never when using unencrypted tcp, i.e. ws. From looking at the code, I can see the problem comes here in the rops_handle_POLLIN_ws function. Having done one read operation in response to the poll event, we call this function to see if there is more data to read. As we can see here, this always returns 0 in the case of unencrypted websocket connection (tcp). Therefore with tcp we always do one read per poll, and everything works fine. However, in the case of wss, if SSL_pending returns a non-zero number of pending bytes, then lws loops and does another read. And after the second read it will check again and if once again there is more data to read it will do so again. And so on, until it has reached a limit of 10 reads in the same poll event. At that point it returns the LWS_CALLBACK_WS_CLIENT_DROP_PROTOCOL event which closes the socket. The implication is that in a case like mine, with a server returning a large number of small frames very rapidly (again, this is a streaming audio application), this limit is easily reached and the websocket is closed. I tested by modifying the code in two different ways, each of which eliminated the problem:
Either of those changes seemed to fix my problem -- the full audio stream is heard and the websocket is not closed. I don't necessarily consider either of those options to be a good solution though without a better understanding more of why the current code is written like this. This assumption that all pending data can be read within 10 tries, given the sending rate of the server may be quite high, seems a perilous assumption to me. Perhaps I am missing something? Surely others have used lws as a client to high performance streaming servers, is there some option I am not setting or some issue in my code that surfaces this problem for my scenario? I'd really love some thoughts from the lws team here.. |
|
It's like this because on some platforms, loss of network connection to the peer manifests itself as endless 0 length receives. On other platforms, 0 length tls receive is a legal event unrealted to loss of network connection. So in an attempt to find a way to deal with both, it has a sanity check for just how often it is spinning and recover from what is otherwise and endless loop by dropping the connection. It sounds like you can take the approach to reset sanity to 10 or whatever on seeing a positive amount of received data and solve it that way... do you want to try that and propose a patch? |
|
ok, interesting! I did log the length received each time and did receive an occasional zero-length read, but infrequently and not enough to be the cause I don't think. I will do some more investigating.. |
|
I'd be interested in testing this as well as we have an application connecting to multiple servers all running libwebsocket. We too are sending many small messages over wss. |
I've been using lws for years now, and just today I am having a strange problem connecting as a websocket client to a remote server. My code, as client, is streaming audio to that server and that server is also streaming audio to me. Roughly 2 seconds after connecting to this server I consistently see a LWS_CALLBACK_WS_CLIENT_DROP_PROTOCOL and the connection is dropped from my side.
I am using lws 4.3.3, and I have a full debug log here.
The last bit of the log leading up to receiving the LWS_CALLBACK_WS_CLIENT_DROP_PROTOCOL looks like this
I've connected with the same application to other servers, and I am trying to gain some insight into why this one fails and lws closes the connection. I'd appreciate any thoughts on what I might be doing wrong...Thanks!
The text was updated successfully, but these errors were encountered: