Checking for SSE event hangs

[dsandy12]

Service_details.py:1027 hangs while getting an sse event.

System Configuration:
Ubuntu Jammy
Python 3.9
Urllib3 2.2.1
Sseclient-py 1.8.0
Requests 2.31.0
Command Line Parameters: https://127.0.0.1:8443, no-cert-check

Observed Sequence:
Server has already sent event to SSE stream

service_details.py:1027 calls read_sse_events()
service_details.py:645 retrieves event from event stream() - at this point there are no more bytes available in the stream and the first and only event has been read. Debug information shows "Dispatching message event..."
service_details:py:646 adds the newly retrieved event to the events collection
service_details.py:645 checks to see if there is another event available
init.py:55 for chunk in... attempts to read more chunk data
utils.py:211 for data in super... attempts to read more data
init.py:58 data = b'', getting ready to read new line of data
init.py:58 for line in ... - there are no more lines within the previously read chunk
init.py:45 for chunk in self._event_source - see if there is more chunk data in the event source stream
response.py (urllib3.response.HTTPResponse.read_chunked():1184 update the chunk length
response.py:1108 - attempt to read a new line (because chunk left is None) --> code blocks

Other Observations:
There may be some timing component to this problem. Setting breakpoints and/or single stepping through the code sometimes will produce different results.

[dsandy12]

Looking into this, I believe the hang is caused by a difference in the assumptions for the SUT. My SUT is very quiet.

The SSEClientTimeout._read() function (as used) will return as many events as are in the queue over a three second period (including those that are pre-queued). Reception of a final event after the three second period is required to trigger the timeout and break out of the event reading process.

In quiet systems, the SSEClientTimeout function will block indefinitely, waiting for the first event reception following the three second timeout period. A more robust solution would timeout after three seconds whether or not the final event is received.

[mraineri]

There have been other issues raised with SSE testing so far. We may back it out entirely and rethink how we're doing SSE testing in general.

[dsandy12]

I am contemplating rewriting the sseClientTimeout class to run continuously in a different thread. The events would be shared back in a thread-safe queue. The overall functionality beyond SseClientTimeout would look the same, but it would eliminate the need for a chatty Sse channel. If you think this would be useful, please let me know and I will write the code and do a pull request.

[mraineri]

It could be helpful; I haven't dug much into SSE, but I'm open to seeing what can address this and other SSE test issues.

[henbin]

Sorry that I have one question about below error.

SERV_SSE_CLOSE_CONNECTION_IF_EVENT_DEST_DELETED DELETE 200 /redfish/v1/EventService/Subscriptions/1 FAIL
After deleting the EventDestination resource, the connection appears to still be open Server-sent events: The service shall close the connection if the corresponding EventDestination resource is deleted.

Can anyone please tell me why SSE stream would be closed after deleting /redfish/v1/EventService/Subscriptions/1?

Thanks.

[mraineri]

There is a requirement in the Redfish Specification to do so. In the "Server-sent events" clause of the spec:

The service shall close the connection if the corresponding EventDestination resource is deleted.

However, as noted by issue #63, there may be issues where the socket itself in the test stays open even if the service is closing the stream. But the intent of this test is to verify the behavior required per the Redfish Specification.

[henbin]

There is a requirement in the Redfish Specification to do so. In the "Server-sent events" clause of the spec:

The service shall close the connection if the corresponding EventDestination resource is deleted.

However, as noted by issue #63, there may be issues where the socket itself in the test stays open even if the service is closing the stream. But the intent of this test is to verify the behavior required per the Redfish Specification.

Thanks for your response.
I manually open SSE connection and find the EventDestination URL (/redfish/v1/EventService/Subscriptions/1), and then DELETE it. Indeed, the SSE connection isn't closed automatically. I'll check the related implementation in the redfish service.