Digi S6 reliability and/or MMW Overloaded

[neilh10]

Title: Initially Digi S6 reliability , then renamed to MMW Overloading, then renamed back to Digi S6 reliability

For a system "test06" running the Digi S6, Starting Sept 10 after 33days Oct 14th, it appeared to stop getting responses from the MMW.
Its not clear where the problem was, see tu_rc_test06_TimeSeriesResults _snap201204.xlsx
This was using a home SSID and was in the edge of the signal ~ -71db
However after Dec-06 changing to an outdoor SSID, requiring an update to the uSD, and system reset (not power cycle), on connecting to MMW some 5119 pending responses where then delivered in one transaction ~ See tty201204-1706test06.txt. Fantastic.

I'm assuming it was something to do with the reliability of the S6. This issue is to manage the reliability of the test Digi S6.
Possibly when the RSSI is not readable the modem should be software reset. Hardware power down or reset is not possible.

[neilh10]

After some extensive testing, by putting in a PACING delay of 2 seconds in transmitting to MMW, all readings are now being delivered. Practically, every time a new reading is POSTED, it creates a new TCP/IP link to data.envirodiy.org
With no pacing, when one POST was complete and TCP/IP torn down, it would then attempt the next link within about 100mS. It seems these where overwhelming data.envirodiy.org.
Occasionally on a manual processor RESET, or a greater inline passing PAUSE - then data.envirodiy.org would respond fast. Even in one case upload 3000 records one after each other.
Conclusion : it seems like MMW response had decayed. There wasn't an easy way to tell if it was WiFi Digi 6b hybrid or something in the network. It seems it was the MMW.

[neilh10]

Well after another week of testing, making small modifications and then another day of soak testing, and also major "Publisher Pacing". The Publisher Pacing appears to make MMW more reliable, when it does respond.

Found out that there was small periods that the Digi S6B was not responding to entering CMD mode.
The section of code is EnviroDIYPublisher.cpp ~ EnviroDIY#240

Not entering CMD mode is a critical failure of communications with S6B.
A fix for one critical event, was to put in a delay before attempting to poll the S6B,

The problem looked like this

+++++++++++++++Client waited 4008 mS for 0 bytes. Stopped after 1952 ms <--EnviroDIYPublisher
Rsp:' ' <--EnviroDIYPublisher

For no response from MMW, this is what the interaction with the S6B looks like

POST /api/data-stream/ HTTP/1.1
Host: data.envirodiy.org
TOKEN: d96bf9fb-faca-4cc3-bcb9-3d23255a1f3c
Content-Length: 408
Content-Type: application/json
{"sampling_feature":"79f702a9-368f-4940-9669-8978ffa3254b","timestamp":"2020-12-19T14:42:00-08:00","2ac49562-07cd-40a8-9165-d19905e37ac2":7,"8c796edd-7863-4fe7-9e54-0cbe0d694d59":4.23,"b03d4384-4623-4df5-b705-f50710d5e4e9":15.06,"67c22f5d-e5d8-4a26-a82b-7f59132e5c81":0.0474,"1ae0dc73-3d90-4c72-8d19-f594343f862c":52.2,"caf4c4e2-7925-4e7a-a629-e52c75e478e6":16.2,"e6159fe0-e30d-4a9d-bebc-1dc5c2435a22":19.25}
[[Client waited 3004 mS for 0 bytes. <--EnviroDIYPublisher
+++OK
ATTM
64
ATTM0
OK
ATWR
OK
ATAC
OK
ATTM64
OK
ATWR
OK
ATAC
OK
ATCN
OK
Client stopped after 459 ms <--EnviroDIYPublisher
Rsp:' ' <--EnviroDIYPublisher
-- Response Code -- 504 waited 3004 mS Timeout 3000

and for a response from MMW

POST /api/data-stream/ HTTP/1.1
Host: data.envirodiy.org
TOKEN: d96bf9fb-faca-4cc3-bcb9-3d23255a1f3c
Content-Length: 408
Content-Type: application/json

{"sampling_feature":"79f702a9-368f-4940-9669-8978ffa3254b","timestamp":"2020-12-19T14:32:00-08:00","2ac49562-07cd-40a8-9165-d19905e37ac2":2,"8c796edd-7863-4fe7-9e54-0cbe0d694d59":4.23,"b03d4384-4623-4df5-b705-f50710d5e4e9":15.06,"67c22f5d-e5d8-4a26-a82b-7f59132e5c81":0.0471,"1ae0dc73-3d90-4c72-8d19-f594343f862c":51.9,"caf4c4e2-7925-4e7a-a629-e52c75e478e6":16.5,"e6159fe0-e30d-4a9d-bebc-1dc5c2435a22":18.75}
HTTP/1.1 201 [[Client waited 481 mS for 12 bytes. <--EnviroDIYPublisher
Created
Server: nginx/1.10.3 (Ubuntu)
Date: Sat, 19 Dec 2020ion+++/json
Content-Length: 2
Connection: keep-alive
Vary: Accept
{}OK
ATTM
64
ATTM0
OK
ATWR
OK
ATAC
OK
ATTM64
OK
ATWR
OK
ATAC
OK
ATCN
OK
Client stopped after 465 ms <--EnviroDIYPublisher
Rsp:' HTTP/1.1 201 ' <--EnviroDIYPublisher
-- Response Code -- 201 waited 481 mS Timeout 3000

====================================
Similarly also there was this problem with S6B response:

updateModemMetadata Entering Command Mode: <--DigiXBeeWifi
++++++++++++++++++++++++++++++Raw signal quality( 5 ): 0 <--DigiXBeeWifi
+++++++++++++++Raw signal quality( 4 ): 0 <--DigiXBeeWifi
+++++++++++++++Raw signal quality( 3 ): 0 <--DigiXBeeWifi
+++++++++++++++Raw signal quality( 2 ): 0 <--DigiXBeeWifi
+++++++++++++++Raw signal quality( 1 ): 0 <--DigiXBeeWifi

The core issue is the lack of response to 10 attempts at +++ ~ which is likely to be very deep in TINY_GSM
The RSSI was turned OFF, as it's not a critical reliability, and the focus became the listening for the response from a POST

[neilh10]

In 0.28.5 S6B WiFi parameters being read before being written so doesn't wear down flash.
Tried with brand new Xbee S6B -
XbeeWiFi internet comms with Digi XBee Wi-Fi Mac/Sn 4F3150CEE HwVer 2730 FwVer 2026
However still not getting a MMW response after 1st connection. Possibly more reliably bad after sleep, maybe socket not had enough time to establish
Idea: a) try and validate a tcp connection has formed possibly a ping before hand to verify connection after sleep or b) delay before using channel to allow socket to be established ,

[neilh10]

Testing against https://github.com/neilh10/ModularSensors/releases/tag/v0.30.0.release1_211012.
Testing on Mayfly with Digi WiFi S6 module mayfly_0.30.0_211012_1719_nano.hex
Testing on Mayfly with Digi LTE module mayfly_0.30.0_211012_1719_LT5_lte.hex

[neilh10]

Merged to EnviroDIY/ModularSensors (develop). EnviroDIY#194
Lots of mods by Sara - some nice c++ features in updateModemMetadata(). Takes more space.