JoinAccept ignores JoinChannel if CFList is empty, causing excessive delays in first uplink

[bwesen]

On the Multitech GWs in US915, they only use 8 channels at a time out of the 64 (called a sub-band).

The JOIN as sent from the stack, is randomly sent among the subbands until one goes through to the GW. This process in itself is annoying and takes a long time, but then the problem repeats for the first uplink message, as it also has to cycle through the subbands unless a mechanism directs the stack to only use the correct subband.

There are 2 such mechanisms and 1 mitigation available as far as I can see, but none work with this stack and the Multitech.

1. After a successful Uplink received at the GW, the GW will send a Link ADR back to the stack, telling it which channels to limit to. The Multitech sends this, so everything works fine after the first Uplink is (eventually) received but takes a very long time in some cases.

2. The JoinAccept can relay a CFList from the GW to the stack, directly telling the stack which channels it supports. This is implemented in the stack but unfortunately the Multitech GW does not send anything here (!)

3. If the stack receives an empty CFList, the rule is that you are supposed to send the first Uplink on the same channel you used for the successful JoinRequest - this is decoded on the stack into applyCFList.JoinChannel, but never used (this is a bug or a TODO I guess?). See ProcessRadioRxDone(). If this had worked, then 1) would solve the problem as the first Uplink would come on the correct subband and then the Link ADR request would solve it.

I've noted that this issue of subbands in the Join procedure is a common Lora issue, see for example this Murata thread where they seem to have the exact same problem as I have:

https://community.murata.com/s/question/0D55F00009NVnW3SAL/channel-mask-issue

I didn't look closely at the 4.6.0 release but couldn't find anything in the release notes about this (I'm on 4.5.2 currently).

[mluis1]

First thing to note is that this project strictly follows the LoRaWAN specifications. Furthermore the LoRaWAN stack implementation is validated by the LCTT LoRaWAN certification tool.

US915 and AU915 regions where designed to use 64 channels gateways.
In practice a lot of network providers are trying to use 8 channels gateways which is not compliant with LoRaWAN specifications.

Therefore when using 8 channels gateways one needs to accept the drawbacks induced by their usage.

To answer your points 1 and 2 it is up to the network providers to correctly handle the usage of 8 channels gateways when 64 channels gateways are required. The best solution is to have the network server to send the CFList with correct channel mask when transmitting the JoinAccept.

Concerning point 3 there is nothing in the specifications saying that the same channel used for the JoinReq must be used for the first uplink. Furthermore this would be unlawful according to the FCC regulations where the channels must hop between transmissions.

The observed behavior will be the same for all versions of this project.

As a side note: The LoRaWAN communications are between an end-device and a Network Server. A gateway is an entity that converts received/transmitted LoRaWAN frames from one medium to another (RF from/to IP network). A gateway role is passive.

In the future it would be nice if you could post this kind of questions on the project Discussions tab. It is a better place to engage discussions and then we can agree if it is an issue or not.
If you don't mind I move this issue to the Discussions tab.

[bwesen]

Well, as this is the default behaviour if you buy a Multitech Lora GW and use this stack (that it doesn't work in US or AU properly) I and many others would consider it an "issue" and not a "discussion" but you're the boss :) Now at least the Issue is described here so others can find it.

I get your point about the frequency hopping requirement, but the behaviour could be that it uses a channel in the same sub-band (group of 8 channels) for the first Uplink then, you would hop but still match the correct sub-band the GW is configured for.

I guess this can be hacked into the RegionUS915.c file, as it has the applyCFMask.JoinChannel in the ApplyCFList function first, and then in the NextChannel selection callback it could use this to prioritize the available channels in the same group, if it can know that this is the first Uplink after a successful Join. If you know a state variable to check (inside the region code) that can determine if the requested channel is the first after a join, I can hack this together myself.

[mluis1]

I understand your point.

But, specifications mandate the usage of 64 channels gateways for US and AU regions. This means that the described issue is not related to LoRaWAN end-device stacks implementations as these need to handle all 64 channels defined for US or AU regions. Please note that all stacks should behave in the same way as this is verified by the LoRa-Alliance certification process.

As most people cannot afford to buy 64 channels gateways the LoRaWAN specification allows the usage of the JoinAccept CFList to restrict the number of channels to be used. By using the CFList one can then use 8 channels gateways.

Forcing the first uplink to be done under the same sub-band as the JoinReq is possible and we can consider adding this feature in a future release.

[bwesen]

Thanks Miguel, I think that compromise might be a useful stack option to handle cases like this (the same-subband thing).

I'll also check with Multitech on the CFList issue, as that feels like the more proper workaround as well.

[reissjason]

2. The JoinAccept can relay a CFList from the GW to the stack, directly telling the stack which channels it supports. This is implemented in the stack but unfortunately the Multitech GW does not send anything here (!)

Assuming you are using the embedded network server on the Multitech GW. This feature is only supported by end-device stacks after RP002 was released. So by default it is not sent to all devices. If the device is identified as using LoRaWAN 1.0.3 or greater, using a device profile, the Multitech Network Server can send the optional CFList in the Join Accept.

[bwesen]

I think the 4.5.2 stack sends an identifier of 1.0.3 or greater, but the Multitech GW seems to ignore this because the CFList is indeed empty. It would be cool if it wasn't, that would also solve the problem. I haven't complained on the Multitech forums about this yet. Maybe there is a way to force or at least enable this behaviour on their side.

[mluis1]

Version 4.5.2 implements LoRaWAN 1.0.4 + Regional Parameters RP2-1.0.1 specifications.

[reissjason]

I think the 4.5.2 stack sends an identifier

The Join Request does not have a field to identify between 1.0.1 and 1.0.4 implementations. As this is the first and only message it would need to identify this in order to work as you are expecting.

Definitely check with Multitech regarding enabling this on the gateway side.

[bwesen]

Seems they don't have any options for this. I got a cryptic reply that you can force individual devices in the GW database to be marked as 1.0.3+, but this won't help for the initial UX as it would be after the join.

So I guess I have to do a hack in the stack to narrow the channels to the same sub-band the join was successfully done on, in the meantime.

[mluis1]

For debug purposes you may use below trick.
This trick restricts the end-device to use the first 8 channels + 64th channel.

If another bank of 8 channels is required please update channelMask variable initialization value.
For instance if one would like to use the second bank the channelMask variable would be initialized as follows:

uint16_t channelMask[] = { 0xFF00, 0x0000, 0x0000, 0x0000, 0x0002, 0x0000};

WARNING: By using this trick the LoRaWAN certification will fail as the end-device will not use all 64+8 channels

Patch to be applied to LmHandler.c file.

diff --git a/src/apps/LoRaMac/common/LmHandler/LmHandler.c b/src/apps/LoRaMac/common/LmHandler/LmHandler.c
index 3ebdcd54..ead77e34 100644
--- a/src/apps/LoRaMac/common/LmHandler/LmHandler.c
+++ b/src/apps/LoRaMac/common/LmHandler/LmHandler.c
@@ -313,6 +313,23 @@ LmHandlerErrorStatus_t LmHandlerInit( LmHandlerCallbacks_t *handlerCallbacks,
 
     LoRaMacTestSetDutyCycleOn( LmHandlerParams->DutyCycleEnabled );
 
+#if 1
+// BEGIN - AU915/US915 8 channels usage restriction.
+//         To be used for debug purposes only.
+//         **Makes LoRaWAN certification impossible**
+#if defined( REGION_AU915 ) || defined( REGION_US915 )
+                // Enabling 1st block of 8 channels (0-7) + channel 64
+                uint16_t channelMask[] = { 0x00FF, 0x0000, 0x0000, 0x0000, 0x0001, 0x0000};
+                mibReq.Type = MIB_CHANNELS_MASK;
+                mibReq.Param.ChannelsMask = channelMask;
+                LoRaMacMibSetRequestConfirm( &mibReq );
+                mibReq.Type = MIB_CHANNELS_DEFAULT_MASK;
+                mibReq.Param.ChannelsDefaultMask = channelMask;
+                LoRaMacMibSetRequestConfirm( &mibReq );
+#endif
+// END - AU915/US915 8 channels usage restriction.
+#endif
+
     LoRaMacStart( );
 
     mibReq.Type = MIB_NETWORK_ACTIVATION;

[bwesen]

Thanks, actually I had the exact same code to test that, must have copied it from some place..

I did this hack in RegionUS915.c now which fixes it as well for my purposes, so the allowed channels follow the "subband" the GW is using instead of hardcoding a specific set:

void RegionUS915ApplyCFList( ApplyCFListParams_t* applyCFList )
{
    // Size of the optional CF list must be 16 byte
    if( applyCFList->Size != 16 )
    {
	// Hack: the Multitech Conduit GW only listens to 8 of the 64 channels but doesn't send a CFList in
	// the JoinAccept reply to tell us which of them as it's not defaulting to LoRaWAN 1.0.3+. Therefore we can do
	// a workaround here and restrict the channel list to the same group of 8 channels that the JoinRequest succeeded on.
	const int subband = applyCFList->JoinChannel >> 3;
	//PRINTF("Empty CFList: restricting to subband %d\n", subband + 1); // Show as the Multitech subband, 1-8
	
	memset(RegionNvmGroup2->ChannelsMask, 0, sizeof(uint16_t) * 4);

	RegionNvmGroup2->ChannelsMask[subband >> 1] = (subband & 1) ? 0xff00 : 0x00ff; // Enable 8 125 kHz channels
	RegionNvmGroup2->ChannelsMask[4] = 1UL << subband;                             // Enable one 500 kHz channel
	
	for(int i = 0; i <= 4; i++)  // Apply the new mask to the current remaining hopping channels
	    RegionNvmGroup1->ChannelsMaskRemaining[i] &= RegionNvmGroup2->ChannelsMask[i];
	
        return;
    }

[mluis1]

The issue with your workaround will be the same as for my proposed trick.
You will not be able to certify your end-device.

The right place to implement an improvement that complies with certification process is to modify the RegionUS915NextChannel function.
I have not yet fully thought about it but, in order to make it work I think that we may need to add a variable that holds the previous uplink characteristics (IsJoined, IsPrevUplinkJoin, PrevUplinkSubbandId). Afterwards if IsJoined == true AND IsPrevUplinkJoin == true then select an available channel under PrevUplinkSubbandId.

[wolsty7]

Thanks, actually I had the exact same code to test that, must have copied it from some place..

I did this hack in RegionUS915.c now which fixes it as well for my purposes, so the allowed channels follow the "subband" the GW is using instead of hardcoding a specific set:

void RegionUS915ApplyCFList( ApplyCFListParams_t* applyCFList )
{
    // Size of the optional CF list must be 16 byte
    if( applyCFList->Size != 16 )
    {
	// Hack: the Multitech Conduit GW only listens to 8 of the 64 channels but doesn't send a CFList in
	// the JoinAccept reply to tell us which of them as it's not defaulting to LoRaWAN 1.0.3+. Therefore we can do
	// a workaround here and restrict the channel list to the same group of 8 channels that the JoinRequest succeeded on.
	const int subband = applyCFList->JoinChannel >> 3;
	//PRINTF("Empty CFList: restricting to subband %d\n", subband + 1); // Show as the Multitech subband, 1-8
	
	memset(RegionNvmGroup2->ChannelsMask, 0, sizeof(uint16_t) * 4);

	RegionNvmGroup2->ChannelsMask[subband >> 1] = (subband & 1) ? 0xff00 : 0x00ff; // Enable 8 125 kHz channels
	RegionNvmGroup2->ChannelsMask[4] = 1UL << subband;                             // Enable one 500 kHz channel
	
	for(int i = 0; i <= 4; i++)  // Apply the new mask to the current remaining hopping channels
	    RegionNvmGroup1->ChannelsMaskRemaining[i] &= RegionNvmGroup2->ChannelsMask[i];
	
        return;
    }

There is an issue with the "RegionUS915ApplyCFList" workaround... on every 9th join attempt it it attempt to join on a 500 kHz channel, leading to the incorrect sub-band being calculated.
Therefore for anyone else who needs this it should be changed to:

    if( applyCFList->Size != 16 )
    {
        // Hack: the Multitech Conduit GW only listens to 8 of the 64 channels but doesn't send a CFList in
        // the JoinAccept reply to tell us which of them as it's not defaulting to LoRaWAN 1.0.3+. Therefore we can do
        // a workaround here and restrict the channel list to the same group of 8 channels that the JoinRequest succeeded on.
        const int subband= applyCFList->JoinChannel<64 ? applyCFList->JoinChannel >> 3 : applyCFList->JoinChannel -64;
        //MW_LOG(TS_ON, VLEVEL_L, "JoinChannel: %d: restricting to subband %d\r\n", applyCFList->JoinChannel, subband +1);// Show as the Multitech subband, 1-8
        
        memset(RegionNvmGroup2->ChannelsMask, 0, sizeof(uint16_t) * 4);
        
        RegionNvmGroup2->ChannelsMask[subband >> 1] = (subband & 1) ? 0xff00 : 0x00ff; // Enable 8 125 kHz channels
        RegionNvmGroup2->ChannelsMask[4] = 1UL << subband;                             // Enable one 500 kHz channel
        
        for(int i = 0; i <= 4; i++)  // Apply the new mask to the current remaining hopping channels
            RegionNvmGroup1->ChannelsMaskRemaining[i] &= RegionNvmGroup2->ChannelsMask[i];
        return;
    }