Provide API for handling multiple publishers for the same trackable id

[kavalerov]

There may be situations in which it is reasonable to expect multiple Publishers to be present at the same time for a single Trackable id. An example of this is a situation in which delivery driver has two devices in the case one is low on battery to ensure that new device can be used immediately.

This, however, leads to potential challenges on the subscriber side. As location data cleaning and enhancement done on the publisher device, having more than one publisher may lead to location data on the channel not being continuous and may result in slight "jumps" of the trackable when visualised on a map.

We should think about ways of solving this. One of possible approaches can be providing an API that allows subscriber to choose the publisher out of the list. This approach would suggest that application developer has other means of figuring out which publisher is "correct" at any given time.

Other approach can be making stronger assumptions about the use case and implementing the logic of choosing the publisher in the SDK itself. For example, logic can be "stick to the publisher you have recieved messages from previously unless it turned quite for X time" or similar. The downside of this approach is that the assumption we would be making is pretty strong here, and this reduces flexibility.

[ben-xD]

Is this a popular request? I didn't expect any delivery drivers to have multiple devices (they would charge their phone 😅).

But sure, here is an idea,

• Since we're using 1 channel per trackable, each publisher should join the presence of the channel.
• Only publish if you're present on the channel.
• Only join presence if no-one else is present (there might be race conditions here, when 2 publishers join simultaneously). Simultaneous publishers can be configured by the SDK by limiting the total presence count

[kavalerov]

This is a direct request from our pilot customer, yes. I didn't expect this to be a popular case - but actually it turnes out to be an expected situation that we should be able to handle correctly.

Only join presence if no-one else is present (there might be race conditions here, when 2 publishers join simultaneously). Simultaneous publishers can be configured by the SDK by limiting the total presence count

This is a valid option, and actually one of the options that I though about originally. However, the problem is that you still may have race conditions in this situation. Thats why the approach that we though could be better is to shift the responsibility for this to the subscriber side.

[ben-xD]

Subscriber logic:

• Can you explain how subscribers will select the right device? Presumably device selection is done by the driver (asset/ publisher side), so I'm not sure how clients can decide.
• Filtering client side will cost twice as much bandwidth (for Ably/ customers, but also end users) for 2 publishers. (And N times for N publishers)

Publisher logic: single-occupancy presence channels

Seems to me, a good solution would be to add a presence limit feature to Ably channels (if not already available) to avoid race condition in the long term. This feature can also limit the size of chat rooms etc (so maybe its already available).

Alternative Publisher logic: generous publishers (so far this is my favorite)

Publishers can stop publishing as soon as another publisher successfully enters presence. They give up their spot publishing by leaving presence. New publishers always replacing old ones is also a nice behaviour to have (instead of having to coordinate turning off the first publisher to allow the second publisher to join presence/ publish). We don't have to have presence limit in this case. This will prevent the rare case of race condition (the user has to start publishing from both apps simultaneously). To publish, publishers must be present (or clients should only listen to publishers who are present, whichever is simpler to implement).

[kavalerov]

Can you explain how subscribers will select the right device? Presumably device selection is done by the driver (asset/ publisher side), so I'm not sure how clients can decide.

This may be handled by customers systems. For example, they may keep track of "active" or "correct" publishers, and then they manage how Subscriber SDK becomes aware of this and chooses the right publisher.

Filtering client side will cost twice as much bandwidth (for Ably/ customers, but also end users) for 2 publishers. (And N times for N publishers)

True, the only (and main) upside to doing things on the subcriber side is simplicity - it requires least number of assumptions and custom logic, therefore should fit most use-cases. And this happens at the cost of higher bandwidth usage and adding this responsibility to customer, which I agree is not ideal.

Seems to me, a good solution would be to add a presence limit feature to Ably channels (if not already available) to avoid race condition in the long term.

I am not sure how exactly this would work (avoiding the race condition), do you mind elaborating a bit? If you think that this is a feature that we should add to Ably as a platform, it actually may be a good idea to write it up here: https://github.com/ably/ideas/issues/new?assignees=&labels=&template=product-idea.md&title=

Publishers can stop publishing as soon as another publisher successfully enters presence

I really, really like this idea. I need to think more about potential downsides and corner cases - but sounds like an excellent option!

[ben-xD]

I renamed the title, but I that begs the question of the desired outcome, it might not be clear to others.

To clarify:
I believe, the desired outcome is to handle transitions between devices for the same trackable ID gracefully, as opposed to allowing multiple devices to publish in parallel and allowing the user to select between multiple tracking devices. If a customer wants to track 2 devices simultaneously, they should have 2 tracking IDs.

WDYT @kavalerov?

[kavalerov]

I would word it slightly more generically:

One asset/trackable may be tracked by multiple devices - and we should accomodate this usecase.

[ben-xD]

One asset/trackable may be tracked by multiple devices - and we should accomodate this usecase.

That's what I'm avoiding by clarifying. That can mean multiple devices publishing simultaneously, which I am trying to avoid.

Or are you suggesting multiple devices should be able to track 1 asset simultaneously? e.g. the vehicle has a tracker, and the driver has 2 phones, so we track 3 devices simultaneously, sending all data to the subscribers. But that would mean subscribers need logic to choose (or in the UI), the benefit is a customer can track all 3 devices (for whatever reason they want). If devices have different locations, I would class them as different trackables, because this is an extra layer of complexity that is easily fixed by just using 2 trackables.

[paddybyers]

It's a valid use-case that a single courier will have multiple devices publishing locations for a single delivery, either overlapping only transiently, or for longer periods. We have to provide a natural way to support that use-case.

[kavalerov]

Exactly what Paddy wrote - it is a valid situation in which there are multiple devices publishing locations for the same asset, even simultaneously, so we shouldn't avoid it.

[ben-xD]

Okay, but we should acknowledge this is changing the definition of a trackable, currently:

asset (trackable): Item that the customer wants to track such as driver, vehicle, container, and so on. This also known as a Trackable in the SDKs. source

I think both @paddybyers and @kavalerov are suggesting this gets changed to: A group of items a customer wants to track related to an asset, including a driver, vehicle, container, the asset itself and so on.

So an asset/ trackable is not an item, but a collection of items.

We have to provide a natural way to support that use-case.

We already provide a good primitive: asset. And this use-case is handled with 2 assets. I initially believed we're solving a problem (the reported glitchy UI). What benefit does this have over asking customers to create 2 assets to track 2 separate devices (this can be added to example apps)? The only disadvantage I can think of is customers must manage these "collections" themselves, but they've got to manage other metadata anyway. They also might want to manage it themselves: to have different icons/ data for each publisher.

Here are some drawbacks of adding this layer (please mention more benefits as I struggled):

• Users can already implement this feature using multiple trackables, and therefore no implementation needed. This is more work
• We're adding another way to do the same thing (API surface complexity), which will confuse users who don't need it (most people),
• There will undeniably be limitations. e.g. Isn't resolution asset-specific and not publisher specific? More work will be needed to make resolution publisher-specific. This is a sensible future request because different devices will have different power/ network capabilities. This feature is already baked into existing primitive: trackable.
• it prevents graceful handover (efficient usage of Ably/ bandwidth) between devices without adding more complexity. This feature can easily be implemented by making SDK publishers give up their presence when another one enters the channel. Now we have to manually stop publishers publishing.

Maybe we should ask the customer what they prefer?

[kavalerov]

So, I will respond to the second half of the message on Monday - but in general all the points you are highlighted are great.

However, I don't agree that the proposed improvements changes our current definition of trackable.

asset (trackable): Item that the customer wants to track such as driver, vehicle, container, and so on. This also known as a Trackable in the SDKs.

The definition in its current form that you quoted doesn't talk about devices at all, it talks about the things that customers may want to track, and this is not changing. It is not saying that asset is one device, it actually doesn't establish any relationship between device, or publisher, or trackable. What we are saying is that in publisher SDK you can set the trackable ID for which you will be broadcasting locations - but we are not saying that the device is the asset.

So, in this case, asset is a trackable. And there can be multiple publishers providing locations for this trackable.

[kavalerov]

Sorry, I haven't had a chance to get back on this today - lets pick up the discussion later in a week when we get back to this ticket.

[rdeoliv]

We had a conversation with another prospective customer whose end users would publish the location of the buses they are on. The goal is to be able to track a maximum of buses and provide better visibility into the transportation network.

This issue rose when multiple publishers are part of the same transport. I do not believe we should have more than 1 publisher per trackable. I cannot find any benefits and I don't believe it's still a requirement from our pilot customer.

I don't believe it's sustainable to have 2 (or more) publishers for the same asset as it would just be waste of resources. But should this happen, the customer ultimately should be the orchestrator and decide which publisher should continue to publish and which ones should stop based on their own factors (battery level, accuracy of the GPS data, time messages has been published...). This could be done by signalling the various publisher outside of the trackable channel.