User experience for in flight records

[mdial89f]

Background

Our system accepts submissions, and synchronously transmits data to seatool and kafka. Upon confirmation that those have been successfully transmitted, our application returns the user to the dashboard or details page, and shows a success banner. We do not wait for data to arrive back from kafka before returning the user.

This leaves a typical 3-5 seconds where the record displayed to the user is not reflective of the actual seatool info. Aside from the modal which (at least previously, i think language changed) says "it may take a minute or more for your change to be refected", there's no notice to the user that the record is in a transient state.

This dicussion is around how we can best mitigate that. I will list out an initial approach, and we can discuss and evolve from there. When we've got a decision, it will be pasted in a separate decision section.

Proposed Approach

1. Initial Update Status: When an action is initiated, immediately set an "updating" indicator to the user. This indicator should be visible before the api is called, and remain visible until the api returns. The purpose is to visually indicate to users that an action is currently being submitted. (exists... loading modal/spinner).

2. API Action flow: After a user submits an action, several things happen:

   • An API call initiates the action process. (exists)
   • A lambda function processes the update logic. (exists)
   • The system writes updates to SeaTool. (exists)
   • A record is pushed to Kafka (maintaining current behavior). (exists)
   • An upsert operation updates the record in OpenSearch with seatoolStatus, stateStatus, and cmsStatus all set to "updating". This ensures immediate reflection of the update status in OpenSearch queries, eliminating any lag in status representation. (NEW)

3. User Interface Feedback:

   • Upon lambda completion, display a banner or visual cue to inform the user of success/failure (exists... banners/toasts)
   • Identify all records that are updating by checking seatoolStatus. For those that have the updating status, show them differently to the user. On the details page, perhaps display the updating status and gray out the record to make it clear. For the dashboard, perhaps display Updating... for the status and temporarily blanking out other non-essential details to emphasize the update process. (NEW)

4. Completion of Update:

   • SeaTool processes the update and streams the data to kafka. When a kafka record appears that has a non null status, that data is indexed to opensearch as usual, thereby removing the updating status values. (exists)

5. Ensuring Timely User Notification:

   • Implement a mechanism (such as polling, websockets, or notifications) to inform the user interface that the update has completed, allowing for automatic refresh without requiring user action. (NEW)
   • For efficient monitoring, store the ID of the updated package in the browser context. Utilize a utility script to check the status of this item at regular intervals (e.g., every second) until the "Updating" status is cleared, at which point, trigger a UI refresh. (NEW)

6. Advanced Real-time Updates (Optional):

   • Explore the use of websockets for real-time update notifications. This approach can provide immediate feedback to the user once the record's update process is complete, enhancing user experience by reducing the need for manual refreshes or polling mechanisms.

By implementing these steps, users receive immediate feedback when records are being updated and are assured of a seamless experience as they interact with the system. The use of websockets, while optional, can further enhance this process by providing real-time updates to the user interface.

[13bfrancis]

A link to a PR that would be a potential solution to point 5 above
#400

[13bfrancis]

A note on this. I think this is a valid first pass. I think eventually a more real time system that uses sockets would make more sense

[pkim-gswell]

@mdial89f
This solution is just what the doctor ordered