Testing Winden with Mobile Device browsers

[donpui]

Ticket for discussion and finding possible option how to improve testability of application.

Currently CI automatic E2E tests runs on Chrome, Firefox, Edge browsers (VM) in desktop mode.
However, mobile version testing is left for manual testing.
Currently we have from 10-20% of Users who check Winden on mobile devices.

It would be good to run E2E tests on real/simulated mobile devices:

• iOS: Safari, Chrome, Firefox (currently doesn't work fully)
• Android: Chrome, Firefox

Few possible options:

• Use cloud services like: browserstack, lambdatest, other..
• Build on iOS/Android emulator
• other?

Note: As we have seen some behaviour differences on Android/iOS devices and mobile browsers, it is important to try test on real devices, instead of emulating layout or user agent.

[meejah]

Can you quantify what differences we have seen with devices?

Personally, I'd really look to avoid a(nother) proprietary service. What are the tests running on currently?

[donpui]

Browsers on Desktop and on Mobile devices are not always the same, even if they built on the same core libraries and support some common things. Also there are plaftorms specific, which impacts.

Good example: iOS all browsers: Safari, Chrome, Firefox are basically build on webkit, not on native chromium or gecko. In reality they are all more or less Safari browsers with some customisation on iOS.
Some known open issues, differences detected from previous manual testing:

• Firefox, cannot receive files on iOS, while Chrome can
• Sending is terminated when iOS device turns off screen
  With Android
• Android 200mb transfer fails
• Google Drive sending doesn't work from Android
• Samsung browser, file is downloaded but not saved in disk

Much more we have already solved.

Overall the goal here is to avoid manual testing or skipping testing at all.

Main scenarios, which we previously were covering: iOS Safari, iOS Chrome, iOS Firefox, Android Chrome, Android Firefox, Samsung Browser.