-
Notifications
You must be signed in to change notification settings - Fork 6
audio
Reference: HTML5 audio element
pros:
- buffers automatically and does a reasonable job
- can easily set start position
- built-in playback rate control
cons:
- some glitching when setting currentTime (used to point the player at the start of a clip); however, this can be worked around fairly easily.
- have to use timeout/setinterval to monitor for end of clip
If we only reset the currentTime when it is out of clipBegin/clipEnd boundaries, then glitching seems to go away.
If we encounter issues with loading files, we can create an asset manager to pre-load audio assets. In this case, it would be helpful to list all the audio assets related to SMIL when the book loads; of course, this can't be done without analyzing all the SMIL files, which could get expensive. One compromise might be to have the MO component, which deals with one file at a time, pre-load audio assets of a single file.
Reference: web audio
pros:
- no glitching
- can specify clip duration up front
- can accept audio filters, for example to scale the playback rate without affecting the pitch
cons:
- have to use timeout/setinterval to monitor the status of an audio clip
- have to manage buffer manually, and it can be slow to load
future pros:
- need for timeout/setinterval monitoring will go away in the future (see this bug)
If we were to use web audio, we would have to progressively buffer windows of data (for example, 1MB at a time). Until it supports clip-done callbacks, we might not need the full power of web audio.
There is an issue that if the tab moves to the background, audio playback is affected because the interval is checked only once every second (read more). There are two methods to work around this.
The calling application knows that it will play several clips. It checks the next clip to see if it represents continuous audio file playback, wrt the current clip. If so, then the audio player is told to "play through", i.e. not stop at the end of the current clip. If playThrough is false, however, the audio player pauses and waits for its next instruction. This means that if the next clip is not contiguous, then the worst case is that the audio player overplays the file for a very short amount of time (max 1 second because when the tab loses focus, setInterval granularity degrades to once per second).
There is a case where some glitching could happen. If the tab loses focus and the audio player has overplayed its clip beyond the bounds of the new clip (for example, if the new clip is very short), then it will reset its currentTime to match the new clip's start time. In practice, this theoretical behavior has yet to be reproduced.
Another edge case is if the user preference for skippability changes after one clip has started playing, which affects what the next clip will be. In this case, the audio player's playThrough setting might ideally have been different. However, this is a fairly remote scenario and would be corrected by the time the subsequent clip was played.
This type of check involves back-calculating the current SMIL node based on audio player position. It could potentially be an expensive search operation, so its use should be minimized; for example, it could be used only when the tab comes back into focus. It may not be necessary if the playThrough option above is in use. Experiments are in progress...