Bug Audio does not pause when game loses focus

[caethyril]

Seen in RMMZ core scripts up to the current version (v1.4.0).

Edit 2021-12-09: as of v1.4.0, Effekseer pauses when the game is out of focus, but audio does not. The thread title and content have been updated to reflect this.

(This thread was previously titled "Audio + Effekseer do not pause when game loses focus".)

---

Description​

The current scene updates only if the game is in focus. Audio does not pause/resume in the same way. This discrepancy can cause in-game desynchronisation, e.g.

1. A cutscene timed to its BGM.
   • v1.4.0 - no change.
2. Audio cues for Effekseer animations.
   • v1.4.0 - partial fix: new sound cues will no longer trigger when paused.
     Audio already playing will continue to play as it would if the game were not paused.

Steps to reproduce​

1. Start the game.
2. Play any audible track in-game (BGS, BGM, ME, SE), e.g. via:
   • The Autoplay BGM and/or Autoplay BGS map settings;
   • A Play BGM, Play BGS, Play ME, or Play SE event command;
   • A corresponding script command via event or console; or
   • An animation with at least one audio cue.
3. Make the game lose focus while the audio is playing, e.g. by clicking on the editor or console.
4. The audio will continue to play while the scene is "paused".

Technical explanation​

The game's underlying audio updates independently of the SceneManager. This loop continues regardless of whether or not SceneManager.isGameActive() is true. More information on the Web Audio context object can be found here:

AudioContext - Web APIs | MDN

The AudioContext interface represents an audio-processing graph built from audio modules linked together, each represented by an AudioNode.

developer.mozilla.org

Potential fix​

Apply the relevant pause/resume methods when the game focus changes, e.g.

Spoiler: v1.4.0 - audio-only plugin code

/*:
 * @target MZ
 * @help Free to use and/or modify for any project.
 */

(function(alias) {
'use strict';

    let isActive = true;

    const getAudioContext = function() {
        return WebAudio._context;
    };

    const pauseAudio = function(context, active) {
        if (active) context.resume();
        else        context.suspend();
    };

    const updateActive = function() {
        pauseAudio(getAudioContext(), isActive);
    };

    SceneManager.updateScene = function() {
        const active = this.isGameActive();
        if (isActive !== active) {
            isActive = active;
            updateActive();
        }
        alias.apply(this, arguments);
    };

})(SceneManager.updateScene);

Spoiler: audio + Effekseer plugin code

/*:
 * @target MZ
 * @help Free to use and/or modify for any project.
 */

(function(alias) {
'use strict';

    let isActive = true;

    const getAudioContext = function() {
        return WebAudio._context;
    };

    const getEffekseerHandles = function() {
        return (SceneManager._scene._spriteset?._animationSprites || [])
                .map(s => s._handle)
                .filter(h => h);
    };

    const pauseAudio = function(context, active) {
        if (active) context.resume();
        else        context.suspend();
    };

    const pauseEffekseer = function(handles, active) {
        handles.forEach(h => h.setPaused(!active));
    };

    const updateActive = function() {
        pauseAudio(getAudioContext(), isActive);
        pauseEffekseer(getEffekseerHandles(), isActive);
    };

    SceneManager.updateScene = function() {
        const active = this.isGameActive();
        if (isActive !== active) {
            isActive = active;
            updateActive();
        }
        alias.apply(this, arguments);
    };

})(SceneManager.updateScene);

Edit 2021-12-18: this approach to pause the audio only seems to work correctly for BGM, BGS, and ME buffers; SE buffers (un)mute correctly, but it seems they update their seek/play position while muted. I'm not sure why, though.

Edit 2022-03-21: sorry for the wait, here's a version that seems to work OK for all audio. It swaps the setTimeout for an onended handler, fixing the issue seen in my previous update. It will apply the Effekseer pause effect as well for core script versions older than 1.4.0.

• view/download CaeF_PauseFix.js (Google Drive)

Spoiler: Full plugin code

/*:
 * @target MZ
 * @plugindesc Pause/resume audio (and Effekseer) with the game loop.
 * @author Caethyril
 * @url https://forums.rpgmakerweb.com/index.php?threads/141642/
 * @help Free to use and/or modify for any project. No credit required.
 */

(() => {
'use strict';

    let isActive = true;

    const SHOULD_LOOP = Symbol();

    const getEffekseerHandles = function() {
        return (SceneManager._scene._spriteset?._animationSprites || [])
                .map(s => s._handle)
                .filter(h => h);
    };

    const pauseEffekseer = function(handles, active) {
        handles.forEach(h => h.setPaused(!active));
    };

    const getAudioContext = function() {
        return WebAudio._context;
    };

    const pauseAudio = function(context, active) {
        if (active) context.resume();
        else        context.suspend();
    };

    const updateActive = function() {
        if (true)
            // 1.4.4 core script does not pause audio
            pauseAudio(getAudioContext(), isActive);
        if (!Utils.checkRMVersion("1.4.0"))
            // apply for pre-1.4.0 core script only
            pauseEffekseer(getEffekseerHandles(), isActive);
    };

    const onAudioEnd = function(index) {
        if (this[SHOULD_LOOP]) {
            // This is the original onended event handler, for encrypted audio loops.
            this._createSourceNode(index);
            this._startSourceNode(index);
        } else {
            // This replaces the original setTimeout method
            // so that end timers trigger when they are supposed to.
            const endTime = this._startTime + this._totalTime / this._pitch;
            if (WebAudio._currentTime() >= endTime)
                this.stop();
        }
    };

    // Also update the audio's "active" status.
    const alias = SceneManager.updateScene;
    SceneManager.updateScene = function() {
        const active = this.isGameActive();
        if (isActive !== active) {
            isActive = active;
            updateActive();
        }
        alias.apply(this, arguments);
    };

    // Assign "onended" event handler to onAudioEnd (see above).
    // A new flag (SHOULD_LOOP symbol) is set for looped encrypted audio.
    // This flag determines the behaviour of onAudioEnd.
    WebAudio.prototype._startSourceNode = function(index) {
        delete this[SHOULD_LOOP];                                   // <- edit 1 of 3
        const sourceNode = this._sourceNodes[index];
        const seekPos = this.seek();
        const currentTime = WebAudio._currentTime();
        const loop = this._loop;
        const loopStart = this._loopStartTime;
        const loopLength = this._loopLengthTime;
        const loopEnd = loopStart + loopLength;
        const pitch = this._pitch;
        let chunkStart = 0;
        for (let i = 0; i < index; i++) {
            chunkStart += this._buffers[i].duration;
        }
        const chunkEnd = chunkStart + sourceNode.buffer.duration;
        let when = 0;
        let offset = 0;
        let duration = sourceNode.buffer.duration;
        if (seekPos >= chunkStart && seekPos < chunkEnd - 0.01) {
            when = currentTime;
            offset = seekPos - chunkStart;
        } else {
            when = currentTime + (chunkStart - seekPos) / pitch;
            offset = 0;
            if (loop) {
                if (when < currentTime - 0.01) {
                    when += loopLength / pitch;
                }
                if (seekPos >= loopStart && chunkStart < loopStart) {
                    when += (loopStart - chunkStart) / pitch;
                    offset = loopStart - chunkStart;
                }
            }
        }
        if (loop && loopEnd < chunkEnd) {
            duration = loopEnd - chunkStart - offset;
        }
        if (this._shouldUseDecoder()) {
            if (when >= currentTime && offset < duration) {
                sourceNode.loop = false;
                sourceNode.start(when, offset, duration);
                if (loop && chunkEnd > loopStart) {
                    this[SHOULD_LOOP] = true;                       // <- edit 2 of 3
                    sourceNode.onended = onAudioEnd.bind(this);     // <- edit 3 of 3
                }
            }
        } else {
            if (when >= currentTime && offset < sourceNode.buffer.duration) {
                sourceNode.start(when, offset);
            }
        }
        chunkStart += sourceNode.buffer.duration;
    };
    
    // Rewritten: end timer now fires via "onended" event.
    // This allows it to respect AudioContext timing changes, e.g. pause/resume.
    WebAudio.prototype._createEndTimer = function() {
        if (!this._loop)
            // (Note to self: multiple source nodes are used for encrypted audio.)
            this._sourceNodes.forEach((source, n) => {
                source.onended = onAudioEnd.bind(this, n);
            }, this);
        // original:
        // if (this._sourceNodes.length > 0 && !this._loop) {
        //     const endTime = this._startTime + this._totalTime / this._pitch;
        //     const delay = endTime - WebAudio._currentTime();
        //     this._endTimer = setTimeout(this.stop.bind(this), delay * 1000);
        // }
    };

})();

 

[Archeia]

Thank you! We're currently investigating!

 

[caethyril]

Updated for 1.4.0 (great update by the way!). Effekseer animations now pause when the game loses focus, but audio does not. I've edited the post/thread to reflect this and made a few other clarifications.

I'm guessing the Effekseer part of this report eclipsed the audio part. However, if non-pausing audio is actually not considered a bug then I'd appreciate feedback on that. No rush, though~

[Edit: I just realised that SE channels specifically seem to keep "playing" while suspended, so I'm guessing that's the reason nothing was done with the audio yet. ]

Cheers!

 

[caethyril]

Updated with (what I think is) a plugin that pauses all audio (and, if appropriate, Effekseer) correctly when the game is out of focus, tested for v1.4.4. I'm releasing it as one of my plugins, but as noted in the plugin's help section, I'm totally OK with anyone else using the code, with or without credit.