how to use Audio API in MV plugin?

Yitzi Litt

Made Nepenthe — An RPG that's one giant dad-joke
Veteran
Joined
Jul 31, 2017
Messages
103
Reaction score
29
First Language
english
Primarily Uses
RMMV
How would I access the Web Audio API in an RPGMMV plugin? Specifically, I want to have a given Boolean variable turn on whenever the audio level in a BGM track reaches a given point, allowing me to then have onscreen actions "react" to the music. This should be possible, but I haven't seen anyone call the Web Audio API in a plugin before. As I'm not all that great at coding, I could use a pre-made template/example to work from.
Thanks!
 

Yitzi Litt

Made Nepenthe — An RPG that's one giant dad-joke
Veteran
Joined
Jul 31, 2017
Messages
103
Reaction score
29
First Language
english
Primarily Uses
RMMV
bump (nothing new to add, but haven't heard any answer yet, so yeah)
 

Yitzi Litt

Made Nepenthe — An RPG that's one giant dad-joke
Veteran
Joined
Jul 31, 2017
Messages
103
Reaction score
29
First Language
english
Primarily Uses
RMMV
...and another bump. Would this plugin be particularly hard to create, or is it just a boring idea?
 

Poryg

Dark Lord of the Castle of Javascreeps
Veteran
Joined
Mar 23, 2017
Messages
4,113
Reaction score
10,545
First Language
Czech
Primarily Uses
RMMV
Maybe you could just do with a parallel process event.

if script: WebAudio._context.currentTime == 3.023 turn on a variable or switch. Nevertheless, due to the fact that 1. MV runs on 60 Fps, therefore checking once every 1/60th of a second, and 2. MV loads files on demand, it makes it rather hard to pinpoint the exact locations of a song.
 

Fornoreason1000

Black Sheep
Veteran
Joined
Mar 1, 2014
Messages
205
Reaction score
94
First Language
English
Primarily Uses
RMMV
You can Actually Do this , infact RPGmaker MV does this to a certain extent,

Basically in most of RPGmakers standard asset, inside the .ogg Metafiles there two parameters called loop length and loop start. these parameters are used to try and make the song loop from a given point.

Unfortunately getting this information is quite cryptic!!!
it does it differently for .ogg and the crappy .m4a formats to make it worse. heres the source code for MV does this

Code:
WebAudio.prototype._readLoopComments = function(array) {
    this._readOgg(array);
    this._readMp4(array);
};

/**
 * @method _readOgg
 * @param {Uint8Array} array
 * @private
 */
WebAudio.prototype._readOgg = function(array) {
    var index = 0;
    while (index < array.length) {
        if (this._readFourCharacters(array, index) === 'OggS') {
            index += 26;
            var vorbisHeaderFound = false;
            var numSegments = array[index++];
            var segments = [];
            for (var i = 0; i < numSegments; i++) {
                segments.push(array[index++]);
            }
            for (i = 0; i < numSegments; i++) {
                if (this._readFourCharacters(array, index + 1) === 'vorb') {
                    var headerType = array[index];
                    if (headerType === 1) {
                        this._sampleRate = this._readLittleEndian(array, index + 12);
                    } else if (headerType === 3) {
                        this._readMetaData(array, index, segments[i]);
                    }
                    vorbisHeaderFound = true;
                }
                index += segments[i];
            }
            if (!vorbisHeaderFound) {
                break;
            }
        } else {
            break;
        }
    }
};

/**
 * @method _readMp4
 * @param {Uint8Array} array
 * @private
 */
WebAudio.prototype._readMp4 = function(array) {
    if (this._readFourCharacters(array, 4) === 'ftyp') {
        var index = 0;
        while (index < array.length) {
            var size = this._readBigEndian(array, index);
            var name = this._readFourCharacters(array, index + 4);
            if (name === 'moov') {
                index += 8;
            } else {
                if (name === 'mvhd') {
                    this._sampleRate = this._readBigEndian(array, index + 20);
                }
                if (name === 'udta' || name === 'meta') {
                    this._readMetaData(array, index, size);
                }
                index += size;
                if (size <= 1) {
                    break;
                }
            }
        }
    }
};

/**
 * @method _readMetaData
 * @param {Uint8Array} array
 * @param {Number} index
 * @param {Number} size
 * @private
 */
WebAudio.prototype._readMetaData = function(array, index, size) {
    for (var i = index; i < index + size - 10; i++) {
        if (this._readFourCharacters(array, i) === 'LOOP') {
            var text = '';
            while (array[i] > 0) {
                text += String.fromCharCode(array[i++]);
            }
            if (text.match(/LOOPSTART=([0-9]+)/)) {
                this._loopStart = parseInt(RegExp.$1);
            }
            if (text.match(/LOOPLENGTH=([0-9]+)/)) {
                this._loopLength = parseInt(RegExp.$1);
            }
            if (text == 'LOOPSTART' || text == 'LOOPLENGTH') {
                var text2 = '';
                i += 16;
                while (array[i] > 0) {
                    text2 += String.fromCharCode(array[i++]);
                }
                if (text == 'LOOPSTART') {
                    this._loopStart = parseInt(text2);
                } else {
                    this._loopLength = parseInt(text2);
                }
            }
        }
    }
};

/**
 * @method _readLittleEndian
 * @param {Uint8Array} array
 * @param {Number} index
 * @private
 */
WebAudio.prototype._readLittleEndian = function(array, index) {
    return (array[index + 3] * 0x1000000 + array[index + 2] * 0x10000 +
            array[index + 1] * 0x100 + array[index + 0]);
};

/**
 * @method _readBigEndian
 * @param {Uint8Array} array
 * @param {Number} index
 * @private
 */
WebAudio.prototype._readBigEndian = function(array, index) {
    return (array[index + 0] * 0x1000000 + array[index + 1] * 0x10000 +
            array[index + 2] * 0x100 + array[index + 3]);
};

/**
 * @method _readFourCharacters
 * @param {Uint8Array} array
 * @param {Number} index
 * @private
 */
WebAudio.prototype._readFourCharacters = function(array, index) {
    var string = '';
    for (var i = 0; i < 4; i++) {
        string += String.fromCharCode(array[index + i]);
    }
    return string;
};

As you can see its kinda crazy and somewhat complex, but i should be able to help with this. As im still working on a Audio Plugin myself (DSEAudioEngine).
Basically it parse some meta (via readLoopComments() ) data right at the moment it loads before the ultra slow decoding begins.

now don't do what @Poryyg said (usually he's spot on BTW), since WebAudio._conext.currentTime does not represent the song at All!!! that starts counting the moment WebAudio is initialized and continues to count upwards indefinably until MV closes or WebAudio is reinitializes. this means setting an event condition on it directly WILL produce a buggy mess.

the MV devs Knew this and already used a work around for this, unfortunately that work around is embedded withe the API itself.

basically you will need some AudioEditing software such as Audacity, open the song who want that event in. then go to edit > metadata and you should see a list with the Author, track number etc. if you open a MV song you should also see a LOOPSTART and a LOOPLENGTH. that my good freind is where you add your Event time in Milliseconds and name it whatever you want.
 
Last edited:

Yitzi Litt

Made Nepenthe — An RPG that's one giant dad-joke
Veteran
Joined
Jul 31, 2017
Messages
103
Reaction score
29
First Language
english
Primarily Uses
RMMV
basically you will need some AudioEditing software such as Audacity, open the song who want that event in. then go to edit > metadata and you should see a list with the Author, track number etc. if you open a MV song you should also see a LOOPSTART and a LOOPLENGTH. that my good freind is where you add your Event time in Milliseconds and name it whatever you want.
So I go to an audio editor, add a metadata tag for when I want an event to happen — and then what do I do to get it to activate in game?
 

Fornoreason1000

Black Sheep
Veteran
Joined
Mar 1, 2014
Messages
205
Reaction score
94
First Language
English
Primarily Uses
RMMV
So I go to an audio editor, add a metadata tag for when I want an event to happen — and then what do I do to get it to activate in game?
No, I need to give you a plugin that will make that happen. Vanilla MV will only read those two tags so putting something in at this point wont make it work.

but i though about it and it may be easier to set that up in MV, because im assuming your going to want more than just one event right?
 

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

Latest Threads

Latest Posts

Latest Profile Posts

Husband has roped me into playing one of his PSO2 characters. Productivity is now negative.
Tileset A is lava! Don't burn D:
as MZ come out what you guys think about.....Some particles engine :3c?
Who wants to sing this song with me?~

Decided to submit my games to the Monday Night Stream. Figured it might be a fun way to get some traction.

Forum statistics

Threads
100,704
Messages
978,552
Members
132,320
Latest member
LavaWave
Top