MUSH Audio Engine - Spatial Audio/Voice Acting/New Channels/etc.

[MushroomCake28]

MUSH Audio Engine [v.1.07]
by MushroomCake28​

Download link: https://kamo-studio.itch.io/rpg-maker-mz-plugin-mush-audio-engine

Introduction

This is my core plugin that revamps the existing audio in RPG Maker MZ. It is the first in a series of plugins that will aim to improve the audio in RPG Maker MZ and add new features. This core plugin not only sets the base architecture for future plugins, it also introduces a lot of cool new features!

Features

Here is a list of the features introduced by the MUSH Audio Engine, but there will be one section dedicated per feature below for more details and information:

• New BGM/BGS system that allows multiple BGMs/BGSs to be played at the same time. ​
• Added 2 optional extra channels to control volume separately: one channel for UI sound effects called UIS, and one channel for voice acting called VSC.​
• Added a master volume option.​
• Spatial Audio for BGM, BGS, and SE (volume and pan dynamically varies depending on player position to audio source(s)).​
• Pitch and Volume variance for SE/UIS on each play. ​
• Ability to load the parent map's autoplay BGM if the current map doesn't have one.​

I will add more features with time as I keep updating this plugin. To have more info on each particular feature and understand how to use them and the plugin commands attached to them, please see the corresponding section below.



UIS and VSC Channels

Spoiler

This plugins allows you to add 2 additional audio channels separate from the default BGM/BGS/ME/SE configuration.

UIS (UI Sound Effect): This is a channel reserved for sound effects of the UI. Basically it's a separate channel for the static sound effects. This will be used for all the Sound Effects in the System 1 tab in the database.

VSC (Voice Sound Channel): This is a channel reversed for voice acting audio. It's a completely new channel.

This will allow you to control their volume separately from the other channels. For instance, sound effects from UI (cursor, confirm, cancel) can still have a volume while other regular sound effects may be muted.

You can turn those two features off easily in the plugin's parameters in the General Features section. If the UIS feature is off, UI Sound Effects will use the SE channel like they originally do. The plugin command for VSC will use the SE instead if the VSC feature is turned off.

You can also turn ON/OFF the option to add those 2 channels in the Options Menu in the parameters in the "Menu Options Changes" section.

New BGM and BGS System

Spoiler

This plugin introduces a new BGM and BGS system. However, the default BGM and BGS system (which only allows 1 BGM and 1 BGS to be played at the same time) hasn't been removed to preserve compatibility with other plugins. The new BGM/BGS system is just an addition system on top of the current one.

This system allows you play multiple BGMs at the same time and multiple BGSs at the same time while being able to control each one of them individually. All the BGMs in this new system are identified by a channel number, ranging from 1 to infinity, assigned to them. To modify or stop a BGM or BGS, just select the channel number associated with it. It goes without saying, but there can only be 1 BGM per channel number, and 1 BGS per channel number (they run in different containers so a BGM and BGS can have the same channel number, but 2 BGMS or 2 BGSs can't have the same channel number).

To play a BGM or BGS with this new system, just use one of these plugin commands:

• For BGM: MUSH_Audio_Engine -> Play BGM
• For BGS: MUSH_Audio_Engine -> Play BGs

Since they are pretty much similar, I'll take the BGM window as an example:


Most of the arguments are pretty simple to understand since it's like a regular BGM: you input a filename, a pitch, a volume, and a pan. Now you also need to add a channel number to be able to identify this BGM in the container of all the BGMs playing at the same time.

Fade In is now an option by default. Why the RPG Maker doesn't offer that option by default but offers fadeout baffles me. Simply input the fade in duration in seconds.

The Auto Remover argument is to control when the BGM will be removed. Since it's a container of multiple BGMs, it is important to keep it clean and remove BGMs that are not necessary anymore. If this argument is set to TRUE, a BGM will be stopped and removed from the container automatically when the player transfers to another map. If the argument is FALSE, then you'll have to remove it manually.

The Interrupt argument also plays a part in the removal of the BGM. Basically, the interrupt argument sets the behavior of the BGM once an encounter is triggered. There are 3 possible behaviors:

1. Ignore: The BGM will just keep playing during the encounter and will continue so until the dev removes it or the player transfers to another map (depending on your auto remover setting).
2. Stop: The BGM will stop and be removed when an encounter appears. It will not restart after the encounter.
3. Pause: The BGM will be paused during the encounter and will continue from where it left after the encounter.

To stop a BGM/BGS, simply call the Stop BGM or Stop BGS plugin command. You will simply need to input the channel of the BGM/BGS you want to stop, and you can set a fade out (0 is no fade out).

Spatial Audio for BGM, BGS, and SE

Spoiler

This feature allows you to add audio sources on a map with a particular BGM/BGS or SE having varying volume and panning depending on the player's position to the audio source(s). There can be multiple audio sources for a single BGM/BGS, the volume and panning will be calculated according to the player's position to ALL audio sources. Beware of performance if you have TONS of audio sources, especially if they are dynamic (more on that below).

Audio Sources are created via events. Basically, you assign a BGM/BGS to a channel number and you attach audio sources to the channel number. So if you add the River BGS in channel 5, you can create an audio source for that River BGS by creating an event and adding this tag in the note box: <bgs_source: 5>. This will make that event an audio source for the BGS in channel 5. Here's the code to be put in the note box of an event for spatial BGM and BGS:

• For BGM: <bgm_source: channel>, replace channel with the actual channel number.
• For BGS: <bgs_source: channel>, replace channel with the actual channel number.

That takes care of the part about audio sources, but you need to actually add the spatial BGM/BGS to the audio channel. You can do so with plugin commands, just run it an auto event which erases itself after running the plugin commands. Use one of these plugin commands:

• For BGM: Add Spacial BGM
• For BGS: Add Spacial BGS

Here's how the window will look for a spatial BGM, but spatial BGS are essentially the same:


The filename, pitch and channel arguments are pretty straightforward. The other arguments can be a bit confusing so I'll go through each one of them.

The Max Volume is simply the maximum possible volume of the audio. Basically, the closer you are to a source, the closer to the max volume you'll be.

The Radius refers to a radius around an audio source inside which the audio can be heard. When outside the radius, the player can't hear the audio attached to the audio source. Inside the radius, the closer the player is to the source, the higher the volume.

The Strength refers to the curve of the volume variation INSIDE the radius. This does not affect the radius at all. Basically, how do you want the volume to increase or decrease based on position? If strength is 100, this will equal a linear function, meaning that if the radius is 10 and the max volume is 100, each 1 distance away from the source will result in -10 volume. If the strength is higher than 100, the volume will decrease slower (imagine a root function), if the strength is smaller than 100, the volume will decrease faster (imagine a parabolic function).

The Pan Type allows you to control how the panning will work if you want that feature (there's an option to deactivate it). There are at the moment only 2 pan types:

1. Origin Expand: the closer to the source, the less panning there will be. The further away from the source, the more the panning starts to kick in. The Pan Start Distance is the distance from the source before the audio starts panning, and the Pan Length Distance is the distance it takes for the pan to hit 100% (distance from when it starts panning, not from the source).
2. Linear Scaling: the pan is simply calculated with the X and Y position of the player to the source. This is more appropriate for side view like the Mario Bros games because if the player 1 tile left from the source and at the same Y position the panning will be 100 (meaning 100% playing from the right speaker). The Pan Start Distance and Pan Length Distance are not used at all with this algorithm.

The Dynamic argument is very important as it will affect the game performance. This should be TRUE if any of the audio sources associated with this channel can move, meaning that their position can change. If all the audio sources associated with this channel will remain at the same position then this argument should be set to FALSE to save performance. Basically if this argument is false, the game will only check if the player has moved to update the audio. If set to TRUE, the game will check if either the player moved or ANY of the audio sources associated with the channel has moved.

The Cross Fade argument is unique to spatial BGMs. Basically if set to true, it will cross fade the volume of the main BGM (the map BGM) and the spatial BGM the closer the player gets to the audio source. It will be just like this:


For spatial SE, it's pretty similar except for a few things. First of all, the SE will update when the player moves. The volume and pitch are calculated depending on the player's position to a single source when it starts playing. This is because SE are usually short and don't loop. A second difference is that there is no channel, the source is simply determined by the event ID (so it's using the event's position to the player).



With this method, you can also decide to turn ON or OFF the pitch and volume variance feature. This will override the general variance setting from the plugin. You can just leave them to "default" if you want it to follow the general variance setting from the plugin.

Voice Acting Channel

Spoiler

This plugin doesn't introduce a full voice acting feature, it just introduces the base architecture for it. I'll have another plugin for a fully functional voice acting experience. This plugin simply creates a separate audio channel called Voice Sound Channel (VSC) which is separate from SE and can have its volume controlled in the Options Menu.

All files for voice acting will require a new folder in the audio folder. To use the VSC channel, please create a folder called vsc inside the audio folder. So the path should look like this:
audio/vsc

To play an audio inside the VSC channel, you can use this plugin command:
MUSH_Audio_Engine -> Play VSC



The filename, pitch, volume, and pan arguments are pretty straightforward. As for the channel, this works exactly like the channel system with the new BGM and BGS system (see above).

The interrupt argument also works like the BGM and BGS system's interrupt argument, but this time it also affects changing to the menu. Here are the 3 types of possible behaviors:

• Ignore: The audio will continue playing without interruption until it ends. Changing to the menu or having an encounter doesn't affect the audio.
• Stop: The audio will stop if the player opens the menu, or if the player has an encounter. The audio will not restart after the player goes back to the map scene.
• Pause: The audio will pause if the player opens the menu or if the player has an encounter. The audio will continue from where it paused upon the player's return to the map scene.

SE Variance Feature

Spoiler

Hearing the same SE again and again can become monotone and a bit annoying sometimes, especially if it's an SE that plays very often. This is where the SE variance feature can really shine.

Basically, this feature, if turned ON, will make it so SE have a slightly varying pitch and/or volume every time they are played. You can activate/deactivate only the volume or only the pitch feature if you want. Essentially, it will simply do +/- to the pitch and/or volume. So if you have a 5% variation set in your settings, the pitch will be +/- 5% every time a SE if played.

You can turn ON/OFF this feature in the plugin's parameters and you can also change the variation %.



Don't forget that all SE in the game will be affected. If you have the UIS feature active, you can decide to turn on this feature for only SE or for only UIS. If you have the UIS function OFF, by default all UI sound effects will follow the SE settings, but you can turn that off with the Static Follow SE parameter.

Parent Map BGM Autoplay

Spoiler

It might sometimes be annoying to have to manually set the autoplay BGM for everyone of your map, especially if many of your maps fall under the same category, like a city made of multiple maps. It might be annoying to have to modify the BGM of all the maps if you decide later to change the BGM. This is where this feature can shine.

If this feature is turned ON in the plugin's parameters in the General Feature section, it will look at a map's parent map to find a BGM if the map doesn't have an autoplay BGM. If the map does have an autoplay BGM it will simply play it like it normally does. If the parent map doesn't have a BGM, it will look at the parent map's parent, and so on until it finds one. It will stop looking if there are no more parent maps.

Do note that there might be a little delay to read the BGM if it has to read the parent's map data, especially if it's many instance behind (parent of a parent of a parent, etc.).

VisuStella compatibility

Although I haven't actively worked on it, here's a message from @Nerine that incorporated my plugin with the VisuStella Options Core:

Okay, it is actually pretty simple, you just have to set the symbol name to Mush's corresponding name, Here's the UIS volume option made by myself. You can copy paste it or analyze it yourself.

{"Symbol:str":"uisVolume","Icon:num":"80","TextStr:str":"","TextJS:func":"\"return Mush.parameters.mushAudioEngine.menuOptions.uisVolumeText;\"","Accessibility":"","ShowJS:func":"\"// Return Boolean\\nreturn true;\"","EnableJS:func":"\"// Return Boolean\\nreturn true;\"","ExtJS:func":"\"// Return Value\\nreturn 0;\"","Functions":"","DrawJS:func":"\"// Declare Constants\\nconst symbol = arguments[0];\\nconst index = arguments[1];\\nconst title = this.commandName(index);\\nconst rect = this.itemLineRect(index);\\nconst halfWidth = rect.width / 2;\\nconst value = this.getConfigValue(symbol);\\n\\n// Draw Command Name\\nthis.resetFontSettings();\\nthis.changePaintOpacity(true);\\nthis.drawTextEx(title, rect.x, rect.y, halfWidth, \\\"left\\\");\\n\\n// Draw Gauge\\nconst rate = (value / 100).clamp(0, 1);\\nconst gx = rect.x + halfWidth;\\nconst gy = rect.y + rect.height - 14;\\nconst fillW = Math.floor((halfWidth - 2) * rate);\\nconst color0 = ColorManager.gaugeBackColor();\\nconst color1 = ColorManager.textColor(26);\\nconst color2 = ColorManager.textColor(27);\\nthis.contents.fillRect(gx, gy, halfWidth, 12, color0);\\nthis.contents.gradientFillRect(gx + 1, gy + 1, fillW, 10, color1, color2);\\n\\n// Draw Value\\nconst text = '%1%'.format(value);\\nthis.drawText(text, rect.x + halfWidth, rect.y, halfWidth, \\\"center\\\");\"","ProcessOkJS:func":"\"// Declare Constants\\nconst symbol = arguments[0];\\nlet value = this.getConfigValue(symbol);\\nconst lastValue = value;\\n\\n// Perform Actions\\nif (value >= 100 && (Input.isTriggered(\\\"ok\\\") || TouchInput.isClicked())) {\\n    value = 0;\\n} else {\\n    value += Input.isPressed(\\\"shift\\\") ? 1 : 10;\\n    value = value.clamp(0, 100);\\n}\\nthis.setConfigValue(symbol, value);\\nthis.redrawItem(this.findSymbol(symbol));\\nif (lastValue !== value) {\\n    this.playCursorSound();\\n}\"","CursorRightJS:func":"\"// Declare Constants\\nconst symbol = arguments[0];\\nlet value = this.getConfigValue(symbol);\\nconst lastValue = value;\\n\\n// Perform Actions\\nvalue += Input.isPressed(\\\"shift\\\") ? 1 : 10;\\nvalue = value.clamp(0, 100);\\nthis.setConfigValue(symbol, value);\\nthis.redrawItem(this.findSymbol(symbol));\\nif (lastValue !== value) {\\n    this.playCursorSound();\\n}\"","CursorLeftJS:func":"\"// Declare Constants\\nconst symbol = arguments[0];\\nlet value = this.getConfigValue(symbol);\\nconst lastValue = value;\\n\\n// Perform Actions\\nvalue -= Input.isPressed(\\\"shift\\\") ? 1 : 10;\\nvalue = value.clamp(0, 100);\\nthis.setConfigValue(symbol, value);\\nthis.redrawItem(this.findSymbol(symbol));\\nif (lastValue !== value) {\\n    this.playCursorSound();\\n}\"","Data":"","DefaultJS:func":"\"// Declare Constants\\nconst config = arguments[0];\\nconst symbol = arguments[1];\\n\\n// Perform Actions\\nConfigManager[symbol] = 100;\"","SaveJS:func":"\"// Declare Constants\\nconst config = arguments[0];\\nconst symbol = arguments[1];\\n\\n// Return Value\\nconfig[symbol] = ConfigManager[symbol].clamp(0, 100);\"","LoadJS:func":"\"// Declare Constants\\nconst config = arguments[0];\\nconst symbol = arguments[1];\\n\\n// Return Value\\nConfigManager[symbol] = config[symbol].clamp(0, 100);\""}

MUSH Audio Engine - Spatial Audio/Voice Acting/New Channels/etc.

Do you ever plan to add palette audio to this?

forums.rpgmakerweb.com

Tutorial Videos

Part 1: Introduction, Extra Channels, Volume Balance, Parent Autoplay, SE Variance



Part 2: New BGM & BGS System, Voice Acting System

​

Terms of Use

1. This plugin is free for commercial and non-commercial use.
2. You may edit the plugin, but please contact me before sharing any edits of the plugin.
3. You can repost this plugin as long as you link back to this thread.
4. Please credit MushroomCake28 in your project if you use this plugin.

Update History

Spoiler

v.1.01 - 2020/10/15

• Initial release

v.1.02 - 2020/10/17

• Fixed a bug where cross-fade would cause the map BGM volume to be set to 90.

v.1.03 - 2020/10/17

• Fixed a bug where the cross-fade would not work with the parent map BGM autoplay.

v.1.04 - 2020/10/27

• Fixed a bug where stopping an ME would set the BGM and BGS volume to null.

v.1.05 - 2021/02/13

• Fixed a bug that silenced BGM and BGS after playing a ME.
• Fixed a bug with BGM/BGS fadeouts with the new audio system if the fadeout was applied on a spatial BGM/BGS while the player was moving.
• Fixed a bug that caused the regular BGM to not return to its original volume if a cross-fade BGM was removed.
• Fixed a bug that caused the mush BGM/BGS to continue playing even during the Gameover scene.
• Added a plugin command to clear mush BGMs and BGSs.
• Added a feature to make sure spatial BGM/BGS are saved when the game is saved. Saving and loading the game should now work with spatial BGM/BGS.

v.1.06 - 2021/05/01

• Made the Game End command stop all MUSH BGM and BGS.
• Made the Return to Title event command stop all MUSH BGM and BGS.
• Updated how the ME works (when it stops MUSH BGM and BGS).
• Fixed the bug that triggered an alert after a battle on a map with spatial audio (by updating how the ME works).
• Fixed a bug that made it so the game didn't update the spatial objects on load sometimes.
• Added a developer mode parameter. When turned OFF, alerts will no longer appear in-game.
• Added a Plugin Command that allows to change the volume of a MUSH BGM and BGS.

v.1.07 - 2023/08/27

• Bug fix: Player transfers onto the same map no longer removes MUSH BGMs and BGSs.
• Saving and loading will now keep ALL MUSH BGMs and BGSs, not only spatial BGMs and BGSs.
• Bug fix: Fixed a volume with MUSH BGMs and BGSs when loading a save file.

Final Words

I'll be constantly updating this plugin as I'm using it for my personal project. If you encounter any bugs or other issues, please let me know and I'll try to remedy it as fast as possible. If it's a complicated issue related to your project in particular, please PM me.

 

[Rex Tenebris]

God, is @MushroomCake28 you? Because this script is godlike.
Spatial audio, two musics at same time, extra channels....

Wish MZ had a switch ON/OFF on the editor to make 3d/2d maps, I could even make a movie with this.

 

[MushroomCake28]

God, is @MushroomCake28 you? Because this script is godlike.
Spatial audio, two musics at same time, extra channels....

Wish MZ had a switch ON/OFF on the editor to make 3d/2d maps, I could even make a movie with this.

Hahaha xD. Thanks! I was just getting annoyed at having a river BGS playing on a huge map while there's only a small river in a small part of the map.

 

[MushroomCake28]

Released update 1.02

• Fixed an issue that caused the map BGM to be set to 90 if cross-fade with a spatial BGM was active.

 

[Heartless_Angel]

Thank you so much for this! I was waiting for a audio based plugin like this.
So I can make dynamic sounds and have more control again. <3

 

[MushroomCake28]

Released update 1.03

• Second bug I caught in the same day lol.
• Fixed an issue where cross-fade BGM would not work if the map's BGM is played by the parent map BGM autoplay feature.

@Heartless_Angel You're very welcome! I also love dynamic spatial audio, it makes the game much more immersive!

 

[Nerine]

Is it possible for you to add footstep sounds to this system?
It's really ineffective doing it with tileid/regions and events.

 

[MushroomCake28]

Is it possible for you to add footstep sounds to this system?
It's really ineffective doing it with tileid/regions and events.

I do intend to add some footstep functions but in another plugin. Region tile for footstep is however an effective way to do it because how else would you determine on what ground you're walking (different ground types will make different sounds)? I suppose it's possible by checking which tile was used when mapping, but that is incompatible with parallax mapping and each tileset has different ground types in different spot.

 

[Nerine]

I do intend to add some footstep functions but in another plugin. Region tile for footstep is however an effective way to do it because how else would you determine on what ground you're walking (different ground types will make different sounds)? I suppose it's possible by checking which tile was used when mapping, but that is incompatible with parallax mapping and each tileset has different ground types in different spot.

What I meant is events + regionid, but I think the best method is tileid and regionid combination since it allows you to set up tiles in such a way that you won't need regions for that allowing you to have more organized space.

And if you ever make a plugin for footsteps, please add an option for multiple sound files for one type of ground.

 

[MushroomCake28]

And if you ever make a plugin for footsteps, please add an option for multiple sound files for one type of ground.

Like playing 2 or more sounds at the same time?

 

[Nerine]

Like playing 2 or more sounds at the same time?

More like playing a random sound from a list.

 

[MushroomCake28]

More like playing a random sound from a list.

Ahh yes, I do plan on having that feature on general (called multi-sampling), like being able to use it the UI sounds too. It's actually my next plugin.

 

[Nerine]

I've got this issue that after a battle bgm and bgs volume is being set to null, any ideas how to fix it?

 

[Nerine]

Adding the plugin from the list again seems to have fixed the issue.

 

[MushroomCake28]

I've got this issue that after a battle bgm and bgs volume is being set to null, any ideas how to fix it?

The bgm and bgs volume from the options menu?

 

[Nerine]

The bgm and bgs volume from the options menu?

Yeah the bgm and bgs stopped playing, but when I looked at the volume in the options it was set to null. But as I said readding the plugin (not even reinstalling, just picking it from the list again) seems to have fixed the issue. No idea why it started behaving like that.

 

[Nerine]

Okay, it still happens and readding doesn't fix it. It only happens after victory and defeat. I thought it was fixed because I pressed escape so it'd be faster, but after returning I realized that the bug is still there, you just have to win or lose the fight.
After the fight is over, you can't hear bgm and bgs because the bgm and bgs is set to NULL and have to manually adjust the volume from the menu, so I can hear it again.

 

[MushroomCake28]

Released version 1.04

• Fixed a bug that set BGM and BGS volume to null when stopping a ME.

@Nerine There you go. It should be fixed now.

 

[Nerine]

Released version 1.04

• Fixed a bug that set BGM and BGS volume to null when stopping a ME.

@Nerine There you go. It should be fixed now.

Thank you.

 

[MushroomCake28]

Just published the first tutorial video on how to use this plugin. Part 1 will convert the extra audio channels (UIS/VSC), volume balancing, parent map autoplay, and the UIS/SE Variance feature. More to come in part 2 and part 3.

​