Bughunting

Andar

Veteran
Veteran
Joined
Mar 5, 2013
Messages
31,293
Reaction score
7,637
First Language
German
Primarily Uses
RMMV
Bughunting

Bughunting - for many the most dreaded and difficult part of game making, and the reason why more and more software companies are using the customers for the last stages of bughunting by letting them pay for "early access" or even selling them beta-versions of the software that are then patched by service packs one or two months later.
Not that there is anything wrong with those practices - you'll need thausands of testers to find even 90% (not all) of the bugs in more complex programs. I just wish they would be more truthfull of this instead of making it sound they would do the customers a favor instead of the other way around.

Bughunting is simply too complex to be easily successfull as soon as the program or game in question increases beyond a certain size.
And for that reason no one can give you a certainity of succeeding - that isn't the goal of this tutorial either.

Instead, this tutorial is intended to give you an understanding why bughunting is so difficult, what you can do to reduce that difficulty and how to proceed to catch and solve at least most of the bugs in your project. It is not a "solve-all" step-by-step, because that is simply impossible to do.
The best any bughunting tutorial can do is explain to you why something is difficult, how to prevent the most common mistakes and start you into the correct direction for your project.



1) Background: There is no random effect in the computer
The first thing you need to understand is that there is absolutely no randomness in the computer - everything has a specific and reproducable cause.
That even includes the so-called "random number generators" - their numbers aren't random, they just look like random to humans, because no human can calculate with the formulae used to determine the number sequence.

Perhaps you have seen some games that ask for a "seed" before starting a random generator? That seed is basically the starting number for the formula to determine the random numbers.
If you always use the same seed for your number generation, the so-called random numbers will always be the same.

However, if you cannot determine the seed (especially if the program uses something like current date/time as the seed), then the results will look truely random to all players.

The reason why I point this out is that you have to know that everything has a cause. If you haven't found it, then that means you have to keep looking.
If you haven't found an autorun freezing your non-script game, then the reason is that you overlooked it and not that it doesn't exist.



2) Background: Why is the developer the worst playtester?
Whenever it comes to playtesting after the game is finished, everyone suggest to search other playtesters. Do not test the game yourself (at least not after development is complete, you'll have tested the incomplete project multiple times already).
There are several reasons for this (including that you need your time for fixing reported bugs instead of searching for them), but the most important reason not to test your game is very simple:
You know how the game should be played, and that limits your ways of looking for bugs.

Example:
Let's assume that in your game the player needs to pass a lot of gardens with walls and doors between them.
You already passed a dozen doors where the player had to identify the correct key and then you made the next door stuck, requiring a crowbar instead of a key.
As a developer, you'll probably continue to check which items opens the door.

The player (who don't know that solution) starts thinking "If none of the keys opens the door, what other ways are to get over the wall?".
He tests climbing at all walls and finds the tile where you set the wrong passability, allowing the player to walk on walls.

That was a simple example - believe me, a player who doesn't have the solution will test a lot more crazy options. And by doing this he'll find bugs that you as the developer never considered possible, because you never even thought of that possible solution.




3) Background: Don't experiment - know what you're doing
There is a lot of "less than optimal" advice floating around this forum, given from people who don't understand what they're talking about and just randomly try changes until something appears to work.
The classical worst advice (seen multiple times) is "if an autorun event results in unmovable player, make it a parallel process".

No, in most cases you need to switch off the autorun to allow the player to move after the event - and a few other solutions, but only rarely the switch to parallel would be correct.
Instead the parallel process will allow the player to move (the autorun is intended to prevent player input), but only at the cost of a lot of lag and possible timing problems that are worse - because both autorun and parallel process loop infinitely.

There is another topic explaining exactly the same effect for scripters instead of eventers - but the problem and the background is the same:
http://forums.rpgmakerweb.com/index.php?/topic/5108-a-good-article-for-you-budding-scripters/

Read the excerpt - and never use random experimenting to solve your problems.
In 90% of all cases this will only look like solving your problem while in reality creating more bugs which are often more difficult to find.
If you don't know a solution - ask before ruining your project.


This is because the worst mistake you could ever make is to use bugs (like using a parallel process at the wrong place) to hide other bugs (like that player-freezing autorun-event that wasn't switched off).
The purpose of bughunting is to find bugs, not to hide their errors or messages. Or in another comparison:
If the oil warning lamp on your car goes on, you need to change the oil - blackening the lamp to hide the red light is the wrong solution, because your engine will fail if it runs on bad or not enough oil for too long.

So if someone gives you advice on how to solve a problem, and you do not understand that advice, then it's time for you to look for tutorials about that option and make a new, temporary project to test what that solution really does.
But do not simply experiment with the advice - the advice giver cannot know your entire game and so cannot know if that advice might cause other problems. You need to be the person who checks for that, and because of that you need to understand what the solution does.

Not understanding what you do gives you a 50/50 chance of introducing new errors, and those errors will be more difficult to find. That is simply because now they are also hiding your other bugs and because you believe them to be needed bug fixes, not additional bugs.



4) Background: why is the word "parallel process" a lie?
When you start looking for the common explanations of the "parallel process", you usually get an answer that it is something that runs at the same time ("parallel") to the player actions and to each other parallel process.
While that is an easy way to descript the visible effects of a parallel process to people, it is technically wrong and a lie.
Simply because the computer does not work parallel for those cases.

Only the newest computers have multiple cores - before them parallel processing was physically impossible on a single-CPU computer.
And even with multicore-CPU's the programs who really are using parallel processes are very few and very specific - in reality Windows distributes different programs (like itself, your virusscan and your game) to different cores to make use of those.
But 99,9% of all programs (and that list includes the RPG Makers) run on only a single core and cannot use real parallel processing.

That means what really happens is that the so-called parallel processes are handled one after another - just so fast that it looks like parallel to the player and most developers.

In truth there is a game loop that contains the player action, all background actions (like BGM or map display), all autonomous move sequences and all parallel processes.
This loop processes all components one after another, but very fast - everything once per frame to be exact.

Since a frame is usually 1/60 of a second that looks like parallel to a human, but in reality it is one after another.




5) Background: Timing problems - what are they and why are they the worst problems?
The worst problems that can happen are timing problems - when you plan something in a specific order and it won't happen in that order because something disrupts the logic you implemented.
But why do timing problems appear? The reason is the game loop I mentioned above.

Let's assume you have two autoruns, one conditioned and switched on by the other. Both are parts of that game loop - but you do not know in which order they are processed.

  • If the conditioned autorun is first in the loop, the switch is not yet ON and it will be skipped. Then the other autorun turn on the switch - and IN THE NEXT loop/frame the conditioned autorun will be executed.
  • If the main autorun is first in the loop, then the switch will be turned and when the conditioned autorun follows, it will be processed IN THE SAME loop/frame.
Do you see the underlined difference? The problem here isn't that the autorun is delayed by 1/60 of a second in one case - it's that the entire loop with all other events may or may not be processed before the other autorun has a chance of working, and that might change a lot in map or switches.

For all praktical purposes, the order in which autoruns and parallel processes are handled is random.
No - in reality it's not (see 1) but anyone even trying to work the logic and sequences behind this needs a hard kick to his/her backside, because that is too complex and too much work to solve.

Instead the solution is to reduce the number of elements in the game loop. The sequence in which autoruns are processed does not matter if there is only one autorun.
And THAT is the main reason why we advice to have every cutscene controlled by one single autorun only, and to use parallel processes only where absolutely neccessary.
You will not see any difference during regular gameplay or developing, but as soon as it gets into bughunting, a large number of active parallel processes and/or autoruns will make your bughunting a lot more problematical.

To conclude this part I'll give you a list of the elements that are placed in the main game loop and may or may not cause timing problems:

  • ) all autorun event pages (both common events and current map events if set to trigger=autorun, and executed if conditions are true)
  • ) all parallel process event pages (both common events and current map events if set to trigger=parallel process, and executed if conditions are true)
  • ) all autonomous movement settings other than "fixed" will create an element in addition to possibly the event page element.
  • ) every single "set move route" command will create a new element, but only one per object (overwriting previously active move routes)
  • ) player action (unless there is an active autorun stopping the game loop, player input will be processed as well)
  • ) background media (display of animations and/or sound). Everything on the screen that needs more than one frame (including the balloon command from events) will create an element in the game loop.
  • ) scripts may or may not add elements into the game loop
Keep this in mind whenever you have problems with sequences



6) For Want of a Nail: Error sequences
If you do not know this proverb, I suggest reading about it:
https://en.wikipedia.org/wiki/For_Want_of_a_Nail
Because it is important to understand that for bughunting.

The computer always reports the last error, when something finally broke down. This is especially true for script errors - the backtracer scripts have been programmed to give the origin of such an error sequence.
You need to understand that solving only the last effect reported will not remove the original error, and usually a bug can have a lot of different results if it is at the wrong place.

Whenever you get a bug message, you need to follow the logic used in that part of the game to the point where it starts, and begin searching for errors from there.
If you ignore the why and the sequence, not only you're wasting effort but you also risc hiding other bug. Because it's a lot more expensive to solve a battle lost than to replace a nail.



7) Errare Humanum Est: Be your own worst critic
It's very simple: If everything were correct, you wouldn't need to ask for our help. And if something that had been done and tested by several hundred thousands of users in the last several years, and now you have a problem making it work - where do you think the error has to be? In your work or with the uncountable others who used the same with correct results?

But despite that simple fact, a lot of people still claim that they did everything correct and the problem has to be in the engine when we point out to where else it has to be.
And usually, a week later there is a small response "you're right, I overlooked XY".

Everyone makes mistakes - even the people of ancient Rome knew that as a proverb.
I learned that about thirty years ago when I calculated (1+...)*(2+...) = 3+... in an advanced math test and didn't detect that error in three complete checks. I knew that my result had to be wrong, but I always started the recalculation at the second line - not believing that I could have made a mistake in the simple step from the first to second line.
OK, it was a bit more hidden in that task and it was the only mistake I made in the entire test (I could check that three times because I finished with all other task of the test early), but I still remember when I got the corrected test and finally found that mistake...

That is one of the reasons why bughunting is difficult: most people believe themselves to be free of mistakes, and going over the same events dozens of times while solving one bug after another gets repeative to the point of booring and no longer really seeing what's there.

Do not fall into that trap, but really check everything again and again, even if you checked it one day ago for a different error. If there is an autorun event on the map blocking you, then open and check every event to find that - beginning in the first map line and following the grid. Do not jump the grid or check another event one line below simply because it's near - you have to make sure that you don't overlook a single event, and that can only be done by checking methodically.



8) NAME everything - but check the IDs, not the names
When you check the events, the database and other components of the RPG Makers, you'll see that almost everything has both an ID (most times the number is in front of the lists) and a name.
However, the engine itself works purely on IDs - the names are for the human developer's convenience only.

That does not mean that you can ignore that convenience - whenever you use something, give it a name before leaving the screen. And preferably a name that will tell you what it is used for.
Naming everything used has several advantages for you. First, if it has a name then it has been used before - you won't make the mistake of using it again for a different function (remember to check your scripts for used switches and variables, they aren't named automatically - see point ).

Second, the name will tell you what you used that for - helping in the bughunt if there is a problem later.

Third, in some places the editor will only display the name but not the ID - for example the event command "change actor name". Please always give your actors names, even if you plan to have the player rename them at game start. Because without a name, a lot of the actor commands are difficult to read in the events - and if you have several unnamed actors you'll never know if the command selected the correct one.

Fourth, in some RPG Makers there are search functions for names and there are similiar external tools for those RMs that don't have build-in search engines. A good naming convention will help you finding what you want a lot faster.



9) Game loop: Movement command elements
"Set Move Route" - have you ever stopped five seconds and thought about the name for that event command? Why it is called "set move route" instead of for example simply "move event"?

The name for that event command is extremely fitting, because it doesn't move events - it programs (sets) a move route that will be executed by that event.
That move route is one of the elements placed in and processed by the main game loop. But there is only one such move route element for each event - events cannot move twice per sequence.

What does this mean? It means that if several move routes are set for the same event, only the move route set last will be executed and all others will be overwritten. Considering that some of those routes might loop (and a route set by autonomous movement will always loop), it is a very bad idea to spam several move routes at a single event from different sources.
This includes mixing autonomous movement and set move route (unless one of them only contains non-move-related commands).

It also explains why a cutscene should have only one controlling event - if events get their move command from different controllers, it can only increase the timing problems already caused by several autoruns and parallels.

This is also the reason why to stop an old move route, you can simply send a new set move route command without any commands in the move route - the last set move command will overwrite all previous commands. This does not work with autonomous movement however, because autonomous movement always loops and creates a new move route as soon as the previous one is empty.




10) Reference: Quirks of the program
There are a few quirks of the RPG Makers that you need to know in order to solve their problems.
I'll simply list the quirks for Ace here, search for the terms to learn more about why that is so important

  • The first tile in the B tab of every tileset has to be set to star passability
  • The A4 walls contain ceiling tiles with a special passability to simulate a two-level-dungeon in a single map
  • autoruns and parallel processes always loop and need to be switched off or erased to prevent a restart
  • erase event only works until the map is entered again - use it for events that need to be processed each time the player enters a map.
  • you need to use a self-switch to switch off autorun/parallel pages that had to end permanently
.


11) Reference: About Scripts
Scripts can break the game engine by entering the wrong commands and cause every imaginable problem (and a few beyond imagination).

Because of this, you need to make sure all scripts function properly before playtesting anything else.

Unfortunately bughunting in scripts is even more problematical than regular bughunting, especially if you don't know how to script.
I suggest following the procedures in this tutorial for the scripts you're trying to use:

http://forums.rpgmakerweb.com/index.php?/topic/40589-how-to-use-a-script/
 
 
 
 
I hope that these tips and infos make your future bughunts easier
 
Last edited by a moderator:

Chiakscare

Resident Tifaholic
Veteran
Joined
Jul 24, 2014
Messages
489
Reaction score
148
First Language
English
Primarily Uses
I am splendidly impressed by the thoroughness thru which you have produced this pristine article. The information is wonderful and revealing; the quality will make me read all of your other articles. I particularly enjoyed your writing on playtesting and what parallel processing truly entails.

Thank you for posting this. I apologize for not giving them a read until this one; continue your excellent work.
 

Nightblade50

Developer of "Delta Origins"
Veteran
Joined
Nov 10, 2016
Messages
2,133
Reaction score
4,253
First Language
English, French
Primarily Uses
RMVXA
2) Background: Why is the developer the worst playtester?
Whenever it comes to playtesting after the game is finished, everyone suggest to search other playtesters. Do not test the game yourself (at least not after development is complete, you'll have tested the incomplete project multiple times already).
There are several reasons for this (including that you need your time for fixing reported bugs instead of searching for them), but the most important reason not to test your game is very simple:
You know how the game should be played, and that limits your ways of looking for bugs.

Example:
Let's assume that in your game the player needs to pass a lot of gardens with walls and doors between them.
You already passed a dozen doors where the player had to identify the correct key and then you made the next door stuck, requiring a crowbar instead of a key.
As a developer, you'll probably continue to check which items opens the door.

The player (who don't know that solution) starts thinking "If none of the keys opens the door, what other ways are to get over the wall?".
He tests climbing at all walls and finds the tile where you set the wrong passability, allowing the player to walk on walls.

That was a simple example - believe me, a player who doesn't have the solution will test a lot more crazy options. And by doing this he'll find bugs that you as the developer never considered possible, because you never even thought of that possible solution.
Wow, you are SO right there
 

Canini

Veteran
Veteran
Joined
Mar 29, 2016
Messages
1,025
Reaction score
686
First Language
Swedish
Primarily Uses
RMVXA
A very throughout tutorial, the explanation for autorun/parallel process were particularly good. A question regarding getting playtesting:
When would you say it is time to get an external playesters? When you got a few hours of playtime or after you have put together a few maps with the different styles of gameplay.
 

TheoAllen

Self-proclaimed jack of all trades
Veteran
Joined
Mar 16, 2012
Messages
5,573
Reaction score
6,503
First Language
Indonesian
Primarily Uses
RMVXA
A very throughout tutorial, the explanation for autorun/parallel process were particularly good. A question regarding getting playtesting:
When would you say it is time to get an external playesters? When you got a few hours of playtime or after you have put together a few maps with the different styles of gameplay.
When you're able to demonstrate a thing or two. Personally, I don't like if I already have x hours of gameplay, I'd call it a demo and get my playtester. But it's when I already have a proof of concept, like this is the prototype of a gameplay I want to make, and already have a milestone like first or second boss, I throw it to my testers. I threw mine to my tester even though it's less than one hour of gameplay (if I play it myself). Although, it mainly for feedback rather than bughunting
 

cabanas_ds

Veteran
Veteran
Joined
Aug 13, 2020
Messages
54
Reaction score
16
First Language
english
Primarily Uses
RMMV
incredible post. super scary tho, it's made me question everything about what i'm doing.
 

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

Latest Threads

Latest Posts

Latest Profile Posts

Evil7 added!


I did a bit of a remake on one of the maps, also I made a particle for jumping into the portal, and one for traveling through the portal. I'll probably add more sparkles later to embellish it. :LZSwink:
Polished the animation and added details :kaojoy:
Ami
What do you think about the Wood Bridge inside of Volcano?
Yet another gaming question, i'm currently playing Last of Us 2 and want to get another game to play next but can't decide between Ghost of Tsushima, Miles Morales & Jedi Fallen Order. After the latest mandalorian i AM on a star wars kick, but i also LOVED the last spiderman game that prequels Miles Morales. Ghost of Tsushima is the bottom of my list but i heard it was AWESOME. Just want a good story & fun gameplay.

Forum statistics

Threads
105,686
Messages
1,015,535
Members
137,362
Latest member
sdramose
Top