I'll take you up on that offer and list some of the random questions that pop into my head!:
- Is there such a thing as too much Quality Assurance?
- As a solo developer it can be hard to balance game creation with going back through and making sure the creation isn't broken. Is there a magic number or rule to follow in creating features/story-arcs versus making sure said creations aren't monsters?
- I have a few family members that volunteer their time to play-test my game. Currently, I ask them to write-down issues they see and their likes/dislikes. Are there specific QA questions or guidelines that would 'improve' their feedback? Should I have a 'QA' Sheet to guide them in their feedback?
- Time is valuable. Any tips/tricks that help testing/QA efforts more efficient?
Thank you for spending your time giving back to the community!
Whooo! Questions! I loves me some questions. They're interesting ones too! I apologize in advance for my poor grammar and abuse of punctuation.
- Is there such a thing as too much Quality Assurance?
Yep there sure is. Whether it be through spending too much time looking at one section or losing all motivation because you think you've found everything. We call this exhaustive testing and exhaustive testing is where things become a big waste of time but we continue anyway because we want a product to be the best it can be. At the point where you feel you can't find any more serious issues you begin to create them and you will pick holes in everything.
In a work environment you have tasks delegated to you and time set aside for what we call exploratory testing. Exploratory testing is when you test with the intention of finding bugs. Sometimes you can have one level and be told to work on it for several days. You may feel after two days you've found everything however and you've still got a day before a new build arrives! Luckily you don't have that problem at home. Assign yourself your piece of content you want to perform testing on and assign yourself a time frame. If you aren't finding any serious issues such as progression blockers, crashes and logic breakers and still have a lot of time to spare then maybe it's time to perform a graphical or audio sweep? Perhaps some destructive testing whereby you attempt to do silly things like closing your game suddenly or mess around with all the options.
If you hit your time limit however and you are still finding serious issues that seriously break your game then it's time to assign. Give yourself another hour or so.
None of this is going to stop you performing too much Quality Assurance however if you aren't motivated to do it. It also doesn't hurt to have someone else look over the same area too, get someone you trust to go over it for the same amount of time. You'll be amazed what other peoples eyes see that yours don't.
- As a solo developer it can be hard to balance game creation with going back through and making sure the creation isn't broken. Is there a magic number or rule to follow in creating features/story-arcs versus making sure said creations aren't monsters?
I may be misunderstanding what you've written here but I'll give it a shot and if I'm wrong then forgive me and we can hash this one out a little more.
I would say you need to review your major changes on a regular basis, things that have knock on effects that may effect logic such as switches and events being triggered in multiple places. Make note of your big changes such as logic and performance and check them maybe once a week or once every three days. Make sure they behave as intended when simply triggered or started up. Perform a little destructive testing so for example go to a location you shouldn't be able to get to until you trigger these big events.
If we're talking numbers here so balancing monsters and stats I would do maybe a weekly playthrough and make a note of what level you are at each area. Obviously if you're struggling you'll know something is too hard. If you're tearing through the levels it's too easy. A great way of testing weapons and monsters however is to create a testing area. Create items that boost you to the intended level, create weapons and armour with their stats and create leveled monsters you can spawn on command.
- I have a few family members that volunteer their time to play-test my game. Currently, I ask them to write-down issues they see and their likes/dislikes. Are there specific QA questions or guidelines that would 'improve' their feedback? Should I have a 'QA' Sheet to guide them in their feedback?
Have a bug reporting structure that you can enter into a database. You'll always need the following.
-
Summary (A brief description of the issue)
[Level 2: Billy's Kitchen] When running along the table top Billy dropped through the floor and out of the map.
-
Severity 1-5 (1 Being a game breaker and 5 being something petty like some small Z fighting.)
1 (Anything that ruins the users progression on the path they were taking is super serious.)
-
Frequency (If you do it three times, how many times out of that 3 does it occurr? You may want to do 5 or 10 times, it's upto you and your patient family and friends!)
100%
-
Steps Taken
1. Started a new game.
2. Progress logically through Level 1: Billy's Goat Shed.
3. At the start of Level 2 turn West and face the kitchen table.
4. Climb the chair onto the table.
5. Press SHIFT to dash along the surface.
You'd do well to find a way of recording their play sessions if they're testing it on PC because then you can save off media such as videos and screenshots.
As for feedback you can ask them questions based on what content you want them checking. For this example we'll say the stability of the level "Billy's Kitchen". Questions such as:
- How would you rate the stability?
Terrible, Poor, Average, Good, Very Good
- Was there anything that ruined your experience?
________________________
- Were there any sections you found difficult(In RPG Maker this may be your monsters/puzzles)?
_______________________
_______________________
_______________________
- Was there anything you'd like to see with this content? (Word that a bit differently so if they're testing a bow you'd say the bow.)
_______________________
- Overall what score out of 10 would you give this content?
__
- Time is valuable. Any tips/tricks that help testing/QA efforts more efficient?
- Get yourself a method of bug tracking sorted and begin giving your builds numbers so you can track what build a bug was entered on. Looking into version numbering and build numbering should help with this process.
- Assign yourself the time to perform QA. If you're creating content for four hours a day assign yourself half an hour at some point to test an area. Even if it isn't the area you're working on pick something and give it a test. Coming back to content you haven't seen in a while will yield bugs you didn't see before.
- Have others regularly give you their feedback on content but unless you've got high priority issues with it don't worry about having them retest it over and over.
- Create test case documents for your content. These are tables in Excel where we perform structured tests. For example you would have a Bow we should check: The graphical components are all there and working, can it be equipped, does it work in battle, can it be unequipped, can it be sold/traded/destroyed, does it function correctly in inventory? Some simple Pass/Fail criteria on these should do the trick. Have a column at the end for your bug numbers in your database/bug tracking document.
for your map checking etc. you need to check things like stability, performance and traversal.
- Prioritize your Quality Assurance! Your crashes and game breakers are super important as is your traversal through your world. Your balance probably comes second as in the early stages you could God mode your way through until you've ironed out any progression blockers to do with movement. Your graphical issues come last, they can be polished up before release. (Believe me!) text and dialogue sits in the middle somewhere.
I hope this goes some way to helping you, a lot of it will make more sense when I upload some documentation in the near future. Hopefully you've got some terms to Google and look into.
