But suppose I think of new challenges for the player. I find myself thinking "why would this happen? Why would the player do this?" I write a sentence and wonder... why? Maybe I'm running into so many whys because I'm making a horror game and it's not actually a realistic thing I guess.
I see this problem a lot. Usually it crops up with newer or inexperienced writers. Keep in mind, I'm not a veteran. I have nothing published. I do not claim to be an expert writer or author.
Here's the problem:
You're making your story secondary to the gameplay. You need to prioritize.
Do you want a gameplay heavy game? If yes, your story should be secondary. You can tailor it later to fit the narrative. But, it will be difficult for you to go into a game without at least the standard "10 Plot Point Structure". At minimum, you will need this. You can ignore every other writing step once you have this and "improvise" as you go along. After all, you're making a game primarily for the gameplay and not the story.
If you want a story heavy game... Then you need to design every single aspect of your gameplay around that story. You can't just "think up a new challenge" and try to justify it later. You need to already have the entire story planned (maybe not all the details, but absolutely every major aspect of it, including the path the player will take from the start to the end), start to finish, before even opening the program. You can fill in the blanks with gameplay if your primary purpose is storytelling.
You have to choose one.
The issue you're having is one which I see primarily in "Quest Creation" in most RPG's. That is to say, the dev wants to add a quest so that there's "content", or a "reward" they want to give out, so they start with the reward, then work backwards to what they think is worth the effort of getting this reward, then backwards again to try to think up a reason to get the player to engage in this effort, and backwards again to try to fill in what's left of the story and find a good place to put it.
The key word in all of that is:
Backwards.
A storyteller doesn't work backwards. A story teller doesn't give the player a Quest unless they have a story to tell. Here's the story I want to tell. Self-contained. Links in with the Main Plot. It's relevant to one of the party members. It fleshes out the Lore of the world. Etcetera. Without a story to tell, a quest will not exist with a storyteller.
The storyteller then creates the story of the Quest, all its twists and turns and plot points, all its locations, and then decide on a reward for completing it when the story being told is done. They don't work backwards.
A mechanics type dev will work backwards. Trying to justify the existence or order of things to be completed after the fact. This isn't the wrong way to go about things, but it will fail to appeal to anyone looking to your game for a good story. If you want people to enjoy the gameplay, you can use as weak of justification as you want for getting the player to engage in more of the fun gameplay. No story required.
Think of the two as the difference between a "popcorn action movie" and "an award-winning cult classic". They are both great in their own right, but they appeal to different audiences because of what they do well (though there is usually some overlap. I enjoy big dumb action movies as well as intellectual pieces).
So, my question here is: would you play a game that has some form of bad reasoning? (Like how people go investiagte a creepy sound in a horror movie when they should not). Would you play a game where you constantly need to go from point A to point B to point A again and again to Point C? I mean, you constantly need to go from one place to another again and again?
Yes, I would. See, I play Earth Defense Force. I also play other games somewhat like it. Here's the gameplay of Earth Defense Force: "Kill B-Movie Tropes (giant insects, kaiju, giant robots, space ships, etcetera), collect upgrades, and move on to the next level as the difficulty ramps up and the map changes." Here's the bad reasoning: The plot is delivered primarily over "radio comms". That is... other soldiers alongside you, generals, tacticians, scientists, equipment liaisons, etcetera. Almost all of it is super hammy and stupid. In one of the games, early on, the scientist says, "we can beat the Giant Insects by using our Wing Divers (girls in jetpacks, using laser/plasma weaponry that fly around)! No matter how much the Insects have evolved, they can't fly!" Two missions later, the Wing Divers are caught in giant spider webs that are easily seen, easily avoided, and relatively easily destroyed. 40 missions later, you encounter giant wasps, and the scientist guy is horrified. 20 missions after that, and you encounter Dragons. Is it a stupid set up that "yeah, there will definitely be flying enemies later"? Yes, yes it is.
However, the entire game is played as a "big dumb action movie". That is to say... it's dumb fun. You run around, shoot stuff, collect items, and do it all again the next mission. There's nothing complicated about the gameplay. You don't have to protect civilians, buildings, troops, or your bases. You don't have to escort anyone, anywhere. You don't have to fiddle with objectives at all. The objectives never get more complicated than "you need to destroy this one specific enemy to end the mission". That enemy is usually a boss monster or a monster that spawns other monsters.
Does the game need a story? No. Do you get one? Yes you do. Is it silly and hammy and over-the-top? Yes it is. Does that make it enjoyable? Yes it does. Why? Not sure. But, it fits in with the gameplay. Why can't we blow up these space ships yet? Because the story says we don't know how to blow them up. Why else? Because the gameplay doesn't have those space ships expose their weak points in this mission to allow you to blow them up. Story to justify the ramping up of challenge, training, introduction of new enemy types, and location changes.
That's it.
It's still immensely fun.
I enjoyed Gears of War 1 for this same reason. Is there a story explanation for much of the game? Not really. Lore? No. You have a chainsaw gun. Cut enemies in half and shoot them in the face. What's the plot? Pick up a bomb, plant it in an underground tunnel to kill all the enemies you've been fighting in this war you know nothing about. That's it. Big. Dumb. Fun. Plot necessary? Not really. When I played it, I didn't even really know what the plot was for 90% of the game, until the final mission when they told me, "Hey, there's a bomb on this train, protect it and activate it, and you win". Okay, cool. I'll gladly shoot my way through 5 levels for that. Who are these characters alongside me? No idea. Their stories? Don't know. Game doesn't say. It doesn't matter. They're pretty funny, mostly. They banter to fill time between combat.
You need to decide what kind of game you are making. Is it a story-driven game or a gameplay driven one? If story, then you need to plan all of that out before you commit to doing gameplay. If gameplay, then you need to commit to making the game fun and then creating a story as a weak justification later.
After all, what's the story of Five Nights at Freddy's? Is there one? There's some Lore. But, it's silly and stupid at best. It makes absolutely zero sense in the context of the real world, or even the world the games take place in. But, how popular was that series? Why was it popular? The gameplay mechanics and the scares. Do people still want to know about the Lore? Sure. Why? Because even in the weak justification they had it provide for the context of the game... they left a lot of mysteries and weird things to pique curiosity.
A story doesn't have to be great to produce a great game. It just needs to fit the gameplay. The reverse is also true. You don't need great gameplay to make a great game. You just need gameplay that fits the story (the Last of Us has pretty subpar gameplay, but people play it primarily for the Story).
I hope that helps.
