When I hear "fear" of much-stronger opponents affecting how you fight, I think of something that is going to create very quick, clean, dominating wins for the stronger battlers. I don't really think of implementing this in a "balanced" way, because giving disadvantages to an already-weaker contender is by definition imbalanced.
I think that kind of imbalance is fine, especially when it's the player that's favored, since making wins as quick and clean as possible against much weaker opponents may save your player a lot of time that they'd otherwise spend mindlessly mashing the Attack command. It's much dicier to use against your player, since the last thing a player wants when they're already facing long odds is to take another disadvantage that adds insult to injury, but if it's really important to you to discourage players from taking on very tough foes for some reason, then you can use Fear against players as well.
A few things you could do to implement this idea of "Fear" into your combat system:
- Have Visible (or even Random) Encounters try to avoid the player entirely if the player is much more powerful than the troops in the area.
- Have battlers (enemies or maybe characters) who are much weaker than their opponents often "Flinch" during combat instead of performing their normal command - have the Flinch act as a lower-effectiveness version of a Guard move. When enemies do this, make the animation and information really short so that it speeds up the easy battles.
- Have enemies - maybe even characters - individually run away when much weaker than their opponents. The reasons for having the enemies do this are obvious. The reasons for having the characters do it are less obvious - this will make the battle almost impossible to win, but it will also allow the player to avoid a Game Over if the rest of their party is defeated since someone got away alive!
- If using an ATB, cause spells (especially ones inflicting status effects) to take much longer to cast, as Fear distracts the battlers. Mechanically, this makes it much harder for the weaker team to cheese the stronger one with Poisons, Sleep, etc. that can make what should be a clean victory much more messy.
On the other hand, there are also reasons to consider doing the exact opposite, and introducing Bravery as a mechanic instead of Fear, giving the weaker team
advantages which allow weak enemies to remain at least somewhat challenging all game long, and too-strong enemies into a stiff but fair challenge. (Again, like Fear, it's fine to make this into a one-way street that will only ever benefit the player - in fact, a majority of the time I recommend doing so!) For example, characters affected by "Brave" could have significantly increased critical hit rates and higher damage multipliers on crits, so that while they will usually fall quickly, they have at least a chance to inflict meaningful amounts of damage and possibly win the battle before they fall.
