I usually come at it from a direction that I wouldn't generally recommend to other designers (unless they already have a lot of experience building up game mechanics), because it's easy to miss the forest for the trees with this approach. Here's how I do it, though:
1) Large Concept
I usually have a place or situation in mind, so I think about what would be a
cool and thematically fun boss to face in that kind of place or situation.
2) Signature Moves
From there, I think of what kind of moves would be cool to see such an enemy do. A huge slime might split itself into five smaller slimes, and it might even start consuming some of those slimes to regain HP. A robot might take a turn to charge up powerful laser beams. A mimic or illusionist might create clones, move around the screen, and challenge you to find the real one (with hints given via animations). One or two additional bread-and-butter damage moves can be nice to round out the boss' arsenal. I'll use whatever enemy behavior tools are at my disposal to make them behave reasonably - for example setting a switch when the robot charges up its laser, and making that switch a requirement for a high priority slot in the Enemy Behavior list in RPG Maker.
3) Additional Viable Strategies
Once I have a few moves in place, I'll playtest the boss myself with a group of characters that I feel would be appropriately leveled for the boss, and often what I'll find I have at this point is an interesting but wildly imbalanced or overly gimmicky battle. The focus at this point is on asking myself what kind of strategies can work against this boss, and what kind of strategies that players might reasonably want to try
won't work against the boss. And I'll add extra mechanics that make those strategies more feasible. If it feels like the player doesn't have enough interesting choices to make, I'll usually throw in some kind of very
visible randomness (such as the boss randomly transforming into one of three modes) or decision points (such as a set of special items you get for the fight where timing is a major consideration).
4) Flavor
The last thing I like to throw in before putting my design gloves down is some dialogue and visual eventing (via Troop Events in RPG Maker). I have the characters talk to each other (or to the boss if it's sentient), make some jokes about the moves being used, use stuff like screen shakes, screen tone changes, and even subtle music changes to enhance the atmosphere, and take a moment to explain any special mechanics I'm introducing, in as natural a way as possible.
5) Tweaks Based On Playtesting
Months later, when I've had several people playtest my game, I consider their feedback and finalize the boss' design by altering their stats if people found it too easy, too hard, too short, or too long... or changing up the mechanics (in the same way as the above paragraph) if they found the boss to be boring, confusing, or overly gimmicky. After this additional tweaking, I usually have a boss I can be proud of!
The thing I'd recommend that beginners do
differently than I do is that they should
start with the mechanics rather than the large concept. Think of what kind of mechanics would make a boss (any given boss) fun and unique, and think really hard about the way that players would go about beating such a boss. The more ways they have to beat it, the better. If you can only think of one or two ways, develop the mechanic further; maybe add a player-favored wrinkle in that opens up additional strategies. From there, come up with a visual/story boss design that fits the kind of mechanic you have in mind.
You are definitely right to ask about plugins. The default tools in RMMV (haven't played around with the previous engines enough to speak to those) are absolutely not enough to make a sophisticated boss fight. They rely solely on the most basic aspects of random chance, and leave zero room for controlling things in a specific way. Both patterns and actual AI (such as being able to respond to the player's actions) are not possible.
I'll agree that using a bit of code (or plugins) can be useful in creating great boss fights, but it's absolutely
not necessary. With creative use of Troop Eventing, States, and Skills (which either use creative formulas or call Common Events to make interesting stuff happen), combined with smart databasing decisions like setting battlers' Evade to 0% and Hit to 100%, you can create intriguing, unique, fun boss battles in RPG Maker without ever touching a plugin or a line of code. The interface will still be clunky and basic (plugins can help improve that), but the battle itself will be awesome.