I don't think experience with playing RPG Maker built RPGs is critical. However, I believe strongly that a good RPG developer must have experience playing RPGs in general --- so I feel that is critical.
In my case, I've played through a variety of commercial JRPGs. This gave me a crucial understanding of how a good RPG is put together --- how much should NPCs say, how to pace the plot, what does a good town layout look like, how to pace experience, what abilities to give classes, etc.
Now, my games are original worlds and concepts, so I'm not slavishly copying these games. But, playing them gave me good experience at seeing HOW to put a good RPG together from the many pieces you can use in RPG Maker. You learn, as a player, what features you like and don't like to see (I hate having to craft every piece of armor for example), and that improves the quality of the games you make.
It's sort of like how a really good writer is also usually an avid reader. The writer learns what s/he likes, doesn't like, what works, doesn't work, by going through examples. S/he learns the classic tropes, so when the writer deliberately diverges or subverts the tropes, it is a conscious decision and usually done very well.
And, then if you want your RPG to diverge in a wildly new direction, you have the solid foundation of RPG play experience to know what would be fun to play, not just develop. Too many new RPG Maker developers fall prey to "shiny toy" syndrome and toss every cool feature in, resulting in a chaotic mess of a game with a ton of half-baked features. Good play experience will quickly show the developer how silly that is and why even ambitious open world games like Skyrim don't implement every possible feature.
I also agree with Matseb2611 and Dalph though. What we all as developers need more than anything is playtesters, to see what we did right and what we need to improve. And if none of us play these games, we'll all be contributing to lower quality RPG Maker games. Besides, playing another game (if you like the plot idea, etc) is a nice break from creating your own game.