Here's my solution.
First, get picture anchors -
http://forums.rpgmakerweb.com/index.php?/topic/34249-picture-anchors
Second, you want to take a picture of your map
Then figure out where your covers should be and draw some rectangles in an Paint or something.
Now you should have your cover as a separate picture saved in the Graphics/Picture folder.
Next, you will start by creating the pictures in-game. Using anchors, you will specify where the pictures should be positioned on the map.
Then you'll set up some player touch events in two places
1. One inside the room
2. One outside the room
The one inside the room will erase the cover for that room, and show the cover for the outside if necessary.
The one outside the room will show the cover for that roomm, and erase the cover for the outside.
When done successfully, you should see something like this (I didn't bother covering up the outside when I entered the room)
The advantage to using this approach is that you can overlay multiple covers on top of each other.
Does your room have another room? No problem, just create a large picture for the bigger room, and then a smaller picture for the room inside, and erase/show the pictures as necessary.
If you need to have a cute kitty pic on your cover, then you can just open your image in your favourite image editor and add a kitty pic.
This is multi-layered mapping with full control over each layer.