I know the Mack and XP sprites are 1.5 times taller than the VX Ace RTP chibi style sprites (48 vs 32 pixels). I think there are differences in color palette and lighting between the styles as well. Lighting sounds strange for a 32x32 pixel sprite, but if you look closely, the sprites are carefully "shaded" using related colors, to give the illusion they are being lit from a particular angle.
The real deciding factor is the tileset you choose to use. If you're using the stock RTP, stick to stock RTP style sprites. If you're using the Mack tileset, you need to use Mack style sprites.
While many of the tileset packs sold are compatible with RTP sprites, some like the Pixel Myth Germania one, are not.
It's important to match your sprite and tileset styles, or things will look strangely "off" in your game. I think Mack and XP style sprites also use a different lighting style than the Chibi style (VX Ace) sprites. So you can't just put in Mack sprites and say "These are Giants in my world." It'll still look off.
I don't know all of the specifics either, but the best advice is always to match the tileset style and the sprite style. VX Ace and its built-in Character Generator create VX Ace style Chibi sprites. Next best is to match the tileset style to a sprite style the tileset author says is compatible.
Chibi style is a synonym for VX Ace RTP style, and it is a fairly popular sprite type, despite the sprite's rather odd appearance (if adjusted to life size, a chibi sprite would appear to have a head 3 feet tall!). Most resources are compatible with it, and VX Ace comes with a slew of default Chibi sprites for NPCs --- monsters, "injured" NPCs and the likes.