it depends on what your intentions about the playing style are.
quick and casual would probably be better with texts, if it is to have a more complex background with immersion or something like that it would be better without texts as the text labels reduce immersion but make things quicker to identify.
In my opinion leave text only on NPC you can interact with, optionaly - I met in some games with term "Point of Interest" where above certain items like chests, rocks or points on map sign "!" was floating.
So I would take following course:
Objects without any marks - no interaction (hidden secret)
Exclamation mark - point of interest - mandatory/optional
Full name over - Interaction with object.
You dont need to text everything, its better to make a combination of and set graphical rules to standarize stuff.
Treasure chests are self explanatory, you dont need a "chest" text over something thats clearly a chest just by looking at it, its redundant and doesnt add anything. By always adding text to interactables and lootables you wont be able to freely add misc and hidden stuff like wallcracks and book cases as you already trained the player to look for the letters or it has no use.
If the "next", "Back, "minigame" and whatever other portals are recurrent and always the same color then you only need to add text to them on the first 1 or 2 maps/zones as you already set a "color means X" rule, the player identifies them by look and once your rules are set text is not really needed.
Text on enemies and npcs works better if they display only when near or targeted, its more useful on npcs as identifying the old wizard as "magic tome seller" is hundreds of times more useful than labeling a slime "slime", unless you label it "Slime lv 12" give useful information.
Dont add tags for the sake of it and dont be redundant.
Finished that egg-sitting mini-game I mentioned here a couple weeks back. Now to fix the weather system so it doesn't rain indoors while you're playing it. >.> A simple fix in itself; leave it to me to take it as an excuse to completely revamp the time/weather/lighting system from scratch.