Basically, this thread already addressed the issue, but let me sum it up:
Don't make equipment linearly progressive, if you want it to remain useful. SNES rpgs are the most guilty of that.
Why have 10 different kinds of swords if all they do is deal slightly more damage, and cost double each other's costs?
Now, having completely non-linear equipment might be a bit crazy, though. But certainly doable. However, I see no problem with starting with some lowly rags that are outclassed by everything in the game.
All them gave already very interesting examples, but let me spice up some more ideas: MAge-type weapons are divided between rods (Increase MAT), staffs (Increase MP) and Blunt (Higher damage, increases RES). This already makes mage-type weapons much more interesting and variable by itself. Now, try to work on particular benefits to specific weapons inside of that range. Maybe there's a staff that's weaker overall but regenerates mana. Conversely, maybe a rod drains mana per turn. Maybe you have a Fire rod that's not much stronger than all the others, but increases damage done by fire spells... And decreases resistance to said element, by the wearer. Maybe there's a staff that adds a Fire Resistance debuff when it connects! A mace that reduces defense when it hits! Things like that. none of them are directly stronger than eachother, but nonetheless they play differently enough for the player to have to make a choice, instead of selling all equipment and buying new ones.
But it's a wasted effort if you do this and then all your enemies are carbon copies of eachother, save for more HP, Strength, Defense and Magic.