It's the number one annoyance, but I wouldn't call it the number one problem. The whole system is clunky and full of exploits :
- Unlocks are random and dependent on xp gained. It's probably impossible to get them all.
- Refining is generally of little use, since you can get metal from drops and bought underpriced weapons. If you try to use it, you get out of hardwood fast and you have to spend hours watching the "waiting" screen.
- XP is based on item price, itself based on DPS.
- If you get some high-level unlock and you can get your hands on the metal, you can craft it for tons of xp and gold, even if you lack 100+ points of skill...
- You can also use it to powerlevel your companions.
- Some high-end items (Pugios, Southern throwing daggers) are sold FAR below material cost. While anything low-tier that does high cut damage is highly priced.
- You can get rich as hell just converting such things.
- In the end, the weapons you turn out are not really that much better, even with perks...
- Bonuses/maluses are not frankly significant.
In the end, the smithing system mostly offers two things : Some customization of looks and a money exploit. The forced waiting is like the cherry on top...