Try making a record of which distances, you get shot at.
There are a lot of things to consider here, but in a vast majority of cases, you're shot at from a distance where the arrow would indeed, duly protect leather, mail, and even scale. High-grade lamellar might endure, but with multiple shots enough damage done to the armor would gradually increase the chances of a decisive penetration.
Another thing to consider, is the amount of armored troops we have in the game. It's rare for the AI, but a player can basically horde a 300-man army armed to the teeth with very high-level armor if one chooses to do so. Filling up your entire ranks with max-tier Imperial legionaries isn't too difficult. So, do we just allow that in the game as a 300-man troop impervious to any normal arrow fire?
AS MUCH as the arrows do unrealistically damage in many cases, it's done in relation to balance. If you want realistic chances of armor blocking arrows even in closer ranges, as well realistic level of equipment given to most archers, then at the same time the game would have to implement some sort of "realistic" balancers -- the most prevalent, would be a need to limit the number of certain high-tier troops from your party. This means your favorite compositions of stuff like 150 Fians or 200 legionaries, would be made impossible by some sort of restriction (unrealistic amount of people with super-high level weapons and armor gathered).
Do you really want that? Think carefully.