From a realism point of view, thrown rocks are actually scary to armored units. Thrown rocks have a ton of raw force behind them, meaning the impact will shock, unlike an arrow which would bounce off to a degree. A rock to the head will vibrate the helmet like a bell, with you inside. If you charge 10 guys throwing rocks while you have armor equipped, you would likely be knocked out or bruised heavily and unlikely to catch and kill them. So in this regard the game is actually fairly realistic. Blunt weapons, such as stones and maces, are the one way to actually defeat heavy armor in a realistic battle.