Not the only reason but still. Try attacking a caravan with mostly infantry, even though the enemy archer formation is nearby your infantry will still prioritize the skirmishing/charging cav when set to charge (because they are momentarily closer I guess).
Result is a slow and frustrating loss...