Yes, we've all seen a clip of glitched out AI. That has been happening for a wile with all unit types where they some times just don't attack. There is a broader issue not pertinent to this debate going on there.
Archers clearly have a fire delay to prevent them from being perfect shots with no delay between loading and firing, and the players were gaming that by turning the archers. But think about what happens if we remove that, archers essentially would have machine guns. Perhaps a change could be made that if they have been holding for a certain time they fire but that opens new ways the cheese the AI.
Never the less what I see in this clip is an inexperienced captain's reaction. They don't control their units fire to concentrate an attack. They single themselves out as a target by not acting in formation. They wait until their own line has been agroed and broken before charging. The only grey area in their play was waiting until the courser was stopped long enough for the xbowmen to shoot them down.
Finally, we come to the question of effectiveness. We can establish that the xbowmen did not
essentially lose most of his men
only one is seen dying in the clip but we can assume an additional one fell before the end of the engagement to be generous. Considering most cav units have around 8 troops and 2 fought here, that would make 1/4 of a unit coming down to bear on the archers. The archers had 12 troops and loss only 2 men. In other words 1/6 of the archers were lost. The archers would continue this trend until the cav is wiped.
You have suggested that a change needs to be made to make the rambo strategy ineffective or unusable. but it already is ineffective against decent players. A captain is meant to command their units in the most effective manner. Usually charging headlong with your cav is not the most effective way to play. Players quickly learned this and have been trying different forms of play.
I understand where your coming from with wanting the captain to stay with the unit. There are plenty of other games out there that operate that way and do so well. I'm just asking you to consider its implications on the broader scope of game play. The game mode is more unique with the ability to split from your units and have them fulfilling other objectives.