I would say it depends on what sort of fantasy setting and what sort of gaming experience is desired.
If a "low fantasy" or "realistic fiction" experience is what is sought, then the game should adhere to the physics and other natural laws of the real world as much as possible. This seems to be what Bannerlord is intended to be, so in that case I agree with you that cavalry should not be able to ignore the density of infantry formations -- in order to give a more realistic feel, and to allow for better suspension of disbelief/immersion. To ignore these things would mean that the world Bannerlord takes place in does not obey the same laws of physics etc. that we do on earth ... in which case there should be more differences, and if it is good fiction, there will be a plausible/setting consistent explanation as to why those laws don't apply.
In fantasy fiction, any rules can be broken. The question is, should they? And the answer to that question will depend on what sort of world you want to story to take place in and why you want to break that rule. Plus how reasonable is the explanation for why that law of nature doesn't apply. That's not a matter of whether it's fantasy or not, in my view, but whether it's good fantasy and how immersive it can be (for my money, chopping down a castle gate with an axe with no reasonable explanation -- like a super powerful magic axe that is consistent with the world's magic levels -- ruins my ability to suspend disbelief and smacks of poor or lazy storytelling).
All that said, this game in in early release/beta format, so some things may not be realistic because the kinks haven't been worked out yet. So I am NOT levelling any criticism at the game itself right now ... just discussing the philosophical point put forward.