The problem is how both bannerlord and warband expect you to assault every settlement like in total war. In reality a decisive field battle would make some of the losing side's castles surrender immediately. The game needs some system like this to make siege assaults less common and shift the focus onto field battles, so that if siege assaults do happen, the defenders can get way more of an advantage and it won't feel as ridiculous as it does now where every other battle is a siege.
I'm not sure if I like this idea. Field battles definitely need to be made much more significant, but I dislike the idea of not having to siege a local castle just because you won a battle in the field.
Conquest needs to be slowed down, your idea would just help it happen faster.
Lords who are defeated in battles should take much longer to come back with a party. So defeating an army should more or less guarantee you aren't going to see another large army for a while.

