I don't know whether this is possible, but gate castles (those castles where there is no siege ladder but only a gate) should be more abundant. They are far more interesting and have fewer pathfinding issues
Well, in all my playthroughs at Brytenwalda I've never been critically hit by a slinger, while bowmen have slayed me a lot times. And I play on 500 battle size, i.e, javelins, arrows and stones flying everywhere.
Are you using lance recruitment? If so, the sense is supposed to doesn't make sense. If you choose native recruitment, you won't have such problems. And the horse lose speed probably because you ****ed it. This is totally correct and accurate to me.