I honestly don't get your problems with large armies.
The game can only spawn a limited amount of units per reinforcement, if you use the right strategy and keep a ratio of 5 kills per unit dead
you can win any battle.
Use formations, use lure tactics, flank fake attacks, hit and run...
I personally use a wall of upgraded spearmen with a wall of upgraded archers behind, and a little pack of mounted guards to raid enemy archers.
It's all about micromanagement and field tactics... My only problem is shielded infantry but i've learnt to expose their flank with my little band
while archers massacre them from the flanks.
My best victory is 158 vs 851 so far.
When they come in thousands the best thing is to cause a lot of mess in their home, most times they will quit the siege and come back.
If things go bad well, you've got one of their castles in the while so...
It also helps to turn all your NPCs to lords then call them to follow you. They have 50-80 units each, if you get 5-6 lords you can count on 600 units.