You're still limited by your garrison in the castle/city and the militia. Which is still heavily limited by food consumption. Which is even more limited in war as it's almost impossible to stop the constant raids of your villages unless you execute everyone.
So when the enemy army of 1,000 to 2,000 is sieging you would get to play your wife/husband, or a companion with 90 to 150 weapon skill. But you don't get reinforcements.
An AI change would help. But as it is now. I've only ever been able to participate in 1 defensive siege in 170 hours played as attacking armies always cancel the siege when I show up with 100 to 200 troops, even though they still heavily outnumber my castle defense.
I have to do the special single battle siege simulator to even have any fun doing a defensive siege.
And from what I've seen, letting the AI do your troop layout and not messing with it seems to be the best way to handle sieges, as once you break your troops from their AI castle defense scripts by giving them orders they get stupid and your defense falls apart. All I can really do is run around solo, trying to kill dozens if I don't want to botch a castle defense. I've tried assigning archer or crossbowmen units to the tops of large towers, and of 20 units, the 3 closest to the edge of the tower where the enemy is will shoot, and the other 17 will stand around doing nothing it seems. I feel like without the ability to individually place units by murder-holes in my castle, I'm just messing things up in comparison to auto-placement by the AI.
I'd settle for a militia commander so I could at least participate in the sieges. Or even a random soldier who can't give orders. But a companion working as governor makes more sense.
Once they fix the pathing with the 3 ladder siege towers it really shouldn't be an issue I'd think on the "being too easy" front. But as it is now, there's virtually no benefit to assigning a governor to a city/castle when it comes to defense. The simulated AI battle calculator seems to simply look at number of and level of troops and just doles out casualties. Even when it's going to be an easy victory I usually start battle and tell all my troops to charge and go afk for 5 minutes because otherwise the battle simulator/calculator will kill a few of my lv26 or 31 troops while I won't lose any (or far fewer) in a real battle.
I guess if you're going to use cheese tactics like bringing a bunch of shield infantry to the top of the stairs on your highest tower, and stand in the back with a long 2-hander slashing everyone coming up the stairs, then yeah it would make the defense trivial. But I don't think that's how most people play. Although, they could make AI changes against cheese tactics like that too where the NPCs just take your keep. And unless it's fixed now, crossbowmen with ammo can still fire up through the floor and kill you.
Heck, even the option to simply watch the castle siege while my Governor, operating on AI, commands the troops and fights would be better than it is now. Not even getting to see the battle much less participate just feels less interactive.
And since perks have no effect in simulated battles, the option to at least watch the two AIs duke it out in a castle/city defense would at least make my governor's combat perks, which affect all their troops, matter.