One thing to point out about having a bunch of troop giving notables inside castles is that it makes raiding villages less valuable, and makes it harder to hurt the enemy's troop production, since it'll increase the pool of "safe" recruits factions have available. I think it's nice how important villages are for a faction's recruiting capacity right now, but maybe that's just me.
The AI is actually pretty efficient at managing their garrisons. Castles' clan members will stop by to deposit troops (or withdraw them if they are in need) fairly regularly based on a calculated "ideal" garrison size. Any member of the kingdom can also stop by and donate troops if the garrison is too low (doesn't apply for the players fiefs).
On top of that, clan leaders will visit all of their settlements (towns, castles, and villages) based on how long it's been since they last visited them. In certain circumstances though, it can take them a year or two (sometimes more) for them to visit some of their castles and villages if they are too remote. They will still always
eventually visit them though, because the "score" for them to visit the settlement will continuously increase as time passes until it becomes higher than the scores for any other action they can make.
The campaign AI functions as a
uility-based system btw, in case anyone is interested. Every action the AI can take at any one time is assigned a "utility score" based on various factors, and the AI will carry out the action with the highest utility. They evaluate dozens or hundreds of potential actions at regular intervals (such as Raid X village, Visit Y town, or Patrol Z castle, etc).