Actually garrison in Warband appears it's all based on wealth.
I put text of Magelord (it is very clever):
It checks daily if they have 2005 wealth or more, and if they do it gives them troops from their reinforcement party (in party_templates) based on kingdom, and it reduces wealth by 400. Then every week they get added wealth. The wealth calculation is 15x prosperity (a number from 0 to 100) + 700, 50% more if it's a city, and then subtract out their troop costs. So a very rich city (100 prosperity) would get ((15x100) +700)*1.5= 3300 wealth, minus all troop costs, per week. A 0 prosperity city would only get 1050. The minimum wealth is 0, so even if they get troop costs over the income they won't go into negatives. So it appears the best way to modify garrisons would be to modify how much wealth places receive. This is in simple_triggers.txt:
We should think how many people want (garrison) in castle or town.
Normally, in native we can meet 300 in towns and 150 in castles if i dont remember bad.
Really, a town of 5,000 inhabitants could keep arms to one tenth of its population (in desperate situations more): 500 men. But during the seventh century in England a few cities can boast of having 5000 inhabitants. I think normally between 2000 and 3000 inhabitants in this time. 200 or 300 men of garrison (or less, maintain an army is expensive).
Castles - we have not really castles in Brytenwalda. If you look at the scenes of Adorno, they are smaller towns or large villages, with a wall. Population would probably be between 500 and 1000 (or 1500 at most) people. Applying the same token, we would have fittings 50 to 100 men of garrison.
In cases of siege, the garrison probably would join part of the population, perhaps doubling their number (although people poorly armed and untrained).