I went with assigning one castle and one village to each fief immediately after providing a skeleton garrison, and upgraded each with every upgrade except Stables (because the renown bonus doesn't carry to the king) and our cavalry was pants.
You have to upgrade the fiefs, but your lord will garrison troops. The more money they have, the better they garrison, so the upgrades are well spent. I kept two castles for myself, and assigned nobles to garrison at one until there was many, then had them sent to another to not have all my nobles in one basket. I ended up with like 900 total nobles though, so it was a bit ridiculous in the end.
Later, after the invasion was put down and I decided to take over all of Caladria, I gave additional fiefs to my lords, rather than recruiting new ones.