We're by far the strongest faction, so that shouldn't be a problem. Currently we (Strugira) occupy about 1/4 of the map.
Vlandia is the main problem, but they're almost completely dead. They have 0 fiefs, 1 clan left (kings clan).
I recruited Vlandia's last 3 lords, each received multiple castles, but they keep defecting back to Vlandia. Some over and over.
Yesterday I spent a few hours taking 5 castles for Strugira. Then as I was about to save and logoff 2 lords defected. One back to Vlandia and the other to Britannia and we ended up losing 6 castles.
Britannia is not as dead as Vlandia, but they're also half dead. They had 2 fiefs left, then 5 after the defect.
I'm considering executing Vlandia's king. Maybe the new king won't have good relations with those lords and they'll stop defecting back.