(I haven't played long enough to see this snowballing happen myself, so perhaps the AI already does this)
One mechanic games tend to use is having factions prefer the status quo, or at the very least prefer no one faction become vastly more powerful than everyone else (because that leads to snowballs, which any faction would want to prevent).
Thus, if one faction is gaining too much power, other factions could look to form alliances of mutual defence, or perhaps even aggressively attacking the powerful faction to take advantage of it already being engaged in a war or preempt an attack.
Ideally, this would lead to factions leaders naturally preferring small tactical gains, vs. trying to outright destroy other factions.
Though the fact it seems unrealistic for factions to liberate captured land may make this only a delaying measure (since the alliance created to stop the one faction would itself just become powerful, and the net result may still be factions being destroyed), rather than an outright solution.
Peasant cultural revolts is also an idea although it would take some effort to make revolts work.
I really like the idea of revolts, both lords from a section of a vast empire revolting to create a new faction (or join an existing one), and civilian revolts when they are treated poorly (or were treated well by the previous lord).
It seems like a great way to create new factions to replace destroyed factions and encourage factions not to completely destroy other factions, even if militarily they could.
