I mean, the end result is you'd end up with a map full of one town factions that can get bullied by anyone with enough parties to keep their villages raided and occasionally siege their capital. I'm in favor of more small factions but single town factions are just terrible because of the way the wars play out.
My preference is for Bannerlord to retain some challenge on the campaign map, because frankly, this game is too easy already. Most of the difficulty isn't even interesting, just stuff like snowballing making it almost impossible to win late game wars and even that has been curtailed recently.
I can't speak to what you do in your playthroughs or what you'd consider enough to be an issue, but in mine there were rebellions in about nine different towns over a ten year period.