My anecdote, is that if you build a kingdom via the main quest, the conspiracy quest, and its growing strength, will continuously increase the likelihood of all other factions declaring war on you. At a late-game stage, if you experiment with save scumming, you will see that every day, one of the kingdoms that's not currently at war with you, WILL declare war on you. If you pass the day without war, the next day, again, someone is declaring war on you.
Currently, I've begun experimenting with just ignoring the main quest and manually building a kingdom in 1.4.1. Theoretically, this should be much more playable in the late-game stages.