Yeah, I'm having issue with this as well...it's the only reason why I can't beat the kingdoms.
Even when we win most of the battles and conquer most of the cities/castles of the enemy kingdom, someone in the guild just straight up leaves, and sometimes with the new fief they've received.