Mixing culture in party or should lower the party moral and possibly result in revolt (in settlement or castle). That could help prevent snowballing campaign where a strong kingdom can wipe de map quickly.
It would add some challenge to in my opinion.
When a kingdom capture a village from an other culture, It should change over time to match the culture of the current owner. Over time or maybe via a ''cultural project'' in the manage town section (or manage castle).