I would be running like there is no tomorrow seeing a 50+ men lord coming to my village if im poaching.
Make them run away to a hideout %80 of the time (within a radius of the village) if the player group is big enough
You can either question villagers to learn their location or search them...
Their military strenth is realistic in my opinion.
They can be nerfed politically by not giving them some policies like Magistrates, Senate (suggest more below)
Also nerfing bow and arrow slightly could help balancing khuzait
I know they dont want to change cultures of villages completely that's already been discussed.
But
A policy that decreases town loyalty 0.4 per day to bring some of your own people to the newly conquered lands would be nice
One npc per Town and one for some villages (replacing already existing...