I will just paste here the suggestion I made a while ago about the same topic. Diverse armies bothers me because when I want to play as a Khuzait horse archer I end up having imperials on my army because I cant afford going back to recruit Khuzait troops, so it becomes pointless. Same goes for playing as an imperial, I end up recruiting Sturgian or Khuzait troops.
So upon conquering a town or a castle we should be given 3 options;
1- "Force your culture on the people" in result of this action we should be able to recruit our faction's troops after sometime like 2 weeks ingame or more, you pick.
We should also get relations hit with the notables of the opposite faction and maybe even decrease in population and production as it will create unrest among the citizens.
Also a high rebellion chance should be present.
2- "Make the population slowly adopt your culture" Similar yet slower version of the above, less hit with relations, more time until we recruit desired troops.
3- "Let the inhabitants keep their cultural traits" Nothing changes, and player/npc can continue to recruit the foreign troops if he/she wants to use them as auxiliary troops.
Or another option could be to take a similar approach to the matter by following total war's religion system.
-Building cultural buildings should change the culture slowly.
-Stationing the desired culture's troops should effect the culture of the town/castle after sometime. Especially if they outnumber the locals.