Let me get out my soapbox, now is my time!
I too would like to see some form of cultural change in game. Wholesale changing of an entire town (or village)'s culture seems a bit too much in a few years in the pseudo-early middle age setting of Bannerlord, but a neat solution might be to represent ethnic enclaves in game.
Many cities in history have had ethnic enclaves -- the Venetian quarter in Constantinople, the Jewish ghetto in Rome, the European district in Shanghai. In some cases these quarters even had their own rules administered by a local police of the same ethnicity -- i.e. Venetian arms and laws would apply to Venetians even in Constantinople.
In-game, this could be represented by letting town owners choose to build a cultural district distinct from the settlement culture. E.g. I own Amprela which is Imperial culture, but I choose to build a Vlandian quarter. This could add 1-2 Vlandian notables to the settlement (enabling recruitment of Vlandian troops), and perhaps also add some Vlandian arms and armor to the settlement markets. As a downside, cultural districts should probably decrease settlement loyalty and/or security, to reflect xenophobia and ethnic tension (and for balance reasons, otherwise there's no reason not to build a district everywhere).
In the long term, I wish TW would enable town-specific policies. This would open up e.g. rules about how different culture notables are treated in the town vs. the same culture, tax policies in specific towns, and generally deepen town management to make it more interesting and give players options.