The Empire is fine, it reminds me of Ancient Rome and its factions - there is clearly politic intrigue in the quest and I like it.
Simply put, the North, West and East Empire were once all one - how could you possibly explain one of them having a completely different culture from the others unless they were like the other nations in Bannerlord? So it makes it pointless, because they would just be like one of the other factions.
"Empires should also be more aggressive to each other rather to other cultures, because their conflict is not resolved yet. This will make them more vulnerable for the "barbarian" kingdoms around them, which will balance Empire's possibility to call to arms strong unique units of all types".
Love this idea, and they don't do it often enough.