Idk if you ever played a modded version of Warband, but if you did then you know why people play Bannerlord. There should be so much more, but why should i want any castles in this game? Every faction could be at war with each other, the moment you the players gets a fief they gang up on you, their entire armies drag their balls through broken glass across the entire globe just to attack your village.
Owning a town in this game doesn't give you any real benefit. Neither does a castle. But it could be on me that i am so disappointed, because i still remember them talking about how you can build your own castles and towns (as in you can customize them and it reflects on the world map) but the harsh truth is, you just pay for a text that increases in +5% more food or w/e. And it just makes me sad to see that all this time in development actually meant nothing for the game.
And i hardly doubt that those things will be modable. Since there is no foundation to build upon.