It definitely should be a option. Like a message appearing that a siege attack is happening and if you want te be participate as your governor. It is not meant to be realistic, but as the focus in Bannerlord is on the clan rather than only on the main character, for me, this would not be to far fetched. The main fun at the moment is fighting battles. The kingdom management is lacking. It is a known problem that the player barely has an opportunity to participate in an defensive siege. This is being worked on in the new patch, but I would stil not like to sacrifice a % of troops to bolster a garrison. This way you get a better connection to your clan members and you get to fight in more high stakes battles, as a player you normally have the ability to avoid unfavourable confrontations.