I don't know, defending besieged castle is kind of fun
at least IMO it's more fun to do than what you did. I had it happen to me only once through my playthrough though so far, however I plan on leaving significant number of my troops in nearby garrison to not scare my enemy and do it again, as it's a lot of fun defending your castle with bow and arrows and smashing your enemies down the ladder they just climbed.