I came upon this as I happily battled for Sungetshe castle once again. The castle in the middle of the mountain pass between the Veagirs and the Khergit. I always liked owning this castle because it seemed very strategic. Then I noticed it doesn't matter, because everyone can just ride on by.
Would it be possible to make the trigger area of castle larger, and react automatically like when enemy party manages to catch up to you? This way just getting close to the castle triggers dialog, and you get some options (not to same with every castle obviously) - like, if you're more or less neutral toward them - pay a toll to get passed. If you're their staunch ally you may just pass. And if you're an enemy, you get a choice - go back to the map (at the same side of the castle), or attack it.
Preferably of course, this system should also somewhat work on AI parties. So if a caravan wants to get past, it would have to pay a toll (which if you own the castle become part of your income), allies parties are allowed to pass, and enemy parties, like bandits and countries you're at war with, are rebuffed - they'd have to conquer the castle to get past.
Or maybe destroy it? Of course, this would require a dynamic changing of the map - only a ruin remains left in the place of the castle. Which, of course, would also open up to raising villages and towns and remove them from the map - or reduce a town to a village. And of course the opposite: the building of new castles, and founding of new villages and towns.
This way, owning the right castles, can become strategically very important.