Very true. I would like to see more behaviour involved, or make Castles a menace to ignore. They could produce friendly units that wander the local terrain like Manhunters did hunting Looters, but more buffed. Something that makes castles important to take down.