As King, my clan member want war all the time, after a while I went against them because they never did anything, and when in a war they never defend their cities or castles, so we lost ground, I had to run around and recapture all the lost cities and castles, however as soon as they got them back they did nothing to defend it.
My suggestion would be.
1. The castle come under the Kingdom control, the King can appoint a jarl to be the castle Lord however the control to ramp up the defenses should be in control of the king.
2. If a clan keeps losing a city/castle it come under the kings control.
3. The King can give away the city/Castle to whoever he wants as long as it's under his control.