Ok, so, I'm not sure if this is a bug or supposed to be this way. However, it seems that at virtually every castle I siege with the requirement of building a siege tower, while my 4 to 6 guys are beside the tower, people start dropping from the castle and attack my guys. It has gotten to the point where I'll have like 20 to 30 enemies hovering around the siege engine. This, naturally, makes those battles almost impossible to play. I usually try to defend my 6 guys. But, when I have like 20 enemies running around and a whole bunch of archers shooting at me, then there is no way to finish the siege.
Anyway, I end up finishing the maps using that fun cheat of Crtl+F4. This should not be. Either I use the cheat code or the maps are fixed.
I will try to update the names of the maps where this is happening for sure as I play. I can say that the 2 castles west of Tihr definitely have this issue.
FS
Edit: This thread still stands even in 1.010
http://forums.taleworlds.com/index.php/topic,49179.0.html
Edit: Ribeyet is one of the castles...