In my recent playthrough i have a castle "LLanon Henc" (sp?) as the only last standing castle in enemy Battania territory. As many of you said a lot of times they break the siege for no apparent reason. But recently they have attacked my 200 strong garrison with 400-500, and gave up when they lost around 300 troops (I've had about 80 left). I replenish the garrison and this happened a few times. Finally i had enough money and food to wait in the castle and try to get the siege defenses lead by myself!
My thoughts when i actually lead the siege, i get absolutely ANNIHLATED. My archers aren't fully deployed on the caste walls, when i try to control my infantry units all hell breaks loose, the AI runs from important areas on the walls that are being taken, and the units at the gates of the castle aren't very effective! Argh its all so frustrating, i should be able to win a 300 garrison against 600, my troops are top tier with nearly 60-70 Imperial Palatines and good mixes of infantry. I really hope they improve siege AI in an upcoming patch. The IDEA of it looks amazing but execution falls very short.