If u want a big castle u should try to conquer grunwalder castle. I conquered it back for King Harlaus and its the biggest castle i have seen so far and i have seen almost all of them. As for me, next time that castle gets besieged i wont help my beloved King and instead will take it for myself har har.
One more thing about castles i noticed is also that there are differences in how well u can defend them. Theres some castles out there which are basically death traps. I learned that the hard way when i tried to defend dramurg castle,one of those wooden castles in vaegir territory against the nords. It doesnt even have a protection from arrows on the walls and the entry from the ladder to the castle wall is as broad as the horizon is far. So its difficult to prevent the attackers from gaining ground.
On the other hand theres some castles which have such a nice architecture that u can basically just stand there and pretty much solo all the attackers which come close to the castle walls.
In this respect the size and looks of a castle are unimportant but just the way it gets seiged which as far as i know always stays the same and is different for each castle.