Well, the largest CASTLE I've seen in the vids has got to be the one from the castles blog. That thing was giant.
As for a TOWN, either the Sturgia's coastal scene in the open demos, or Empire pic from above.
We know for a fact that defenders will retreat to the keep if overwhelmed in case of a castle, however there have been times(like that huge castle) where there are two gates, but in the most recent demo, the enemy just retreat all the way to the keep instead of holding the one gate chokepoint.
As for towns, the scenes are huge, and I'm not sure if they will be simulated entirely in game, or will have a linear fashion to them. Still, it would be great to have a checkpoint defense which should work perfectly considering how they organized their town scenes.
The walls - Defenders hold the walls, until one side(likely siege tower) bears down too hard on the troops, so a mass retreat is called and defenders get reinforcements from the castle.
Marketplace - Defenders hold the few chokepoints from the walls to protect the abandoned market place, with more gritty chokepoint battles and few vantage points for archers.
The castle - Defenders retreat to their final defense of the town, inside the castle(not keep).
The keep - Last line of do or die.