Merentha said:
You do realize that would mean you'd never, ever be able to take a castle, right?
As phbbbt107 said there should be other ways to capture a castle, direct assault should always be last resort since it should be costliest method - I shouldn't be able to storm fully garrisoned castle even with my Level 38 character which can hack all defenders in two or three attempts.
As it is know it way to simple to conquer a castle and almost impossible to hold it since you can expect entire enemy army to attempt to retake it, medieval warfare was all about long sieges:
http://en.wikipedia.org/wiki/Siege#Greco-Roman_and_medieval_siege_warfare
http://en.wikipedia.org/wiki/Siege_of_the_Sogdian_Rock
So we could have capture owner of the castle and using him to enter and launch surprise attack, choosing to scale the wall without any armor eliminate guards and open the gate, bribing defenders etc.
Also size of the garrison should have impact on Castle stores, so large garrison should have food for few days only but can easily repulse any attack while small garrison can last for months but could be worn down after few assaults.