Welcome to the club.
TW has been informed of this stuff since almost 2 years now. I do hope they re-vamp and fix the sieges before they remove the ability to solo the garrison as it help deal with the complete **** show many sieges turn out to be.
Also, why don't they sally out when you attack alone, or are you defending in some pics where you don't have retreated party?