I noticed this happened when I was NOT in the town when the siege started. I didnt want to sacrifice units to break into the town, so I attacked the siege camp, but ended up spawning in town with the garrsion and a few other parties, without siege weapons, and the ai was not attacking gates, nor did they have siege weapons.