All hideouts are like this currently. My ONLY solution to the forest guys was to wait around until all/most of their dudes left and then attack, but that could take up to a week as they rarely leave once they're there.
IMO the more they have in their hideout the more you should be able to take, plus perhaps the ability to pick the party beforehand (so we can stop bringing in heroes into high risk matches)