Hideouts are easily one of the most frustrating mechanics on the campaign map. You have a large army, capable of conquering ones larger than it with a line of shields, spears, swords, arrows, crossbows, etc...
When you go into a cave or clearing, it's a roulette whether you'll get those...