I gave up on that quest after studying the maps. Then I was out wandering around on other business, and suddenly something clicked. Checked the map, and yeah, it looked the same. Scoured around, and there it was, the hidden cave!!! It was a damn good feeling, let me tell you.
The obvious thing is that you need to look in forested areas, as he was hunting (and it is safe to assume he was not hunting gophers), and, I assume, from finding him 3 times in different games, not too close to any towns or villages. You see, the thing with hidden caves is that they are hidden.