For dena pirates and Franks, their lair/landings are pretty easy to find. For bandits, I don't really care so much about them. I usually leave them to the patrols and young warriors.
Actually, for the pirates, once you can easily beat them, you can just loot and bury their corpse while the others gang up on you. Dangerous but quick way to get you and your companions your first mails and decent weapons, as well as honour and money. Ah, if you want to do that, I personally suggest that you go buy a two-handed axe. Use it to cleave those bastards in half while they're occupied by your troops.