I ran into the same issue, but further down the "boot" toward Tarentum. There was a bandit lair, and upwards of a dozen Sea Raider parties at any given moment, ranging in size from a handful up to over 40 per party. Caravans get crushed regularly by swarms of them closing in from all sides, and most of the bandit groups have more captives than members. There's one patrol covering the area, which has grown to around 350 troops by collecting the captives, but it's WAY too slow to catch the bandits unless they're in combat against a caravan.
I raided the bandit lair and destroyed it twice, but it just respawned about twice its own width away each time.
If I need to fill up the roster in a hurry with semi-trained troops, they're a steady supply due to all of the captives in tow from the attacked caravans, but I'm currently near my force limit of around 40 men.
The bigger issue is that I tried joining a faction, and got called to follow the faction's marshal, but he just chased back and forth about half a day's march north and then south again for a week or so. After about the 10th-12th time back and forth, it started getting REALLY annoying. Now I've been called up for the second time, and it's the same thing again: race up and down Italy repeatedly, with nothing to show for it while my supplies run low.
