It happens over time. they'll occasionally recruit from their prisoners, and then those prisoners upgrade over time, and if the stars align just right, they'll survive long enough to build up into a monster collection of units.
I know roughly why it happens. As you go up in level, the sizes that bandit armies can be also increases, and if they've survived a long time, they'll start recruiting prisoners to fill up their ranks up to that limit. Where as entirely new spawns are simply given the appropriate types of bandits, and will only occasionally begin recruiting if they lose a bunch of units and aren't completely wiped out (which is usually what happens).
It's not that rare to see bandits that have a few non-bandit units, but it is rare for them to survive long enough to turn into massive armies like those.
And you can see by the caravan guards, that they've clearly been getting prisoners from caravans, which also include some faction troops.
Also...bandits have no where to drop off prisoners (they don't seem to drop them off at bandit camps), so they keep all their prisoners they've ever captured until killed or they recruit them all.