Massive bandit armies depend on luck: as Piggyboi says, it depends on your level, but to see such large groups of bandits, one starting group has to defeat an army with a lot of prisoners, adding them to the victors' army and this has to keep happening - preferably against Manhunters who only knock enemies out. If you check, you will often see that the biggest bandit groups are made of more than one type - that's because they have beaten several groups of manhunters or lords and liberated their prisoners, increasing in size. It would need quite a lot of victories like that to get up to 100.