You can always edit the bandit troops (for what equipment they have) or their party templates (for how many can be in one group) with Morgh's Tools, if it's bothering you enough you want to change it rather than offer a balance suggestion.
But as others have said, game start is dangerous, you should be able to outrun folks when it's just you on a horse, etc, etc.