Things i want to see to improve a bandit PTif based on the current bannerlord, I think they can add something called "infamy" which is basically the bandit counterpart of influence. You could spend the infamy to have other bandits to join your army. So by the time you grow stronger, you will constantly leading a larger army but with with lower troop quality and morale. So formation and morale management is now more relevant. And when you have high infamy some bandits will hear your deed, and begin to beg to join you. This now plays way more like a very charismatic legendary bandit leader I suppose.
Also taking over and perhaps improving hideouts is highly needed. Bandit converting to reg troops also needs a rework.
Faster ways to increase your roguery skills
a infamy / dread score should be possible for all characters. this would be used to attract other bandits to follow you. With high infmay/dread Honorable nobles would want nothing to do with you, dishonorable nobles would seek you out to do issues for them.
You should be able to take over hide outs by killing the gang leader in a duel.
You should be able to take over street gangs for passive income (protection money) same as traders use workshops
Now and then you have to defend against other gangs trying to take over, now and then you are challenged to a duel for leadership of the gang.
You should be able to recruit bandits from gang leaders.
Looting a village should be more rewarding
Your hide outs /street gang combo should be able to sell stolen wares.
If you after a long time want to take over cities and start a proper kingdom, then positive relations with nobles is almost impossible. I suggest that a dread score would also work to make nobles join you and stay.