I had something very similar to this, only on a much larger scale.
I was defending Shariz from about 1500 Sarranids (whilst playing as a Rhodok vasal), but unfortunately there were a few Sarranids hiding behind the downed siege tower and I was unconscious at the time (Diplomacy Battle Continuation ftw!) so I was forced to order my men to retreat.
I then engaged the Sarranids as they fled, but, unbeknown to me Sultan Hakim was riding towards me.
Hakim had been absent in the siege and thus still had his full army...
Needless to say he kicked my ass as my men were mostly unfit for battle, BUT I did manage to salvage the day with a mere 17 troops still standing....
That's when the desert raiders attacked.
It turns out that while I was chasing down the defeated Emir's I had marched dangerously close to a Desert bandit hide out.
So here I was, with 17 battle weary troops and over 75 elite Sarranid prisoners (thanks Tweak MB), facing 50 mounted bandits.
The funny thing was, even though I was defeated by the bandits, in the long run they ended up helping me.
Their army was of a much higher-caliber than the surrounding Sarranids' and they wreaked havoc amongst the Sarranid towns and villages for many months.
The last time I saw that particular group was a week or two ago in game. I hope that they're still out there, kicking Sarranid ass, but I fear they may have met their end at the hands of a Sarranid war party that made their way towards Shariz a few days ago.