Summary: A friendly caravan got attacked so I moved in in time to help. I have the quests active to train borrowed troops and to destroy bandit parties, if that has something to do with that, strangely the looter parties weren't marked by exclamation marks. As soon as all looters started to flee...