Well, there are many things broken with the game, but I haven't had any issues with Escort missions. First of all, caravans travel really, really slowly. You must have been overburdened or something. Make sure you dump all the excessive cargo when doing this mission. Also, buy some mules, but just as much as to balance your party number, because if you have more than that, then it goes up into the "herd" category, which further slows you down. You need to have as many mules as needed but not more.
Also, buy some cheap horses for your foot soldiers. They won't fight on horses on the battle map but if you have as many horses as you have a footman in the party it gives you a very good bonus too.
And at last, do have a decent scout in the party (or appoint yourself as a scout). If you do all these Escort The Caravan missions are one of the best in the game. I usually make good money by trading together with my caravan, buying and selling in towns we visit and attackers drop decent loot as well (well, in comparison to usual looters anyway). And the salary isn't bad usually... merchants pay good money per day of travel.