There's a lot more that goes into map speed. Namely party size, morale, inventory weight/number of pack animals, and the terrain/time-of-day. Base speed is calculated as the average of (overall average unit speed) and (speed of slowest unit). Individual unit speeds are determined by Riding skill (mounted) or Athletics skill (dismounted).
So, it stands to reason that a smaller force of higher-level units (with more points in those stats) is the best escort for a merchant player. I usually start my games by killing bandits and such with peasant armies to raise a decent amount of capital, then firing everyone, hiring a small force of mounted mercenaries, and trading around the map until I've amassed a fortune.
I recommend a small team of Mercenary Cavalry or Caravan Guards, a few cheap pack horses (shop around as you travel and buy the least-expensive ones) and a few mounted hero companions. This will give you enough speed to trade unmolested and enough fight to take on bandits occasionally to bolster morale or train up companions/make CGs into MC.