Charge not only affects the amount of damage you deal when you slam into them, but the chance that they will be knocked out of the way. If you have a low charge, and aren't moving very fast when you hit someone, your horse will rear and you'll be cut down in short order. Speed also has to do with this charge power. If you had a high charge, but are moving slow (maybe you're going up a hill when you run over someone) you will still rear rather than trample them. In short, you'd better have a warhorse or charger (preferably spirited) if you expect to plow through enemies.