Since the game has considerable amount of physics other than horse collisions, and you run the game at 50fps, it doesnt sound like the CPU is the reason, even if it is a bit slow by modern gaming standards. Either that, or horse collisions are much more complicated than the rest in game.
Im just speculating though.