1-Increase, substantially, the incline of the ground when in castle sieges. Castles were meant to be able to be held by far fewer men successfully. The terrain it was selected to be built on, heavily favored the defenders. This made approaching it on foot an extremely difficult task, let alone attempting to bring a siege tower to it.
2-Eliminate siege towers from castle sieges. Castles were often manned by a few dozen men in order to uphold the principal of economy of force. Only during dangerous times was it fully manned. Bringing a ram to its gates was an extreme under taking.
3-Ram speed substantially decrease in castle sieges (the incline of the terrain should simulate this).