I am having the same issue. They seem to chase enemy armies specifically, even if I am not nearby. So like a 50 man force moves to engage a 600 man army like a full day away from me. It feels like if they are moving to come into your army and an army comes into view, they move to engage it regardless of how far away backup is.