I experienced the same thing many times as a vassal for the Aserai. This is mostly just annoying, but can become very problematic if you are waiting for some time in a city/castle/town when the call to join an army comes. You will be shown as still in the city/castle/town, but once an army is created your army actually heads straight to the new army while showing you are still waiting. If you do not immediately leave the place you are in, you will end up across the map sometimes (depending where the army was created) once you do stop waiting.
