I've had some thoughts about this and I think a cool way to help alleviate this issue (not totally fix it, but a way to partly address it while fleshing out another aspect of recruiting) is to break down the recruiting possible in towns. Basically right now it's a single option to get a couple of mercenary troops in the Tavern. That isn't very dynamic or fun anymore.
My idea is to add 3 types of recruiting options. These all will take a minimum amount of time, but will yield different better options than the current system. Also, this new way will allow the outcomes to be tied into different game systems like reputation, clan ranking, kingdom status in a much more immersive and rewarding way.
1. "Round up local peasants who want to join your army". Takes 2-12 hours. Based on things like your reputation in the town and in the connected villages, a certain amount of kingdom specific low tier troops will offer to join you. Perhaps anything from 5-30 troops?
2. "Find any willing professional soldiers in town". Takes 12-24 hours. Based on things like your influence, clan rank, and kingdom status, a certain amount of kingdom specific high tier troops will offer to join you. Perhaps 5-15?
3. "Travel the taverns to find the best mercenaries up for hire." This one will be the one that involves mercenaries. This one can be more static to as it represents the improved version of what is already in the game. So like 8-12 hours, 8-12 mercenaries?
I just feel like that would be a ton more fun and dynamic way to handle things. Not only does this open up reasonable ways to expand your army faster, you get to pick the exact way you want, and things like reputation, influence, kingdom status can all play a part in what you get offered which will make the world feel more alive as well.