Good suggestion. Although rather than finding them in tavern etc casually, I would suggest to have an extra line in tavern window for us to pick, something like "Entertain your soldiers - XXXX gold per night" where gold is calculated based on your army count. So that this can both impact their XP and morale.
Seeing them doing daily activities would be more immersive to see if we would have a camping option like in Warband. And there we could perhaps play games or drinking with them etc.