They (vaguely) tell you at the end why they leave. There are 3 things they say: Unhappy about your choice of companions, your leadership and state of affairs. The main thing you must keep in mind is that they don't get along with certain companions. Certain events affect the state of affairs. Like looting vilages, having no food, losing battles. I'm not sure what affects their approval of your leadership. Losing and winning battles, I guess.
If you don't feel like playing the matchmaker, you can disable this system with the program called TweakMB. Check out the sticky called TweakMB for more info. It allows you to add a dialogue in which you tell them that they can only leave once they die, either in battle or by your hand!

But seriously, they are deserting and stealing your expensive gear in the process, so it's not like they don't deserve such threats... If I could, I would impale one of them to show them what I do with deserters, like Vlad the Impaler did. That would teach them!