I would imagine so because one of the factors they keep in mind is "How respected/liked will I be at the new court compared to the old one."
But only a veteran/expert/code-reader could tell you for sure
Lords switch factions if they are not treated well or dont have any fiefs.
I had a lord and I think its Jarl Olaf and he has 2 towns,3 castles and 2 villages and he still quited .
So its kinda random. Sometimes lords dont switch even if they dont have any fiefs. I've made a little test to see if I will get infected a treason. I had -100 relation with the King and he infected me for a treason. You need at least -100 relationship so a lord be indicted for a treason.
Its because he is quarrelsome - lol i had him as my marshall, and then he left when we had defeated the sarranids - then he got kicked out of every other faction and returned crawling to me - i needed him and i knew he was a good fighter and marshall so i accepted again - but ye he is a really bastard and a f*ck*r to deal with