There are a lots off things that must be adjusted / added to the relations system. For example it does not improve when you save a lord from a losing battle as it was in Warband or when you take a town / fortress.
And for adjustments : in my main game my clan was fighting for Sturgia and as soon as the town of lord Olaf is under siege he switches sides moments before my army was ready to break the siege. So now 1 city and 2 fortresses are to the enemy side + one of the larger clans we had.
So naturally, as soon as i catch Olaf i take the traitors head, but surprise no congratulations from my fellow lords, instead all of them are angry at me and i mean very angry (relations -100 with all lords of Sturgia, we are enemies no). Olaf still was there dear friend, he didn't get any personality for switching sides.