I figured out most of the problem.
After managing to finish off the Nords and Vaegirs and turn to finish off the Sarannids, I noticed that whenever I assigned a fief I'd lose relations with only ten or so lords out of my forty.
I looked into it a bit more and I discovered that it seemed my relations had locked with my older vassals, from the grumbling/suspicious ones to the devoted.
I'm guessing that Diplomacy is causing this, and that as a result those vassals I said were devoted had been with the ruler long enough to lock relations, and that the trigger for them to degrade relations again would require you to take their faction's last fief.
I'm not sure if this is WAD or a bug, and if this relation lock would be triggered by certain domestic policies or that the game just isn't showing the relation updates for whatever reason.