Im having a similar problem in smaller scale. I got 160 renown, then joined the Battanians, since they have the Lawspeaker (or lawbringer, can't remember), i lose 1 influence everyday. I don't have any fiefs, and been in countless city and castle sieges, but since i have a -70 influence by now, i can't vote for territory awarding or for law approvements, and if i join the votation without the influence, the only option is to abstain, and lose relations with someone. I can't revert the negative influence, because in battles you only win a single amount, and joining armies only gives +0,37 right now, not even enough to stop the influence loss.