Sometimes you just have to drop some of that influence to boost your relations even when it doesn't make sense to do so. I've found that by spending influence on votes, regardless of which lord has the most support, you can balance your relations and maintain high relations with all clans. This is because you gain MUCH more relations when voting for than you lose when voting against.
Granted, in my playthrough the influence is highly unbalanced. Some clans have a great abundance of influence that they will never be able to use all of (over 40k and increasing!) while other clans have less than 500 at all times. This is perpetuated by the policies being passed in the faction that consolidate influence. As it stands, it seems to me that the influence system needs a bit of re-balancing.