Governors grant bonuses to town they are in charge of according to their governor skill perks. every skill tree have a few governor perks, some of them might be particularly meaning according to town in question.
another reason is if the town is not matching your clans culture you will get a substantial loyalty malus. you can have a culture matching character be the governor and negate the loyalty penalty.
We are going over the the skill experience gains in general, thank you for letting us know. We are already making changes so governors will gain various other skills like engineer leadership trade etc albeit in a slower rate than stewardship but, I`ll make sure to go over governor skill experience gains in detail.