Doing quests will help prosperity, but you'll need to keep doing them to have a real impact. I own Ayyike (Vaegir Village) in my current game, and it started off Very Poor. After completing the Grain quest twice, Cattle quest twice, and Training quest once, it's up to Poor now.
I'm not sure whether buying things from the village help its prosperity or not, but I have bought food, fur, and iron from mine.