On the contrary, I think it's a waste of time to focus too much on the prosperity of your village. Your village is usually the first one to get raided when your faction is at war. In fact, lords will cross through enemy-infested territory at great personal risk just to loot your paltry village.
Your best bet is to do some trading, raiding, bandit-killing, and/or betting and rack up some money to buy an ironworks in Curaw. Even with the nerfs to trading that the last several WB versions suffered, that's a safe bet for your first enterprise. Costs 3500 denars, gives enough income to support a modest group of troops, and you get 3250 denars back when you close it (probably to open up a dyeworks), making it extremely cost-effective. Your next best bet is to continue with the trading, raiding, etc., and build a dyeworks in Rivacheg. After that, the money will come rolling in to the point where you can build enterprises in most towns. That is a much better source of income than a village.
Honestly, I think villages are only useful to give to bad-personality vassals when you have your own kingdom. Yeah, in the beginning you should keep bandits away from it, if only because that also gives you a chance to fight and level up. Otherwise, you really shouldn't sweat it.