@Yozzer: Did you wait for a week? Seems like, with the villages, that you need the population to be assigned to specific tasks. And then when a trigger fires, looks to me like it might be weekly, you'll get one of each building supply per villager you have assigned (i.e. 1 timber per woodcutter) and two food for each farmer. It also looks like each villager consumes one food each time the trigger is fired as well. Don't quote me on it, though.
How it has worked for me is that, after a while of my villagers being assigned to tasks, my town will have a supply of wood, stone, and tools. I can then, at the drafting table, then pick an option to have my villagers haul supplies to the construction site. When I've filled the requirements for the specific project, an option to complete construction shows up. After I hit that, it seems I have to back up out of the area, then it spawns the building. The only issue for me is that my game will crash when I attempt to complete construction on the towers.
@Plaidrab and @ArtistX: I don't get that dialogue at the drafting table for any improvement at the village. I do remember one drafting table that I could interact with at the village that said something along the lines of "You need 10,000 Florins to upgrade this production building. Each level makes it produce 1 unit extra." It was under some sort of overhang, but I could not figure out what the table improved.
Try this: If you've initiated construction with the Village Elder on some improvement, walk around the village and see if you can find a different crafting table. I spawned right next to the crafting table for the towers and for the walls. The others were hanging around the village in various locations, generally around something that looked like the frame of whatever was being constructed.