Given that a mine does not run itself, it woul e logical that mines are tied to villages. Some villages get fields as a sorce of goods, others get mines.
Also, al these raw materials need to be processed somewhere.
I don't think there's much point in plundering the mine though, what would you do with half a ton of unprocessed iron ore? I'd rather hunt the caravans with the final product.