Gold itself has almost no intrinsic value for ordinary villagers because it's useless for normal transactions. Can you imagine trying to pay for a beer with gold coins? Or even bronze?
If the idea of barter exists and an object such as gold of value can be bartered more easily than having to give someone a very specific amount of cows for a certain product. Gold being the alternative as an object of value can be easily exchange with the next potential barterer who will exchange it. That is how the idea of money or currency begins.
All archaeological evidence suggests that the advent of coinage is the result of large scale, long distance markets, or the markets that form around armies. It has to be maintained with cash taxation otherwise there is no incentive to use it.
To modern people in developed capitalist countries it can often seem like money has some metaphysical use-value that exists outside of the state or society, but that's because we've been essentially forced to use it for so long that our society and culture has formed around it, and this ends up impacting how video games depict money, but it wasn't really the case prior to the 1500s.
Bannerlord depicts all of these, especially in the role of the player therein. That's where I justify it. To be more specific, I said if Bannerlord went into the path of Politics and Statesman simulator rather than combat social-ladder climbing simulator... It is most definitely not a peasant working for his daily bread simulator.
Think about it, it really doesn't make any sense that a state would mint coins, dole them out, and then periodically demand a fraction of them back if it was just for the intrinsic value of the gold. The point of monetary taxes is to force people to use the currency you've just invented, and control the amount that people work. This is why taxes caused riots in premodernity. When most people are subsistence farmers who typically don't interact with markets much, having to pay a set amount every year (in a currency you can only obtain via markets), is an act of indirect violence, and a more effective form of coercion than just taking taxes in kind or in labour (which is how most peasantry was exploited in medieval europe).
This can often seem a bit counter-intuitive, for example the chapter I linked mentions how in early medieval europe everything had a set monetary value, from crimes to body parts, but these were abstract guides used to resolve disputes when there was no other alternative, and in these cases actual money rarely came into the equation. For the most part (and anyone who has lived in a village will have experienced this to some extent), transactions in premodernity were tied up in social bonds and cultural norms. Straight barter was practically unheard of, most exchange was a back-and-forth of informal gifting and uneven debts, and the values used in disputes were not referenced by this kind of socio-cultural exchange.
I rambled a bit there but my point is that no game really depicts this kind of thing. Bannerlord exists in a world that is more capitalist than modernity, let alone the period it's supposed to depict, and it makes money the centre of everything. I think this is the source of most of the problems in the campaign, and if they made it more realistic in this regard it would actually be more fun to play.
Regardless of the history or what happened back then, games still have to be games. If we were to put the most historically accurate asinine bull**** then it wouldn't be fun having to pour over broken sticks to figure out tax records. By game design, I think it's the best balance as it offers dynamic emergent gameplay and makes your kingdoms and economy as an actual thing that has to be cared for in order to maintain your army. I do admit that this method would result in much fewer battles and more waiting as large armies cannot be supported without a very strong economic base, but I'm hoping the diversity of politics and economics that came along with it would fill in the dead air time. However, if you wish to talk history, very well. Know that I'm in agreement on the specifics but I do not agree that it was all violence from one group to another.
Yes, it's an easily abusable system, but there were many other legitimates as well who came together not by force of arms but originally by agreement because they each found each other's skills to be more useful for their times and so willingly entered into a social contract to form these medieval societies.