The thing is about quarrying and mining and farming is that it can be avoided really easily. For example,from the beginning of the game I looked for cities that held feasts and you can do around 3-5 tournaments on them,that means around 30-40k money.I pilled up the $ and bought houses that gave me prestige,that way I really had nothing to do with the peasant things. I suggest you put a restriction on the participation of tournaments.You should be at least middle class before entering a tournament. Imagine a tournaments where most of the lords participate and then a peasant shows up and he actually gets the first place in that tournament. Another bad thing about the difficulty of tournaments( I have not looked into options,maybe there is a way) is that after you get some companions, which will be low level anyway,you won't have any kind of difficulty if you want to try the fastest way to get money out of the tournament,that being 3 teams of 2 players with max bets( you need to get 4 points ) which is really easy, I finish a round in 30 seconds.
I bet if you put a restriction on this,more people will be annoyed for sure,but they would test the stuff you put out in the game,having to rely on it,not just test them.