The longest I've played in this game is 9 years, and my own major problems with mid-to-late game are as follows:
1) The Stash System: Inventory management is a hassle in this game, and it impacts your ability to manage resources. I would love to stockpile weapons, metals and charcoal to upgrade my smithing over time, but a full inventory slows down your army and makes it harder to do things like farm bandits and catch smaller enemy groups. Thing is, you only get access to such a stash when you own a city, and for some reasons castles don't have such an inventory. Back in Warband, you could use a Workshop as a place to store excess items long before you swore fealty to a liege lord.
Taleworlds needs to add different stash locations for players - perhaps buying a warehouse in a city? It might come with the same risks as owning a workshop - an enemy faction might claim it and take all your stuff. This is already a risk when your city is taken by an enemy force (I think).
2) The Garrison System: It should be a given that, same with the inventory, the player will want to be able to station their troops in different locations - especially if they want to configure their army composition. It's a lot easier to level recruits when there aren't any veterans hogging the kill count. And of course, I want to secure a city for myself as a base of operations. There are few moments in a campaign more thrilling than being granted your first city - and I want to line the walls with crack troops who will guard it while I'm campaigning. The problem is that the food system for the garrison ensures that whenever your faction is at war (which is almost all the time) that your settlement starts starving - so even puny garrisons of 50 men will eventually start deserting you. And yet, Taleworlds has upgrades for cities allowing for close to 400 men garrisons.... really? 400?
Again, Taleworlds needs to fix this. Either remove the garrisons connection to the settlement food supply or give the players a way to supplement garrison rations. Because right now I simply can't station anyone in a city, much less use settlements as I wanted.
3) Your Missing Siblings: You could be on top of the world. A personal army of close to 200 men, millions of denars, a prince or princess as your spouse and a weapon you yourself forged to legendary status that is ready to be handed down your lineage. And none of that matters because your siblings, since the beginning of your game, are still being held by a raggedy group of slavers.
I know Taleworlds was laying the ground work for the player to start with a family rather than as an adult orphan like in Warband, but at present it's a glaring problem with the game. Until they finally complete their work on the household system and allow you to save your siblings, it feels just bonkers continuing the so-called "main questline." Which brings me to my next qualm...
4) The Main Questline: Since With Fire & Sword and Viking Conquest, I was always under the impression that Talewords was absolutely terrible at weaving a story into their game system. That hasn't changed. The so-called Main Questline is just a series of ever more painfully difficult objectives that need to be checked off without any sort of payoff.
If Taleworlds decides to add extensions or changes to this mess of a quest, they need to make it more character centric and easier - because the real story-line in this game should be the one the player makes - not the developers.