I would probably break down the multiplayer scene for MnB into 3 main groups. There is obviously crossover between each but roughly you can split them into:
-Large scale fights and events
These people primarily like the game for the spectacle of having massive multiplayer battles. 100v100 sieges and battles etc. You could compare this to games like Battlefield or the like where it's all about the technical and atmospheric achievement, moments where you're less concerned about winning and closer to roleplaying a big movie fight or something. Siege and Battle was the main modes from Warband for these guys, but Captain can supply some of this.
-TDM, DM, randomers
These people just want to hop in and do some casual, fun stuff for an hour or two. They don't care about balance or the like (although their enjoyment is impacted by it, so it matters) as long as they can have fun. The class system absolutely does not work for these guys.
-Competitive/Tryhards
This is the smallest but most hardcore of the scene. They genuinely enjoy the game at a competitive level, want to get better at the game, and are very opinionated on what needs to change. Skirmish was geared towards these guys, but was not wholly accepted.
So with this in mind let's think about what Bannerlord has done with MP and think about how it impacts each group.
Servers
No private servers or options for hosting means events are out of the question, and that the few game types around that do cater to the large events crowd are wholly reliant on matchmaking and servers with the rules set by admins. Further, these servers kick you out after a game, meaning sometimes long waits between games unless everyone immediately requeues or rejoins. However servers are hosted for TDM which broadly works. For the competitive guys, matchmaking mostly works ok but runs into balance issues in terms of stacks vs solos and new players, which has not been addressed quickly enough.
Gameplay
The game has been heavily criticised by the more experienced players for numerous balance and quality of life issues, while it's likely a minority compared to the actual multiplayer scene, they are loud and impact the general atmosphere. I also personally think they are correct (I've been one of them) and while devs have worked to address some of the complaints, they have done so extremely slowly. It's been over a year for some of them considering beta started pre summer of 2019.
All that said, I do think the game has been playable from a casual perspective for a while now, so this primarily impacts long term play and doesn't necessarily account for the quick drop off of player numbers. It mainly impacts the competitive/enthusiast scene and so the tournaments that could be happening (combined with the server issues above) are not as hype and populated as they could be.
Classes
Classes were introduced to make it easier for players to pick something and get into the game without being unarmoured noobs, and also for balance reasons across all game types. It works fairly well and has had a lot of thought put into it - unfortunately, it also hinders player's who want to take certain approaches or equipment, which doesn't feel satisfying. As a competitive player, I disliked being forced into one class with X equipment or another class with Y - the Warband system allowed for more flexibility. For casual players, it loses its charm quickly and people want to go naked/troll/just spawn with a greataxe, whatever. And for balancing, it's actually does as much harm as good, as relatively simple issues ended up needing large overhauls across the economic and class system, and other measures to try and fix things end up having unintended side effects.
The only thing this does work for is Captain mode, which overall seems to be the most well made game mode, which seems to have a bit of a dedicated following.
Conclusion
The result is one where: large event players have nowhere to go, TDMers have somewhere to go but don't really enjoy it long term, and competitive players are at best, split on whether the game is even good.
So how do we fix this? Well, for me I think the devs need to stop trying a one-size-fits-all approach. Many of these groups need completely different solutions. For immediate actions though:
1) Private servers or at least the ability to host large scale matches via TW dedicated servers. This should have been a priority for day 1 early access, instead the devs have actively blocked it. This will rather quickly repopulate the game and address a bunch of other problems.
2) Scrap the class system for modes where it clearly isn't working or enjoyed. That's TDM, duel, and arguably Battle if something like it ever comes. It cannot be that hard to import the equipment system from SP into a basic Infantry/Archer/Cavalry set up, and to have the MP Armoury switch to that, with classes kept for Captain and maybe Skirmish. I bet a few modders could (and will) do this in a month.
3) Another round of combat testing. The first was successful in fixing some basic issues, but changes from a few hours feedback didn't trickle into the main game for something like 3 months. I know it's not as simple as inputting what 20 random people say on Discord into your code, but it should be a little quicker. I think this comes down to the one-size-fits-all approach, but the balance concerns for PvP are totally different from the concerns of SP or Captain mode - this could allow a diversified approach where you more quickly adjust combat and other balance issues for solely PvP modes.