Algorithmic handicapping is WRONG for Online Games

Users who are viewing this thread

cuthbert5

Recruit
I read about the implementation of Bannerlord's new ranking system with a mixture of deja vu and dread, in particular this part:
The number of points you gain or lose with each match depends on the average ratings of your team and the enemy team. If your team has a lower average rating than that of the opponent and you win, you gain more points compared to a game where the average ratings between the teams are more balanced. And if you lose against a higher average rating team, you lose fewer points.
I've encountered this in my gaming career, and I have read and written about it extensively. I have dubbed these terms of use as 'algorithmic handicapping,' a form of game rigging (consumer fraud). American author, Harvard professor, social psychologist, philosopher, and scholar Shoshana Zuboff as coined the term 'instrumentarianism' to describe the wider industrial phenomenon that algorithmic handicapping is a part of.

I wrote about this on the Overwatch player forum in 2017, and I have maintained the thread for 5 years. Allow me to excerpt it the beginning and provide a link:
This thread is a plea for truth, democracy, and regulation in online gaming. It’s about a rising schism of cognitive dissonance, caused by a new form of consumer fraud that emerged in the last decade without anybody really noticing. It’s about the rights of online gamers, and users of all social media, not just as consumers but as democratic citizens. And if you follow it to the end, you will find yourself on the wrong side of a shield wall.


**Video adaptation**
I invite you to meditate on this subject with me further, to the backdrop of gameplay from Overwatch, one of the games in question. With musical score by The Toxic Avenger and Scattle, you see me playing as my avatar, Cuthbert, and my favorite character: a shield-bearer called Reinhardt. My name is Lars Bohr.

**Objective**
I seek to outlaw the use of Bayesian skill scoring frameworks and algorithmic handicapping in online games. Such applications turn skilled, experienced, competitive Overwatch players against each other without their knowledge or consent, based on hidden performance data and even protected personal data such as gender, income level, age, and location. Game holding corporations have found a powerful and profoundly unethical application for mathematical statistics, violating their customers’ rights to fair competition and informed consent of use.

**Plea for democracy**
Five years of debate and polling have proven with strong majority votes that players want transparent terms of use for Competitive Play, and non-discriminatory treatment from the matchmaking system. In three polls on the Battlenet forum, we rejected the handicapping of ranked matches with votes of: **295** to 56, **320** to 42, and **403** to 50. On this forum we have voted to ban handicapping of ranked matches at **388** to 64, **528** to 94, and **443** to 86.

**Plea for regulation**
I argue to remove Match Making Rating (MMR) from Competitive Play, based on statements from Blizzard/Activision reps and filings from Activision Publishing Incorporated with the United States Patent Office. The 2015 patent filing for Activision’s “Matchmaker” describes it as an anti-competitive, algorithmic operation to discriminate between players in every match. Blizzard’s implementation of the Matchmaker turns skilled, experienced, competitive Overwatch players against each other without their knowledge or consent.

**Plea for truth**
Blizzard/Activision say the Matchmaker ‘ensures the quality’ of gameplay. It designates teams to make every match ‘engaging,’ near 50% chance for either team to win. The Matchmaker is designed to constantly make ‘balanced’ matches to provide players with the most ‘exciting’ user experience. But because it ensures those 50% odds by arranging teams based on hidden skill metrics (MMR), the Matchmaker:

* Covertly handicaps Competitive matches,
* Favors new players over experienced players,
* Fails to prove the skill difference between players, and
* Lowers the quality of gameplay across all competitive tiers.

How is such a travesty possible? It is possible because of Blizzard/Activision’s deception, by omitting information about handicapping from their product. It is possible amidst the global dearth of regulation for tech companies like Blizzard/Activision, and a prevailing global culture of moral relativism. It is possible because the public withholds judgement on private corporations and the operation of their game holdings.

But most of all, it is possible because of players’ reticence to demand transparency in our terms of use, and fair rules in what is being sold to us as an “E-Sport.” Most players do not realize their matches are being handicapped. And those of us who know of handicapping fail to see its implications. Overwatch’s own designers seem to have missed the point.

**Overview**
Overwatch’s designers say they “balance” matches with MMR. The system sorts the twelve players from each match into teams, based on the merit each player has shown in matches past. Matchmaking uses merit-tracking algorithms (MMR) to keep matches from being ‘uneven.’ Principal Overwatch Designer Scott Mercer explains:

*"When the matchmaker creates a match, it determines the % chance for each team to win based on the match it made. The VAST majority of matches are usually near to 50% (especially if you’re a player closer to median skill rating and you’re not in a group). When we do put you in a match that we know isn’t a 50/50, we adjust your SR gain or loss based on your calculated change of winning.*

*"We model the synergistic effects of players being together in a group. Based upon the data we see in groups, we predict the win % for each team. We try to match similar sized groups together.*

*"The amount of MMR (and SR) you go up or down isn’t simply a matter of whether you won or lost, and what was your predicted chance of winning. There’s a couple of other things at work. One is the matchmaker’s confidence in what your MMR should be. Play a lot of games, it gets more certain. Don’t play Overwatch for a while, it gets less certain. You go on a large win or loss streak, it gets less certain. The more certain the matchmaker is about your MMR, the less your MMR will change in either direction based on a win or loss.*

*"We also do evaluate how well you played the heroes you used in a match. The comparison is based on historical data of people playing a specific hero (not medals, not pure damage done), and we’ve done a lot of work to this system based on the community’s feedback.*

*“While it’s a minor factor compared to wins/losses (The best way to increase your SR is still to play together and win as a team!), doing so does help us determine your skill more accurately and faster.”*

**However**
In Quick Play, we do not count wins and losses as we do in Competitive Play. We do not stake our rank and reputation on a number, like we do with SR. And MMR skews everyone’s SR. Because if you are a relatively skilled player for your SR, handicapping/MMR makes your teams worse than they would be on average, by random chance. Inversely, if you are relatively unskilled for your rank, handicapping/MMR makes your teams better.

**Semantics – “Balance” vs. Handicap**
This discussion has a fulcrum, a single word it turns around. A word that Blizzard has chosen incorrectly, misappropriated from the design parlances of casual, non-competitive games. The word is “balance,” which is actually handicapping in the context of a competitive game.

Dictionary .com defines a handicapped contest as one in which “certain disadvantages or advantages are placed upon competitors to equalize their chances of winning.” For example in old Quebec (French Canada), parishioners had a tradition of racing home from church in horse-drawn sleighs or wagons, which they would handicap by placing different numbers and sizes (weights) of passengers in either vehicle.

That’s an example of a friendly competition where handicapping is appropriate, because the important thing isn’t who wins the race; it’s the closeness of the race and the fun to be had along the way. The race itself is merely a pretense for a good time. Any scoring that took place between drivers would be in jest. That is what some of us expect from a game mode like Quick Play.

But players expect Competitive Play to be different. We have a rank and “Skill Rating” (SR) that ticks up or down when we win or lose. That number is both our reputation and our right to compete with other players of our caliber. Handicapping makes light of that number and, in turn, it makes light of Competitive Overwatch players.

When you play Competitive Overwatch you may be a horse pulling your team along, or you may be a passenger just along for the ride. And the handicapping system might designate you as such correctly or incorrectly. But those designations happen to determine the nature of every match you play.

***This is where the difference between individual and team competition comes in; players participate in matches as teams, but as individual in the ranking system system.***

**The big lie**
Handicapping teams is not the same as treating individuals fairly in the ranking system. Blizzard wrongly conflates those ideas, distorting players’ very notion of what fair competition is, and what they are doing in Competitive Play. Blizzard says that handicapped matches are fair for competitive players, and that is patently false.

**Related question for Activision**
Overwatch players from this thread want to know exactly how a patent, belonging to Activision Publishing Incorporated, is being used in Overwatch and its Competitive Play game mode. The patent was filed on May 14th of 2015, titled: “Matchmaking System and Method For Multiplayer Video Games.” I cannot provide a page link on this forum, but anyone can find it on the United States’ Patent Office website.

**Background**
I refrained from referencing this patent for a long time, to avoid making wrong or unnecessary assumptions about the implementation of Activision’s Matchmaking in Blizzard Entertainment products. From the first year of Competitive Play I argued on *principle and reason* that the algorithmic handicapping *we know about,* from the disclosures of *Blizzard* representatives, is wrong for ranked competitive play.

In a perfect world, a good argument and persistence might effect change. But this is not a perfect world. After half a decade of stony silence from Activision/Blizzard reps, I admit my original thread was stymied. The following content is a recent edition.

**Necessary assumption**
To continue discussion of gamers’ rights in ranked competitive play, we must assume that Blizzard/Activision have implemented a version of their 2015 Matchmaking patent in Competitive Overwatch, as they have done in the Treyarch/Activision Call of Duty franchise. And we must examine the patent more closely.

**Examination**
The abstract of the patent (number: US 10,322,351 B2) reads as follows:

*“A matchmaking system and method is provided that facilitates optimization of player matches for multiplayer video games. The system may provide a generalized framework for matchmaking using historical player data and analytics. The framework may facilitate automatic determinations of an optimal mix of players and styles to produce the most satisfying user experiences. The system may dynamically update analytical processes based on statistical or otherwise observed data related to gameplay at any given time. In this manner, the system may continually tune the matchmaking process based on observations of player behavior, gameplay quality, and or other information.”*

Nowhere in its description does the patent suggest this system could be applied to a ranked/ladder style game. I’m sure that consideration would be lost on the examining lawyers, unless there are gamers working at the United States Patent Office. And I know that the USPO does not determine or enforce the law. But I know that the Federal Trade Commission is supposed to enforce consumer protections against false claims in advertising, such as calling an algorithmically handicapped game mode ‘ranked Competitive Play.’

**The FTC website states that:**

*“The FTC Act prohibits unfair or deceptive advertising in any medium. That is, advertising must tell the truth and not mislead consumers. A claim can be misleading if relevant information is left out or if the claim implies something that’s not true.”*

With vague wording and deception by omission, Activision is tricking players to accept an absurd and dystopian paradigm for online gaming: algorithmically handicapped ranked matches. To understand why the ranked game mode of Overwatch is not truly competitive, and why the branding of ‘ranked Competitive Play’ is false, we must examine anti-competitive aspects of design in Activision’s patented matchmaking system.

According to Activision’s patent description, the “scoring engine” (analogous to SR/MMR) allows the Matchmaking system to operate based on variables that:

*“May include, without limitation . . . a relative skill level, a presence of preferred players (e.g., clan mates or friends), a team composition (e.g., play style, avatar specialization), a time that a given player has been waiting to be matched (e.g., in a game lobby), a player preference, and/or other information used to assess a potential match).”*

“Relative skill level” is a crucial aspect of the problem with this design. It implies that the matchmaker is discriminating between players within the same ranks of the ladder, which defies player’s expectation of impartial treatment in ranked competition. The patent goes on that:

*“In one implementation, the analytics and feedback engine may analyze game data (e.g., whether a given game level or match favors play styles)., historical player data (e.g., types/styles of player, strengths/weaknesses of players, etc.), and/or other information to assess a quality of player experiences.*

*“The analytics and feedback engine may analyze game data to determine satisfying types of game play that should be provided through the matchmaking process. For example, the analytics and feedback engine may determine whether given combinations of role types (e.g., sniper, run-and-gunners, etc.) lead to satisfying gameplay. Such analysis may be performed for specific portions of a game (e.g., a game level) and/or generally for a game.*

To illustrate Activision’s business interest, the description continues:

*“In some implementations of the invention, analytics and feedback engines may determine the quality score based on one or more business factors that describe a business value derived from a given gameplay session. For example, and without limitation, a business factor may include a business concern such as an amount of revenue derived from a given gameplay session (e.g., number for amount of in-game purchases, number of impressions of an advertisement or other rad-based revenue stream, etc.), a level of customer engagement, and/or other information that can be used to assess level of value derived from a given gameplay session.*

*“For example, player information may include, without limitation, a style of gameplay (e.g., aggressive), a role preference (e.g., an explicit by the player of such preference), a role actually played, a duration of gameplay sessions, a number of gameplay sessions played in a given login session, in-tame items used or purchased by the player, membership in a clan or team, preference to play with clan mates or friends, demographic information of the player (e.g., geographic location, gender, income level, etc.), win/loss records, scores, and/or other information that may be used to determine whether a player will enjoy a given gameplay session, a match and/or a game.”*

**Reasonable expectation of fairness and transparency**

I assert that the implementation of Activision Publishing Inc.’s patented matchmaking invention in Overwatch’s “Competitive Play” violates reasonable expectations of fair and transparent competition, by algorithmically handicapping players’ ranked matches. I further assert that Activision/Blizzard’s branding of a handicapped game mode as ‘ranked Competitive Play’ is false advertising, enforceable under U.S. Code, “Section 54. False advertisements; penalties.”

**Semantics**
As often happens in legal issues, our dispute involves semantics. Nowhere does Activision say the word ‘handicapping,’ in the description of their patented Matchmaker. But that is skirting a historically recognized term of game culture and gaming industry. It doesn’t take an expert to infer that many aspects of the Matchmaker (described above) constitute handicapping.

By withholding critical information about Overwatch’s design, Activision has created a comprehension vacuum in which nobody but their own staff can say what is going on with Competitive Play and its underpinning systems of ranking and handicapping. The patent refers glibly to ‘gameplay quality’ and measures of ‘satisfaction.’

**Fairness versus ‘quality’**
There can be no doubt that the invention of Matchmaking sacrifices fair and transparent competition. The sacrifice is in order to make Overwatch more addictive as a product, at the expense of its best players, by stymying their careers and converting their efforts into spectacle. In one passage of depraved usury from the patent description, Activision even describes how they would measure your biometrics for the purpose of making your experience more difficult and addictive:

*“Examples of quality factors include, without limitation, a player quitting a match or gameplay session while other players are still playing (indicating dissatisfaction), a duration of a game session (e.g., a longer duration may indicate greater satisfaction), a gameplay performance factor (e.g., a kill-to-death ratio in a shooter game, a lap time in a racing game, etc., where greater performance may indicate greater satisfaction), a player engagement factor (e.g., a speed of player input, a level of focus as determined from camera peripherals, etc., where greater engagement may indicate greater satisfaction), a competition level of a game (e.g., whether lopsided or not, where evenly matched games may indicate greater satisfaction), a biometric factor (e.g., facial expressions, pulse, body language, sweat, etc.), explicit feedback from a player (e.g., responses to a survey), and/or other observable metrics related to gameplay.”*

Bannerlord players and developers, please heed my message. Help me break the dystopian trend that has set in to online gaming.
 
So you want some kind of fascist meritocratic dystopia that favors the established elites and keeps the newbies out?
Here's a participation trophy for your effort given to you by an algorithm that guessed you are a sad person.
9fb154176b197e3e3e950b8bba5cbdee.jpg
 
Prime example of mathematical clouding to spew non-sence.
Every ELO sysem is a statistical model, every statistical model has axioms.
Certain approaches to this modelling might create unhealty seeding match-ups.
Non of the elements you are critisizing about Overwatch Matchmaking is applied here.
If anything experience of the playerbase is towards personal performance having very little impact.
 
I wrote about this on the Overwatch player forum in 2017, and I have maintained the thread for 5 years. Allow me to excerpt it the beginning and provide a link:
You wrote a book, which I read about half before tiring. Allow me to summarize your complaints I managed to make it to:

1. MMR is too granular and instead anyone in a Skill Rank should be considered an equal to anyone else in the same skill rank.
2. Overwatch's width of MMR matching (1k) is too wide, you would prefer that people's actual MMR be obfuscated and that the bucket SR categories be used instead, which, hilariously, are probably about as wide as the MMR range.
3. Every match is unfair to the best player on the team, for both teams, even though both teams have better and worse players, so instead, one team should have all the good players and the other should have the bad players.

Amazing.
 
Allow me to summarize your complaints I managed to make it to:

1. MMR is too granular and instead anyone in a Skill Rank should be considered an equal to anyone else in the same skill rank.
2. Overwatch's width of MMR matching (1k) is too wide, you would prefer that people's actual MMR be obfuscated and that the bucket SR categories be used instead, which, hilariously, are probably about as wide as the MMR range.
3. Every match is unfair to the best player on the team, for both teams, even though both teams have better and worse players, so instead, one team should have all the good players and the other should have the bad players.

1. I never said "granular," but I have said that MMR is a double standard which effectively lets the Matchmaker segregate players based on measured differences in skill. I believe that rank should be the basis of matchmaking, not MMR. But your take on this is respectably accurate.
2. MMR should not exist. I don't know what scale SR would exist at if the system was redesigned, but it would probably have to change on some order of magnitude. The range for matchmaking should be much more narrow, at any rate.
3. It is not the logical conclusion of my argument that all of the best and worst players should be on opposite teams from each other all the time. However that should be the case at least some of the time. MMR guarantees that it practically never happens, which is an artificially anti-competitive construct.

Thanks for your close reading and I hope you'll accept these notes.

So you want some kind of fascist meritocratic dystopia that favors the established elites and keeps the newbies out?
How on earth is that your takeaway? I never said anything remotely like this. At least cite a passage if you think this is what it means.
 
I believe that rank should be the basis of matchmaking, not MMR.
MMR should not exist.

Again non-sence, match-making is either totally random or there is a MMR number.
Some games decide to hide the number from the playerbase, some games even does rated matchmaking but even hide vague ranks.
Or very rarely MMR is calculated as a temporary variable from scratch every time player joins the queue (from players history).

3. It is not the logical conclusion of my argument that all of the best and worst players should be on opposite teams from each other all the time. However that should be the case at least some of the time. MMR guarantees that it practically never happens, which is an artificially anti-competitive construct.

What evidence are you proposing that, this is not happening time to time in BL? It has been observed that if you are on a win-streak you start getting very unfavourable(according to player's current rating) games and rise really fast if you win these unfavourable seeds.

On a side note, Competitive Overwatch Streamers has taken accounts that were "stuck" in bronze/silver over a year and rose them up to Grandmaster before.

Finally, I have no idea if you are completely ignorant of "Game Theory" or gone mad because you studied it too much, but I would like you to define this theoritical (with mathematics not ideas) match-making system that does not include a MMR.

What is it? a State-Machine? a Neural Network? a bird? a flying man?
 
Current issues with MMR are:
1. When you win you gain about 1/3 of the points that you would lose if you lost. Simplified, I am gaining about 2 or 3 points per win, while losing 6 or 7 per loss.
2. After a certain amount of matches your gain is very low, now this can be fine as it is (because if the system works correct it would find your skill level, and progress would be slow but it would be there, as your mmr would rise slowly along with your skill), but gaining 2-3 points for matches where you are hardcarrying makes no sense. (A minimum should be established in my opinion of about 10 points)
3. It's just not balanced right now
 
What evidence are you proposing that, this is not happening time to time in BL? It has been observed that if you are on a win-streak you start getting very unfavourable(according to player's current rating) games and rise really fast if you win these unfavourable seeds.

On a side note, Competitive Overwatch Streamers has taken accounts that were "stuck" in bronze/silver over a year and rose them up to Grandmaster before.

Finally, I have no idea if you are completely ignorant of "Game Theory" or gone mad because you studied it too much, but I would like you to define this theoritical (with mathematics not ideas) match-making system that does not include a MMR.

What is it? a State-Machine? a Neural Network? a bird? a flying man?
I don't know exactly what is happening in Bannerlord. But I know the design trends of online games, and I see Bannerlord falling into a really terrible one that I am familiar with. It sounds like you are also familiar with 'performance-based' ranking systems. Do you know this from Overwatch?

Part of my thesis is that we cannot count on PBSR and other systems to make accurate judgements about player skill. I believe that should be measured by the match result and the win/loss record of each player. The problem with MMR and other algorithmic handicapping systems is that they pervert the match result, making it intrinsically unrelated to the measurable skill differences between each player, and necessitating performance-based rating in the first place.

By the way, these systems are not comparable to ELO because ELO is designed for tournaments of a 1 v 1 game. MMR, EOM and other algorithmic handicapping systems are being implemented in games that have many players on each team, and are not true tournaments. The multiplayer aspect is crucial, because algorithmic handicapping systems depend on multiple players to balance the scales.

The alternative I propose is simple and obvious, to anyone with the slightest grasp of game theory. Matchmaking for ranked competition should be based on rank, and it should be solo-queue only. Rank should be based completely on match results. I.e., did your team win or did they lose, what is your win loss record? Anything else is a travesty.
 
Last edited:
Overwatch's system involved comparing your stats on the hero you played to the players who are "better" at the specific hero.
This "betterness" is decided by overwatch devs according to their whim, which became meaningless with every meta change.
Which resulted in players focusing on stat padding on specific heroes instead of doing their best to win, bc this is more advantageous if you just wanna raise your mmr. This was further elevated by the reality that due to Overwatch's nature as a game, statistics are not as meaningful as other games.
Matchmaker failed to seed games both fun or fair, this doubled with virtue-signalling balance patches has killed a great multiplayer game which could have shined for decades.

BUT non of these factors are prelevant in Bannerlords matchmaker currently, hence me disreagarding your critique.

Overwatch and Bannerlord are similar in the fact that Developers who dont actually play the game but look at Spreadheets and Design docs instead, and assume they grasp the nature of the game better than the players, raining down cheesy comments about the "process" and "slow steps" form their ivory towers.

This similary does not however extend to the matchmaker.

Thanks for Listening to my TedTalk.
 
Back
Top Bottom