Tuesday, May 18, 2010

Bonus Points are bad

I have been playing some Starcraft 2 on the Beta and have found a new mechanic that I really do not like - Bonus Points.  The way that the ranked matches in SC2 are set up is similar to a chess ranking - each player has a number and you are generally matched against players close to your own number.  Your point gains or losses from a match are changed based on relative ranking so that facing a higher ranked player is not a disadvantage from a points perspective.

The way Bonus Points work is that when you start you get a number of bonus points on your account and they slowly accumulate as time passes.  When you win a match you get your normal number of points and that gain is increased by up to double by taking points from your bonus pool.  So if I have 20 bonus points sitting around and win a match giving me 12 points my ranking goes up 24 and my bonus pool drops to 8.  The number of bonus points that you gain over time seems like enough to make a very big difference if you play 3 games a day and very small if you play 15 games a day.  This allows people who don't play often to more easily achieve higher ranks in the game but unfortunately it has some really poor side effects.

Here is the issue with Bonus Points:  They artificially inflate the score of players who don't play often above their skill level.  For example, I play every couple days for a few games.  This means that even though I only win 1/3 of my games I still end up staying at the same point total, so even if my skill level implies that I should be ranked 1100 instead I am stuck at 1300 and play constantly against people who play a lot and are better than me.  I still win sometimes but the fun of the game is significantly lessened when I can't possibly break a 50% win rate no matter how well I play.  Even if I were to magically become hugely more skilled I will be trapped at a low win rate forever unless I can substantially up my playtime.

I doubt that either side of this equation particularly enjoys it.  The people who play constantly get easy wins against casual players but cannot get ahead in points acquisition and the casual players lose all the time.  The best way to have a ranked ladder where people play against those of similar ranking is to have ranking be proportional to skill rather than have ranking be inversely proportional to the amount of time spent playing.  The ideal way to rack up ranking under this system is not to play a ton and be awesome but rather to play unranked matches constantly to practice and only play ranked matches when you have a full bonus pool waiting to double your gains.

I hope very much that this mechanic does not make it to the live game.  While I do recognize the value in learning from being defeated by strong opponents a big part of the fun in a competitive game is winning.  Setting up people who only play occasionally to be forever losing to better players at the same ranking isn't fun and isn't a good way to keep those casual players playing the game.  I want to play against others who are about the same skill level as myself regardless of how many hours per week we each play.

4 comments:

  1. I don't think your analysis is accurate here. I don't know full details about the system, but my belief is everyone accumulates bonus points at the same rate. In particular I know my bonus pool has gone up while logged in and playing, so it's not equivalent to double xp in WoW.

    So, assuming everyone has won enough to have exhausted their bonus pool, everyone has the same points inflation. Instead of a base of 1000 points, for example, everyone's base is 1400. This won't change anything about game matching, it just makes everyone feel better because they (on average) have a bigger number than they did when they started.

    Now, some people won't have chewed through their entire bonus pool. (Jer and I have hundreds of bonus points as our team hasn't played recently.) As such, we're disproportionately lower than everyone else. Their base is 1400, ours is still 1100. We rate to get easier matches than we 'deserve', but as soon as we play enough to chew through our pool we'll be back to normal.

    It seems like the people who benefit most from easy games in this case are the people who rarely play. People who sporadically play and people who often play end up in the same spot since they won't have a bonus pool to speak of.

    (This is only true if everyone actually has the same bonus pool over time, of course. I don't know how it works for people who join months later than others. They may be in for 'easy' games forever!)


    Your ideal way of ranking up is also incorrect. Your way to maximize rating compared to wins is certainly to only play with full bonus pool, but someone who plays a lot of non-bonus games will still end up higher than someone who plays none assuming that person can improve over time. If they're never going to get any better they'll just end up at their equilibrium point faster.

    ReplyDelete
  2. I don't think your analysis is accurate either, here are my reasons:

    I also have observed my bonus pool going up as I play, so I am sure it is a time based thing, not a offline-time based thing. However, you seem to suggest that this means that people will just have a static bonus to their totals and I don't think this is accurate at all. Assume for the moment that our theoretical dude has fixed skill and wins 50-50 at 1000 and 33-67 at 1400. This means that if he only plays 3 games a day or so he will get up to 1400 and stay there. If he then ramps up his playtime to 30 games a day he will still only be winning 33% at 1400 so he will lose points quickly and settle in just slightly over 1000 (maybe 1050?) and win 50% of his games and get a few bonus points each day.

    My ideal way of ranking up consisted of something you ignored: Playing unrated games constantly. There are plenty of competitive things you can do that aren't rated on the same scale and improving your play doing those while only playing in your competitive bracket with a full bonus pool means you rank up at maximum speed. Ideally you set up custom games on the maps you want to practice against good players or your buddies and play over and over until you are ready to go play a couple games with a full rack of bonus saved up. I don't particularly know that people will regularly abuse this but in a ranked system the best plan should always be to play as much as possible not to avoid playing unless a factor outside the game says it is favourable.

    ReplyDelete
  3. The thing you're missing is our theoretical dude who wins 50-50 at 1000 _now_ doesn't win 50-50 at 1000 in a month. The whole scoring system has shifted, and the same guy with the same skill actually now wins more like 75-25 at 1000. This is because the mean rating is no longer 1000 like it was at the start but is now 1400.

    Someone who gets to 1400, wins a third of their games, and stays still isn't actually staying still. They're losing ground relative to the median, likely because they're worse than average. (They are, after all, failing to win half their games at the median point.)

    Your ideal way of ranking up is flawed. I'll explain why in a second, but first I have to take a detour.

    Ignore bonus points for a second, and assume skill is fixed. Everyone starts at 1000 and plays an arbitrarily large number of games. Everyone is going to end up at their equilibrium point, right? This is where their rating 'should' be, and where they win about half their games against similarly rated people.

    Now, lets remove the large number of games restriction. Where do people end up? They start at the average (1000 in this example) and slowly meander their way to their equilibrium point. It's not a clean journey but generally speaking they should head towards that point. Someone 'bad' will be losing points and someone 'good' will be gaining points at the start and eventually they'll balance out.

    The thing to realize here is that to 'maximize' your score you need to play a lot when you're below your equilibrium point and to stop playing when you're above it. If you play more games when your rating is inflated you expect to lose rating so you shouldn't do it. If you play more games when your rating is deflated you expect to gain rating so you should do it. I think that's pretty straight forward.

    Now add in bonus points. How does this change the way things work?

    Well, it does two things. The first is the median is constantly rising. It's no longer a zero sum system, so having games played at all by other people who have bonus pool artificially deflates your rating. Your number hasn't changed but your place in the world has. 1000 is no longer an average dude; 1400 is and 1000 is rather bad.

    The second thing it does is it changes the win percentage needed to gain rating in the short term. Normally you need to win over half your games to improve your rating (assuming a decent matching system where you play people who average around your rating) but with bonus points you only need to win over a third of them. While you have a bonus pool, that is.

    ReplyDelete
  4. So, in this system, how can you maximize your rating?

    Well, if your rating is inflated such that you win between 33-50% of your games then you should play when you have a bonus pool and never any other time. Then you're only ever playing games where you have +EV in terms of rating.

    If your rating is massively inflated such that you win less than 33% of your games you shouldn't play at all. Even with the bonus pool you're going to go down.

    If your rating is deflated then you need to keep playing regardless of bonus pool. You expect to gain points even with no bonus pool, so you need to keep playing. Even if you're only winning 51% of the time, your win-loss record won't be great but your rating will slowly rise.

    Remember, again, that the existing of bonus points is inflated the average rating and therefore deflating a given individual's rating. As such, people who are near their equilibrium point who want to max their rating out constantly need to keep playing. Not only do they want to use their own bonus pool but they need to keep ahead of rating creep.


    Your ideal way of ranking up is flawed because all of those unranked games aren't helping you. Anyone with a deflated rating (and in a perfect system with no quitters there will be about as many of these as people with inflated ratings) wants to keep playing rated games to get closer to their (constantly moving) equilibrium point.

    Playing practice games with teammates is of course a great way to increase your skill. This is a way to boost your equilbrium point and therefore also to boost your maximum rating. You should absolutely practice. But if you're someone who 'should' be at 1400 normally and after bonuses have gone out 'should' now by at 1800 you're much better off playing rated games until you get there than just using your bonus, sitting at 1550, and practicing more.


    Your final conclusion is flawed in any zero-sum system. The best plan can't be to play as much as possible for half the people! They started out higher than they were supposed to, and therefore have nowhere to go but down. They're already at their highest point, playing more can't be good.

    And there's no easy way to fix that without just benefiting the people who can play 24/7. If the system is non-zero-sum in a way based on playtime then playtime can become a bigger factor than skill. At least now there's a minimum amount of playtime (enough to eat your bonus) to keep up and then extra gets you to equilibrium faster instead of extra just getting you to the top faster.

    ReplyDelete