Does rematch option disturb ranking system?

Discussion in 'Fantasy Strike' started by DrFaustus, Apr 18, 2012.

?

Should the scoring mechanism for rematches after QM be changed?

  1. Yes, it is too exploitable.

    12.5%
  2. No, it is ok as it is.

    0 vote(s)
    0.0%
  3. I'm not sure.

    37.5%
  4. I don't care.

    50.0%
  1. DrFaustus

    DrFaustus Active Member

    Ok, I'm actually pretty sure, that it does; so the question should be: Does it matter?

    First, let me explain, what the problem is: When you start a QM, you are matched via the match making system against an a priori unknown opponent. Knownledge about counterpicks helps in that first match only via the current meta (popular characters, weekly free caracters...). I consider this as the fair match. The problem arises, when both players agree to rematch and the loser (let's call him Bob) of the first match changes his character (or the bank in Puzzle Strike). So Bob now has an advantage, because he new the opponent (and bank) and could counterpick. Still the outcome of the rematch effects the Elo score of both players in the same way as the first (fair) match.

    Consequences: Someone playing to maximise his Elo should never agree to rematch after winning and always try to rematch and counterpick after losing a match.
    Also, using statistics about top or average Elo scores to estimate character strength/balance becomes unreliable, since some characters are more prone to counterpicking, while others might be used mostly as a counter pick and different players behavior might bias this even further.

    There are different possible solutions:
    • Don't let rematches after a QM influence the Elo score
    • When rematching after QM, allow both players to change characters
    • Use a seperate Elo score for groups of rematches, such as best of 3/best of 5 (first game effects the character Elo, the whole game (all matches with standard counter picking) effects the best of 3/5 Elo score
    I know, those solutions aren't very elegant. That's why I ask: Is the problem important enough to even consider the trouble with those solutions?
  2. garcia1000

    garcia1000 World Champion Staff Member

    !

    I thought rematching allowed both players to change characters.

    That seems fair in a best of 1 ranking
  3. CrystalChaos

    CrystalChaos Moderator Staff Member

    Currently both players can change characters for a Yomi rematch, but not for a Puzzle Strike rematch.
  4. Logo

    Logo Well-Known Member

    Don't forget the changes to ELO after the first match, which makes the situation a bit different. In the rematch there is quite an additional ELO gap between players (up to 2x what was at stake of the first match) so the ELO at stake changes.

    Going WL means you almost always end up with a lower ELO than you started with, LW the opposite.

    Overall your rating will equalize regardless of rematching or not so it's not a big issue. At most rematching puts you in a temporary low or high point, something that can happen by chance opponents anyways. Re-matching might help someone artificially boost their rating, but it would do so at the expensive of someone else's ELO so it's no different than handing your login information to someone and having them grind up your ELO. It's the opposite (queue dodging) that really messes up a rating.

    Still your consequences are true anyways. In ELO it's optimal to play players whose ratings are at a peak (as their elo is likely to be higher than their actual skill) and anyone who just won a game is more likely to be at a peak than someone who just lost a game (whose rating is likely to be below their actual skill).

    ---

    Besides ladder is never a true reflection of skill anyways, only an approximation.
  5. ApolloAndy

    ApolloAndy Well-Known Member

    I'm pretty sure that using the leaderboards or their related ELO as a measure of anything real is a mistake. For one, you don't even get your "real" ELO until master 6? 7? somewhere in there. So a lot of it is just who's played more. Which means that for a whole lot of people, 18 or so of their characters are vastly underrated and playing against them if you care about ELO is a terrible idea.

    Also, if you really care that much, there are a million other ways to game it, like creating fake accounts to beat on, only playing your "serious" characters against n00bs, only playing your serious character on weeks when the free chars favor him, leaving matchmaking if you see an opponent you don't want to face etc. etc.

    But yes, in a vacuum, you are exactly right. Counter picking is one of the many ways to game the system.
    Delha, Leartes and rabid_schnauzer like this.
  6. DrFaustus

    DrFaustus Active Member

    Ok, thanks. I only saw it in PS so far, didn't know about Yomi.

    The reason I came up with this is that I saw Rabid_Schnauzer try to construct a character ranking of PS-characters out of the top players Elo scores.

    So far, it doesn't matter much to most of you, as I thought, and I agree, that it's a minor point. Also, there are obviously other options to game the Elo rating, if you want to. Still, if there is a ranking system, I think, it should at least try to be as accurate as it can be and not animate players to use a feature, that lessens its accuracy.

    I know, tournament rules of counter picking are fun, but they make more sense in a best of 3/5 environment. The Yomi version to allow both players to change characters for a QM is more reasonable. In PS there is the question on who may change the bank.

    So i figure, the optimal solution from an Elo point-of-view would be new random bank and both players may switch chars for a rematch. If some other rules about bank/character selections should be chosen, the rematch should not affect the Elo ranking.
  7. ApolloAndy

    ApolloAndy Well-Known Member

    I actually like that, too, though I think the winner should have the option to veto a non-counting rematch (if you're grinding, for instance).

    If anything, my biggest annoyance is when I'm playing an opponent and I keep winning (it happens "on occasion") and I grow tired of the char/matchup/bank I'm playing but the opp. doesn't want to switch. My char. in particular can't be changed by the opponent so even if they do want to switch, I'm kind of locked in. I know I can requeue, but I don't want to lose the chance to play if the opp. decides he's done or whatever else.
  8. Delha

    Delha Active Member

    You'll also notice that a good chunk of that thread is spent discussing holes in the metric and possible explanations outside of raw character power. That set of rankings is understood to be just one of multiple way to look into balance, so I wouldn't worry about it. Nobody's getting nerfed to the ground or anything based on that list.

    I very much doubt any PS characters are getting changed at this point unless some spectacularly degenerate new strat is found, and I consider that unlikely.
  9. rabid_schnauzer

    rabid_schnauzer Well-Known Member

    Yeah, the leaderboards are mainly an indication of who has played the most.with those characters. There are a number of ways to game the system and a couple of characters have seen final changes since the last reset.

    And so far, tracking the top 3 leaderboard players per character over a few weeks has revealed that such are really pretty static. And by static I mean a lot of the top leaderboard players maintain their positions without actually playing a single match with that character week to week.

    Major changes in character based w/l ratios seem to only happen when new players break into the top three places on the leaderboards
  10. Delha

    Delha Active Member

    FTFY :p

    I don't forsee my Jaina or Val ever getting near the top. I like how they play, but I think that for whatever reason just bad with them. Same thing for DeGrey minus the liking how he plays, heh.

    Yeah, I've been too busy to play much lately, but I know my Sets kept dancing around 3rd to 5th for a while. When I was playing her a lot, I'd check periodically and it seemed like the other people around there didn't change much, if at all. I'm probably still somewhere in that range now, and I think I played just one game on her in the last month.
  11. deluks917

    deluks917 Yomi League 1 Champion

    In my opinion the yomi leaderboard works sorta ok. I think if you are good you should be able to get a high ranking. However having a high ranking does not mean you are good.
  12. Fry

    Fry Well-Known Member

    Regardless of actual character power level, Jaina, Onimaru, and Midori are on top of the PS leaderboards because it is easy to win games very quickly with them, so you can gain more rating points per amount of time played. As opposed to, say, Quince, where your goal is a ridiculously long game.
  13. rabid_schnauzer

    rabid_schnauzer Well-Known Member

    Fry, while you're likely right about speed of matches being a major factor in the leaderboards, currently there isn't a Midori in the top 20.

    Jaina accounts for 7 of the top 20, Oni and Val show up three in the top 20, Perse twice while Rook, Sets, Menelker, Lum and Arg each once.
  14. Thelo

    Thelo Administrator Staff Member

    Strictly speaking, yes, rematching does affect the integrity of the leaderboards, if only by giving you control over which opponent you're facing. We figured that overall, the convenience that allowing rematches gives to players (less time spent waiting in queue for a game, being able to refine your strategy against a given opponent) outweighs the drawback of making the leaderboards a little bit less serious.
  15. swordsman3003

    swordsman3003 Well-Known Member

    I almost always rematch as much as I have time for, even MORESO when I'm playing against somebody who is losing to me. I'm well aware that each match, there is more at stake, but that makes it exciting for me. When I can beat someone who is able to counter pick or change the bank multiple times, I feel really confident in how I'm playing a character. I'm also pleased when somebody finally picks up on how they should be playing against me and starts becoming more of a challenge. I got 3 PS characters to grandmaster rank using almost entirely rematches so I don't think it's necessary to game the system with rematches, if you really are better than someone you can consistently beat them.

Share This Page