Player skill is always a hot topic and games like World of Warships and many attempts have been made to come up with a simple metric for evaluating it, such as PR or Personal Rating. While such a metric can be used to potentially measure a players skill using other metrics such as winrate and average damage, how do we validate (prove) that what we are measuring is really skill?
While there are ways of validating a measure in the ‘real world,’ this particular instance makes it harder, however, not impossible. In this game, PR is based on averages. While sometime like win-loss is pretty binary, damage and kills is not and can vary quite a bit from battle to battle, thus the requirement for averages. However, what we don’t have access to is standard deviations, which make the average (mean) a more useful value. What we do have though, are timelines of players. From that we could, in theory, calculate our own standard deviations or use it to determine the face validity of PR (or any other metric for that matter) in determining and validating skill. Why would we got through so much effort though when the raw timeline gives all the information we need.
When we look at the timelines, what we want to see is consistency. While we expect to see ups and downs, I would want to see those to be relatively minimal with an overall upward trend. Looking at PR strictly is not recommended in this case, but rather looking at the stats that contribute to its calculation, namely damage and winrate. I would also look at average experience as well since as there are many other factors, including damage and wins, that go into the calculation of XP1 2 3. Players that are consistent while still showing signs of of improvement tend to be ‘truer’ to their predicted skill level than those that are not. Those that are inconsistent, I would say are more the product of luck and being carried than shear skill. Another point to that end, would be to consider the ships that a player tends to captain. Things that I would look for are ships that have a high impact on wins, such as destroyers or CVs, but overall low damage and high kills. While not too common, I have seen players that have higher winrates and kills but relatively low damage. Their ability to ‘skill secure,’ although not quite skillful in my book, will help lead to more wins as less guns in the battle means less things shooting. Furthermore, and finally, players that spend a majority of their time in divisions. These players also tend to have a higher than average winrate, but may not have the damage or kills to match and since winrate is such a large part of the PR calculation, that value could be inflated.
While none of the measures that we have at our disposal can individually be truly validated as a determination of skill, using them in conjunction with all the other information that is there tends to do a very good job in my opinion. It’s also worth keeping in mind that with a number of the individual statistics, luck players a significant role, from damage to winrate. The roll of the dice determines not only the over ‘skill’ of your team, thus your overall chance of winning, but also how your shells will land on target. Sure, you can be the best shot in the game but if RNG says no citadels from you, you get no citadels. So can we truly validate skill in a game that relies so much no chance? The short answer is, “No.” But we can certainly make a go of it and to see a player who has over 8,000 battles and a PR of over 2,000 compared to a player with over 8,000 battles with a PR of less than 1,000, I would comfortable say that the former is more skillful than the later. To what degree though is up for debate and the assessment of other information.