Hello everyone, and welcome to the final post of the r/NFL Top 100 Players of the 2022 Season... our Post Mortem thread!
Included here are the individual ranker lists, the master list, and rank breakdowns from this year's ranking process.
Ranker Lists / Master List / Calculations
Here are the things I’m sure everyone wants to pour over - the individual rankings, their corresponding sheets, and the master list with associated fancy data.
The lists that follow are the personal opinions of 53 individual people combined to find an average rank. Some of our users shared their lists as we went along with each reveal. As promised from the beginning, all data is being made available to you.
As a refresher, here is a quick run down of the methodology:
Methodology
Link to more detailed writeup on our methodology
-
Step 1: A Call to Rankers right after the Conference Championship games
-
Step 2: Rankers from each team nominated players to rank, with a 11 game minimum threshold. Players are associated with the team they played for in 2022
-
Step 3: The Grind. We instructed users to tier positions groups into T25, T50, etc based on 2022 regular season play only. This took several weeks as the rankers tiered each position group and discussed them. There were no individual player threads and no arbitrary position caps. Just questions and rankings.
-
Step 4: Users submitted their own personal Top 125 lists.
-
Step 5: User lists were reviewed by myself, u/mattkud , and u/MikeTysonChicken . The rankers were expected to answer questions about their lists. They were allowed to make any changes to their list, and were not forced to make any changes
-
Step 6: The Reveal… which we just finished!
With that said, here are the lists:
These are the completed ranking forms from 1-125 for each individual. After all lists were submitted, u/mattkud and I combined the lists into one sheet to calculate an average rank for each player and a standard deviation to use when trying to find outliers. There were 194 players nominated; with rankers tasked to rank out to 125, any player that didn’t fall on a user's Top 125 was assigned an unranked value of 140. This was used to help calculate the average rank.
We then used conditional formatting within Google Sheets to highlight ranks that were 1 standard deviation off a players mean rank in addition to using 2 standard deviations. The biggest reason why we also used 1 standard deviation is that numerous players had large standard deviations to begin with, as you’ll see. This makes sense, especially towards the bottom end of individual lists when players can be unranked by numerous rankers. While it’s not ideal, we had to ensure that we caught players that were accidentally omitted from user lists. Highlighting both standard deviations made this a possibility.
We then reviewed each user's list with assistance from /u/MikeTysonChicken. As a former showrunner of the Top 100 list, he gave us an additional set of eyes to observe the process . Part of our hesitation with performing the list reviews is we don’t want to indirectly coerce rankers to move players to certain spots. That would go against the entire spirit of this project. So, we simply stuck to the conditional formatting as outlined above and asked each individual ranker their thoughts on the players identified as outliers, requiring justification on players 2 or more standard deviations from their mean. There were some users who realized they forgot to rank a certain player, or mistakenly had a player ranked lower (or higher) than they intended, and they were free to make corrections. For example, more than a few ranked Jaguars Josh Allen when they intended on ranking Bills Josh Allen and vice versa. If rankers felt like they needed to make any other adjustments, they were free to do so at this time, but they were under no obligation to do so. If they felt their list was justified and were fine with all of their submission, once they provided that concurrence, we locked their list. Once all mistakes and changes were made for other users, their lists were locked as well. All completed lists were reviewed and locked with the concurrence of the individual ranker.
Once this was complete, we calculated the ranks to find the Top 100 Players of the 2022 Season.
This year we continued to remove 1 high rank and 1 low rank to calculate the average rank. In the case of ties, we broke them by including the 1 high and 1 low rank and recalculating their average. This occurred twice in this years rankings.
Historical List
And now for a fun little trip down memory lane... Here is the Historical List:
LINK TO HISTORICAL LIST
Master Ranking List
Notes:
-
Column BG is the sum of all the ranks for the player
-
Column BH being the average rank of Column BC.
-
Column BI is the sum of all the ranks for the player minus the sum of the highest and lowest rank for each player. If there are multiples of the same value, it only subtracts 1 of the highest/lowest. This is what is used to find the final rank. The formula looks like this:
=SUM(E2:BE2)-LARGE(E2:BE2,1)-SMALL(E2:BE2,1)
- Column BJ is the average value, and thus rank, for the player minus the highest and lowest rank divided by the total number of sheets (53) - it's highlighted in yellow. Essentially, Column BI is divided by the total number of sheets. This is the final average rank for each player. The order, 1-194, is determined from this value, low to high. You’ll notice 4 cells highlighted in green in this column for the Top 125, this was to visually indicate ties to me.
Calculations and Outliers
Explanation
Final Calculations - w/Adjustments & Concurrence: This is the conditional formatting we used to determine outliers for the sheet reviews. This is NOT the sheet we used for the outlier sheet, this is merely an example of the formatting that was used for the outlier sheet. You can clearly see certain ranks are color coded to indicate ranks that were much above or below the congregate average. The original outlier calculation sheet included several mistake ranks that were corrected, as well as a single list that was removed during the calculation stage. This is included for transparency’s sake, just so y’all can see a bit under the hood.
-
Red cells are high ranks that are 2 standard deviations from the mean, as in the ranker was too high on a player strictly per standard deviation.
-
Blue cells are low ranks that are 2 standard deviations from the mean, as in the ranker was too low on a player strictly per standard deviation.
-
Yellow cells are high ranks that are 1 standard deviation from the mean but less than 2 standard deviations as well. These are not outliers and rankers didn’t have to account for them. I simply noted them in case there was an error.
-
Orange cells are low ranks that are 1 standard deviations from the mean but less than 2 standard deviations as well. These are not outliers and rankers didn’t have to account for them. I simply noted them in case there was an error.
CAN THE STANDARD DEVIATION METHOD BE IMPROVED UPON?
Yes, this is a real question we could use an answer on from the sub. r/NFL suggested to us a couple years ago to use standard deviation when trying to identify outliers. I believe the method works very well as you can see in the data above though I do think it has its flaws. This is largely due to how the ranking average is calculated.
You’ll notice a high number of players have standard deviations in the mid-20’s, or even 30’s, which makes identifying true outliers really difficult. A good example of that is Aaron Donald. He has been the best defensive player over the last decade and was still extremely good when on the field, but his playing time made him really hard to judge. Some people didn’t care about missing time due to injury and ranked him highly, other’s cared a lot and ranked him very low. Neither viewpoint is necessarily wrong, but it does make his standard deviation very high and using it to find outliers seems rather pointless. And he didn’t even have the biggest standard deviation!
For now, we’ll continue with the sheet review as done above. However, if anyone has a way to improve how we identify outliers without ranking out all players to 200+ let me know! I know this is basically a copy/paste from the question we posed last year, but we’re always looking for ways to improving the list and this is one of the bigger things that could be improved
Breakdowns
Divisions Combined
| Division | Total | Division | Total |
|---|---|---|---|
| NFC North | 9 | AFC North | 12 |
| NFC East | 20 | AFC East | 15 |
| NFC South | 8 | AFC South | 7 |
| NFC West | 15 | AFC West | 14 |
| TOTAL NFC | 52 | TOTAL AFC | 48 |
By Team
| NFC Team | Total | AFC Team | |
|---|---|---|---|
| Eagles | 11 | Dolphins | 6 |
| 49ers | 8 | Chiefs | 5 |
| Vikings | 4 | Bills | 4 |
| Lions | 3 | Chargers | 4 |
| Cowboys | 3 | Steelers | 3 |
| Commanders | 3 | Ravens | 3 |
| Giants | 3 | Bengals | 3 |
| Buccaneers | 3 | Browns | 3 |
| Rams | 3 | Jets | 3 |
| Seahawks | 3 | Titans | 3 |
| Packers | 2 | Raiders | 3 |
| Saints | 2 | Patriots | 2 |
| Panthers | 2 | Colts | 2 |
| Falcons | 1 | Broncos | 2 |
| Cardinals | 1 | Texans | 1 |
| Bears | 0 | Jaguars | 1 |
Positions (+/- from Last Year):
| EDGE | IDL | LB | CB | S | Total Defense |
|---|---|---|---|---|---|
| 12 (-1) | 11 (+2) | 7 (-1) | 10 (-2) | 8 (0) | 48 |
| QB | RB | WR | TE | OT | OG | C | Total Offense |
|---|---|---|---|---|---|---|---|
| 10 (+1) | 6 (+1) | 14 (+1) | 3 (0) | 9 (+1) | 6 (-1) | 4 (-1) | 52 |
Thoughts / Recap
I just wanted to take a moment and give my own thoughts on the changes and recap this year's version of the r/NFL Top 100 since I largely tend to stay out of the conversation.
1). u/mattkud and I are extremely thankful that you guys have stuck around this long to read the list! I may have been the more public face here on r/nfl by posting all the threads, but kud was crucial behind the scenes creating the various spreadsheets and virtual infrastructure needed to organize everything. I want to stress this was not a one man show, but a team effort. The rankers put out a fine list this year and were (mostly) well-behaved in the comments section.
2). For the last couple of years we’ve instituted a minimum games threshold to the list. This year it was 11 games, which was a rise from 10 games on our 2021 list but more in line with our 10 game threshold in a 16 game season in 2020. This meant that guys like TJ Watt and DJ Reader were left off the list before the process could even start. We got a fair amount of comments about the games requirement wondering why guys like Watt weren’t showing up here. And I don’t want to single out Watt here as being especially problematic, even if we reverted the games threshold to 10 games, then guys like Cooper Kupp and Rashan Gary would be left out of it, and these guys very well could be seen as worthy of being on the list. As of right now we give the rankers a fair amount of leeway with how they want to take injuries into account, some (rightfully) don’t care about missed time and grade the players simply on the quality of whatever snaps they had, while others (rightfully) penalize players for missed time, because after all availability is the best ability. This means that players that did hit the minimum games but still missed significant time like Aaron Donald and Jordan Poyer have very large standard deviations. I’m not sure this is even really an issue because everything eventually averages out, but I felt like the missed games thing was especially apparent this year. This is after all a community driven list so I do want to ask, is this something that you think should be re-evaluated? Is an 11 game minimum threshold too strict? Is it too lax? Should we use snaps instead of games? Should there be a limit at all? What is the airspeed velocity of an unladen swallow? I wonder what you guys think about that
As always, we cannot thank enough the showrunners who came before us, u/MikeTysonChicken , u/Mister_Jay_Peg , u/skepticismissurvival , and u/Staple_Overload . These guys have made kud and I’s jobs a lot easier by providing the foundation that we used to build the ranking process. Instead of trying to reinvent the wheel, all we’ve had to do is copy and reformat the existing spreadsheets in order to fit the current year. It’s been a joy to run the r/NFL Top 100 list this year (for the most part) and I personally cannot wait to run it back next year
Closing
The book is now closed on the r/NFL Top 100 Players of the 2022 Season. If you want to look back at any of the releases, the Hub Post should have you covered
Keep a lookout for the Call for Rankers sometime around the Super Bowl if you would like to join next year.
Walang komento:
Mag-post ng isang Komento