Hello everyone, and welcome to the final post of the r/NFL Top 100 Players of the 2020 Season... our Post Mortem thread!
Sorry for the week delay, was pretty busy.
Included here are the individual ranker lists, the master list, and rank breakdowns from this year's ranking process.
Ranker Lists / Master List / Calculations
Here are the things I’m sure everyone wants to pour over - the individual rankings, their corresponding sheets, and the master list with associated fancy data.
The lists that follow are the personal opinions of 56 individual people combined to find an average rank. A lot of users shared their lists as we went along with each reveal. As promised from the beginning, all data is being made available to you.
As a refresher, here is a quick run down of the methodology:
Methodology
LINK TO THE HUB POST WITH A MORE DETAILED DESCRIPTION OF THE METHODOLOGY
- A CALL FOR RANKERS just before the Super Bowl.
- Rankers for each team nominated players to rank. 10 Games Played Minimum Threshold. Players are associated with the team they finished the 2020 Season with.
- The Grind. Utilize ranking threads for individual rankers broken up by positional groups. Users were tasked with ranking players within the following tiers based on their evaluation: T-25, T-50, T-100, T-125 based on 2020 regular season only. There were no individual user case threads. There were no arbitrary position limit caps. Just questions and rankings.
- Users submitted their individual Top 125 list.
- User lists were reviewed for outliers by me with assistance from two former rankers. Users were permitted to correct any mistakes found. Once complete, lists were locked.
- Reveal the list… just completed!
With that said, here are the lists:
These are the completed ranking forms from 1-125 for each individual. After all lists were submitted, I combined the lists into one sheet to calculate an average rank for each player and a standard deviation to use when trying to find outliers. There were 203 players nominated; with rankers tasked to rank out to 125, any player that didn’t fall on a user's Top 125 was assigned an unranked value of 140. This was used to help calculate the average rank.
I then used conditional formatting within Google Sheets to highlight ranks that were 1 standard deviation off a players mean rank in addition to using 2 standard deviations. The biggest reason why I also used 1 standard deviation is that numerous players had large standard deviations to begin with, as you’ll see. This makes sense, especially towards the bottom end of individual lists when players can be unranked by numerous rankers. While it’s not ideal, I had to ensure that I caught players that were accidentally omitted from user lists. Highlighting both standard deviations made this a possibility.
I then reviewed each user's list with assistance from /u/wrhslax1996 and /u/thamasthedankengine, two former rankers. They served as a sounding board for me just so I made sure I looked at each list fairly. Part of my hesitation with performing the list reviews is I didn’t want to indirectly coerce rankers to move players to certain spots. That would go against the entire spirit of this project. So, I simply stuck to the conditional formatting as outlined above and asked each individual ranker their thoughts on the players identified as outliers, requiring justification on players 2 or more standard deviations from their mean. There were a few users who realized they forgot to rank a certain player, or mistakenly had a player ranked lower (or higher) than they intended, and they were free to make corrections. If rankers felt like they needed to make any other adjustments, they were free to do so at this time, but they were under no obligation to do so. If they felt their list was justified and were fine with all of their submission, once they provided that concurrence, I locked their list. Once all mistakes and changes were made for other users, their lists were locked as well. All completed lists were reviewed and locked with the concurrence of the individual ranker.
Once this was complete, I calculated the ranks to find the Top 100 Players of the 2020 Season.
This year I continued to remove 1 high rank and 1 low rank to calculate the average rank.
Link to Player Card Albums & Historical List
Special thanks to /u/rasherdk for providing me with the player cards weekly! It was nice to have them back.
| Rank | Link | Rank | Link |
|---|---|---|---|
| 100-91 | HERE | 90-81 | HERE |
| 80-71 | HERE | 70-61 | HERE |
| 60-51 | HERE | 50-41 | HERE |
| 40-31 | HERE | 30-21 | HERE |
| 20-11 | HERE | 10-6 & 5-1 | HERE & HERE |
And now for a fun little trip down memory lane... Here is the Historical List:
LINK TO HISTORICAL LIST
Master Ranking List
Notes:
-
Column BJ (heh) is the sum of all the ranks for the player
-
Column BK being the average rank of Column BJ.
-
Column BL is the sum of all the ranks for the player minus the sum of the highest and lowest rank for each player. If there are multiples of the same value, it only subtracts 1 of the highest/lowest. This is what is used to find the final rank. The formula looks like this:
=SUM(E2:BH2)-LARGE(E2:BH2,1)-SMALL(E2:BH2,1) -
Column BM is the average value, and thus rank, for the player minus the highest and lowest rank divided by the total number of sheets (56) - it's highlighted in yellow. Essentially, Column BL is divided by the total number of sheets. This is the final average rank for each player. The order, 1-203, is determined from this value, low to high. You’ll notice 4 cells highlighted in green in this column for the Top 125, this was to visually indicate ties to me.
Calculations and Outliers
Explanation
Outliers - Standard Deviation - For Sheet Review: This tab is where I calculated the mean and standard deviations for the player ranks submitted by the individual users. This tab has not been updated with their corrections and is the raw data set prior to reviewing sheets. This is not the sheet where final rankings are determined. Here is the conditional formatting breakdown:
-
Red cells are high ranks that are 2 standard deviations from the mean, as in the ranker was too high on a player strictly per standard deviation.
-
Blue cells are low ranks that are 2 standard deviations from the mean, as in the ranker was too low on a player strictly per standard deviation.
-
Yellow cells are high ranks that are 1 standard deviation from the mean but less than 2 standard deviations as well. These are not outliers and rankers didn’t have to account for them. I simply noted them in case there was an error.
-
Orange cells are low ranks that are 1 standard deviations from the mean but less than 2 standard deviations as well. These are not outliers and rankers didn’t have to account for them. I simply noted them in case there was an error.
Final Calculations - w/Adjustments & Concurrence: This tab is the Final List sheet with all finalized ranks on it after all corrections/changes were made. It’s the same sheet as the Master List linked above. It has the same conditional formatting used on the Outliers sheet to show you the ranks that remained outliers (or not) after the ranking sheet review period. Really, you should only focus on the red and blue cells since they are actual outliers if you are going to pour over this thing. The CHANGE column shows the change in rank (+/-) after all corrections/removals were made from when all submissions were originally locked down.
Final List 2020 - For Comparison: This is the Final List for the 2020 Season from the Master List above. Just cleaned up and added here for comparison's sake.
SHEET REMOVAL
One thing I always talk about in each post is how I will be transparent with all ranks, allowing the sub to see all information. I do this in each reveal, showing where each user ranks a player for that reveal grouping. This is to ensure accountability and to allow you all to see how we arrived at the list order. Last year, I added this sheet review period I presented above to ensure we caught all simple mistakes like a user forgetting to rank a player they would have had in their top 30. Or, more importantly, to catch a joke submission list. We have fun with this list but ultimately the final product is and has to be a real attempt at ranking.
I say all that to disclose to everyone that I removed four user submissions this year. There was no shitpost or meme lists; there were just four lists that were submitted with an extreme number of outliers (+/- 2 std dev) and/or a high amount of extreme outliers (+/- 3 or more std dev) to be included with the final product and weren’t redone. The Outliers - Standard Deviation - For Sheet Review tab on the calculations sheet includes the four removed lists to show the actual outliers from the original version but is hidden to not show the user names or their ranks. This was the original version of the outliers list where the original calculations were done when all lists were submitted. I’m not trying to throw anyone under the bus. I firmly believe shit happens and that’s all that happened here, unfortunately. I’m just sticking towards full disclosure. The Final List does not include those sheets in the final average. I will not share the lists.
CAN THE STANDARD DEVIATION METHOD BE IMPROVED UPON?
Yes, this is a real question we could use an answer on from the sub. r/NFL suggested to us two years ago to use standard deviation when trying to identify outliers. I believe the method works very well as you can see in the data above though I do think it has its flaws. This is largely due to how the ranking average is calculated.
You’ll notice a high number of players have standard deviations in the mid-20’s, or even 30’s, which makes identifying true outliers really difficult. There are a lot of good reasons for this, like Zack Martin for example. When Martin played, he was as great as you’d expect… he just missed a lot of time. That time missed causes a lot of variation among rankers that weigh missed time from eligible players differently as you can see in the data. Thus, it makes their standard deviation difficult to use.
For now, we’ll continue with the sheet review as done above. However, if anyone has a way to improve how we identify outliers without ranking out all players to 200+ let me know! I’m always down for real process improvements.
Breakdowns
Divisions Combined
| Division | Total | Division | Total |
|---|---|---|---|
| NFC East | 8 | AFC East | 10 |
| NFC North | 18 | AFC North | 15 |
| NFC South | 14 | AFC South | 9 |
| NFC West | 13 | AFC West | 13 |
By Team
| Total | NFC Team | Total | AFC Team |
|---|---|---|---|
| 9 | Green Bay Packers | 7 | Cleveland Browns |
| 7 | Tampa Bay Buccaneers | 5 | Kansas City Chiefs |
| 5 | Seattle Seahawks | 4 | Buffalo Bills |
| 4 | Los Angeles Rams | 4 | Pittsburgh Steelers |
| 4 | New Orleans Saints | 3 | Denver Broncos |
| 4 | Minnesota Vikings | 3 | Los Angeles Chargers |
| 3 | Chicago Bears | 3 | Indianapolis Colts |
| 3 | Washington Football Team | 3 | New England Patriots |
| 2 | San Francisco 49ers | 3 | Houston Texans |
| 2 | Arizona Cardinals | 3 | Tennessee Titans |
| 2 | Dallas Cowboys | 2 | Cincinnati Bengals |
| 2 | New York Giants | 2 | New York Jets |
| 2 | Atlanta Falcons | 2 | Las Vegas Raiders |
| 2 | Detroit Lions | 2 | Baltimore Ravens |
| 1 | Philadelphia Eagles | 1 | Miami Dolphins |
| 1 | Carolina Panthers | 0 | Jacksonville Jaguars |
Positions (+/- from Last Year):
| EDGE | IDL | LB | CB | SAF | Total Defense |
|---|---|---|---|---|---|
| 12 (+1) | 9 (0) | 8 (0) | 10 (+2) | 11 (+2) | 50 |
| QB | RB | WR | OT | OG | OC | TE | Total Offense |
|---|---|---|---|---|---|---|---|
| 9 (-1) | 5 (-3) | 12 (-1) | 10 (+2) | 8 (+1) | 4 (0) | 2 (-3) | 50 |
Thoughts / Recap
I just wanted to take a moment and give my own thoughts on the changes and recap this year's version of the r/NFL Top 100 since I largely tend to stay out of the conversation.
1). Potential Modification to Sheet Reviews?I posed the question above to try and find a process improvement. Using standard deviation as a guide to impartially review Top 125 sheets was a great suggestion that came from the sub. At minimum, the process will remain the same for next year; I’m just trying to find an improved way to do it. I think this method can be problematic for players in the bottom portion of the list where you find the most variance. A standard deviation of 25 isn’t the most useful if a player is ranked in the 90s - though not useless. Just looking to see if we can improve here.
2). This was probably my favorite year running this: I thought the list was great last year but it was great to follow it up with another great job this year, especially with the return of Thomas Brady to the list. There are obvious disagreements with the nuance of certain placements but I think on the whole this year's version was great.
3). I don’t really have anything else: AMAA? If there is anything you want me to know, ask away!
4). I don’t have a survey ready. I’m not really sure what to ask. If there is anything you want to see or know or anything at all, speak freely here. I will try to address anything and everything.
Announcement and Goodbye
I’m not trying to make this post about me or anything but this announcement is extremely relevant to the list moving forward. I will be stepping down after 6 years of participating in this, with the last 3 years running the list. I’ve had a lot of fun doing this as a ranker and organizer which makes stepping away bittersweet. Although, I’m definitely excited not to have anything extra on my plate anymore.
Long story short, I’ve been getting busier and busier with my career and am looking to continue to advance. Additionally, I try and focus more on spending time with family and friends as much as possible, internet aside. The increase in work responsibilities has effectively cut into my free time doing things r/NFL related, including this and write ups. Running this list is by no means a time suck - it’s not overbearing. I’m just short on time as it is and don’t want to spend the free time I do have now focusing on this anymore. I originally planned to transition out of this role after next year's list while having a successor shadow me behind the scenes so they can just be familiar with how the sausage is made; I struggled to keep up with the work there is to do throughout this year and decided it’s best to accelerate that timetable up a year. While I won’t be running the r/NFL Top 100 again next year, I’ll be a sort of advisor for my successors before being totally out of running it to ensure a smooth transition. While I loved doing this, it’s just becoming more of a struggle than it should be for me to continue.
Real quick, I just want to thank /u/Mister_Jay_Peg for doing the bulk of the grunt work behind the scenes when he was running this as it made my life incredibly easy. It’s a small amount of time commitment that remains that way since he made it so I didn’t have to reinvent the wheel. I also want to thank the rankers the last 3 years; I had a lot of fun and we did a good job (mostly, also FFF). Also, thank you to the users of r/NFL, especially over the last two seasons. I definitely feel like a lot of the debate and conversations in the reveal threads were about as perfect as you could possibly want on the internet.
With all of that being said, I can assure you all the list is in extremely capable hands. I had two rankers interested in carrying the torch and didn’t want to pick one great user when both were available. There doesn’t need to be a singular person running the list so the fact there are two will ease any possible burdens. The two users who will manage the r/NFL Top 100 moving forward are:
/u/mattkud and /u/packmanwiscy.
I’ll be around in this thread answering as many questions and taking suggestions as they come up but also ask these two as well.
Closing
The book is now closed on the r/NFL Top 100 Players of the 2020 Season. I apologize for the delay in getting this posted but I have been pretty busy.
Keep a lookout for the Call for Rankers sometime around the Super Bowl if you would like to join next year.
Also, thanks for all the fun!
Go Birds.
Walang komento:
Mag-post ng isang Komento