The RTT computer BlogPoll ballot: the raw numbers

by Joel on Sep 28, 2008 4:55 PM EDT in College Football Comment 27 comments

[Note by Joel, 09/28/08 9:43 PM EDT ] Change in plans. Some big news about the BlogPoll on the horizon, and there is now a deadline for draft ballots as well as the Wednesday deadline for final ballots. I'm going to go ahead and make some of the revisions. If y'all have any, try to leave a comment before 10:00 tomorrow.

Time for yet another iteration of the RTT Ready, Fire, Aim BlogPoll ballotty thing. For a complete refresher on what we're doing, go here, here, here, and here.

Adjustments for this week:

I'm using hooper's "square root of the sum of the squares trick" because frankly, it just sounds impressive and it's a fine way to limit the criticism to only other mathy types. What non-geek is going to argue with that? Divide and conquer, say I. I'll work up something out of a law journal, preferably in latin, for next week, and hooper and I will have conquered the world.
Seriously, hooper's infatuation with squares seems to make a lot of sense, and the results looked more reliable than the other way, so there. That's the only tweak this week.
The categories again: Win/Loss, Strength of Schedule, Pass Efficiency Defense, Rush Defense, 3rd Down Offense, Total Defense, Pass Efficiency Offense, Experience, Opponents Points Per Game, Total Offense, and 3rd Down Defense. The AVG R column is Average Ranking, and the last column is Weighted Average.
The current weighting of the categories: 30, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1. Then tweaked by the square root of the sum of the squares manuever.
Current spreadsheet here.

All right, then. The Top 35, according to the computer, and for those of you that are new this week, no, this isn't our BlogPoll ballot. It informs our eventual BlogPoll ballot:

Rank	Team	WL	SOS	PED	RD	3DO	TD	PEO	EXP	OPPG	TO	3DD	AVG R	W AVG
1	Oklahoma	1	31	4	12	18	15	3	59	13	7	11	15.82	39.55
2	Utah	1	22	61	5	28	5	16	14	41	28	23	22.18	58.91
3	Alabama	1	41	18	3	14	18	42	95	15	57	13	28.82	58.97
4	Kansas	20	13	13	21	4	35	17	32	18	12	47	21.09	63.83
5	Texas	1	55	40	2	12	26	2	116	6	10	14	25.82	75.02
6	Florida St.	20	5	35	8	21	3	76	53	3	38	3	24.09	75.13
7	South Fla.	1	83	17	4	20	8	31	66	28	16	26	27.27	83.25
8	Missouri	1	55	58	24	17	79	5	32	47	2	45	33.18	85.79
9	Boise St.	1	2	54	73	29	44	6	114	20	19	12	34.00	86.41
10	TCU	19	53	31	1	9	6	96	26	14	44	10	28.09	88.13
11	Georgia	19	7	69	6	27	22	30	90	39	30	37	34.18	88.22
12	Duke	20	31	14	71	22	33	56	21	26	61	8	33.00	89.26
13	Penn St.	1	97	10	15	5	14	13	38	11	9	9	20.18	90.97
14	Wisconsin	20	36	16	34	67	30	47	77	30	37	79	43.00	91.21
15	LSU	1	83	24	10	54	19	39	77	18	27	22	34.00	92.62
16	Ohio St.	19	55	21	22	66	12	50	59	23	83	56	42.36	93.88
17	Vanderbilt	1	41	26	45	74	73	65	90	30	112	65	56.55	95.83
18	California	20	41	11	29	80	28	44	90	37	20	39	39.91	96.51
19	Nebraska	20	11	55	35	85	64	12	32	40	33	70	41.55	101.26
20	Georgia Tech	20	55	8	63	36	25	52	109	17	43	95	47.55	103.03
21	Northwestern	1	92	36	36	44	41	86	59	9	67	15	44.18	109.47
22	BYU	1	112	27	33	2	27	8	49	7	10	68	31.27	109.69
23	Wake Forest	20	46	5	74	89	24	46	49	30	92	78	50.27	110.31
24	Air Force	20	46	67	55	30	48	102	7	30	85	44	48.55	113.39
25	Michigan St.	19	55	29	44	84	63	80	68	28	42	34	49.64	113.45
26	Oklahoma St.	1	109	33	56	8	42	4	45	53	4	7	32.91	115.09
27	Texas Tech	1	113	28	25	16	61	23	68	23	3	4	33.18	115.70
28	Notre Dame	20	25	32	60	86	85	54	95	34	88	57	57.82	115.72
29	Kentucky	1	106	3	13	77	4	89	38	1	73	5	37.27	117.47
30	Ball St.	1	81	20	102	3	84	9	53	45	13	83	44.91	117.92
31	North Carolina	20	78	42	65	45	56	18	72	47	80	39	51.09	119.64
32	Florida	20	103	15	17	37	7	33	116	10	71	21	40.91	121.64
33	Minnesota	19	50	45	70	81	75	24	112	49	58	52	57.73	122.96
34	Arizona	20	111	2	41	25	2	26	77	21	32	6	33.00	123.67
35	Cincinnati	20	55	99	31	37	68	14	59	63	29	89	51.27	123.95

That's starting to look pretty good, I think. A few differences with the AP and Coaches polls:

The computer is bullish on Utah and Kansas, which are 15 and 16 respectively in both traditional polls and 2 and 4 in our modest little computer ranking.
Ditto Florida State, Boise State, TCU, Duke, Cal, Nebraska, and Northwestern. Why?
The computer is bearish on Penn State, LSU, and BYU. Why?

Let's take a closer look at some of these differences.

Utah. The Utes are 5-0 against Michigan, UNLV, Utah State, Air Force, and Weber State, a schedule that ranks 22nd in the country. They rank fifth in rush defense and total defense. They rank no worse than 28th in any particular category above with the exception of pass efficiency defense, and their average ranking in those key stat categories is 22. Is that better than Alabama, Texas, or Missouri? Maybe not Alabama, which has beaten No. 9 (at the time) Clemson and No. 3 (at the time) Georgia, but can we really say that Texas' or Missouri's resumes to date are clearly better?
Kansas. The 3-1 Jayhawks have been knocked for their 34-37 loss to South Florida, which was No. 19 at the time and is currently No. 10. Kansas' other wins have been fairly dominating, but they've come against Florida International, Louisiana Tech, and Sam Houston State. I'm not quite sure what to make of their 13th ranked strength of schedule because that certainly doesn't sound very difficult. I'm thinking the pollsters are right on Kansas and the computer's wrong. Is a tweak necessary to correct this? If so, what?
Florida State. The Seminoles are 3-1, with whoopin's of Western Carolina and Chattanooga, a victory over Colorado, and a loss to Wake Forest, which was No. 18 at the time and is currently 25th. And here's where we identify the trend of error of the computer rankings. The strength of schedule data is wack. How is that the 5th-most difficult in the nation? Any ideas here? Strength of schedule should be a heavily-weighted factor, I think, but I'm skeptical of the data. Perhaps the cumulative opposition metric is more accurate? It would seem that past opposition would be more in line of a resume system, but what do y'all think?

Okay, so let's do essentially the same thing this week that we did last. Rather than me doing more issue-spotting of the computer results and implementing the human interventions myself, you all implement the interventions.

Let's say we can change ten things in the top 25 above. What would you change? For instance, one revision might be "move Alabama above Utah." Another might be "move Kansas to 15." You get the idea. Yes, it will get messy, but we'll deal with that as well as we can. I won't actually submit the ballot MGoBlog's site until we're somewhat comfortable with it.

So have at it. Ten revisions. Go.

Current draft:

Rank	Team	Delta
1	Oklahoma	5
2	Alabama	5
3	Utah	8
4	Texas	4
5	Penn State	13
6	LSU	3
7	Missouri	2
8	South Florida	6
9	Brigham Young	1
10	Georgia	6
11	Vanderbilt	9
12	Texas Tech	14
13	Kansas	4
14	Boise State	5
15	TCU	2
16	Wisconsin	7
17	Florida	15
18	Ohio State	2
19	Northwestern	7
20	Virginia Tech	6
21	Oregon	5
22	Florida State	4
23	Wake Forest	8
24	Nebraska	12
25	Duke	1

Dropped Out: Southern Cal (#1), Ball State (#21), Clemson (#22), Illinois (#23), Colorado (#24), Fresno State (#25).

0 recs | Comment 27 comments |

Story-email Email Printer Print

Comments

My quick stab at revisions

1. Move ’Bama above Utah.
2. Move all non-1 W/Ls to the bottom except for Georgia.
3. Lower Florida State, Boise State, TCU, Duke, Cal, Nebraska, and Northwestern due to the funky SOS ratings.
3. Bump Penn State, LSU, BYU for the same reason.
4. Tweak from there.

Because it’s getting late, I’m going to go ahead and post the draft ballot in the main section of the article above. Comments are most definitely not closed though — we’re to continue to solicit feedback and revise as appropriate until the final is due on Wednesday.

Go Vols!

by Joel on Sep 28, 2008 9:57 PM EDT reply actions 0 recs

Aw, you're no fun...

Why not let the computer have its way? Brian’s far more entertaining if you do. :-D

by hooper on Sep 28, 2008 11:49 PM EDT up reply actions 0 recs

Yikes, Duke hiding at 16

A loss to Northwestern and wins over James Madison, Navy, and Virginia? Good enough for ranking, maybe, but not at 16. Moving and re-posting, so you’ll see the revised ballot above.

Go Vols!

by Joel on Sep 28, 2008 10:28 PM EDT reply actions 0 recs

Strength of schedule:

It is odd. Chattanooga also played Oklahoma, IIRC, so that might be a part of their “toughness” on SoS. Colorado also beat West Virginia. I don’t know who W. Carolina played and I don’t care enough to look, but I could see a road game or two for cash that bumps their apparent “strength”.

I think when the conference games start up in earnest, you’ll see that balance out somewhat. Perhaps reducing SoS importance would help, but I don’t know enough about the numbers to explain further.

by hooper on Sep 28, 2008 11:53 PM EDT reply actions 0 recs

SoS problem

If your SoS metric only measures who you play, and not who you beat or at least play well against, then that could be the problem. Maybe you should come up with a “Quality of Wins/Losses” ranking (not to be confused with the pointless “Quality Wins” bonus in the old BCS computers rankings). If I play three good teams and lose to them all, but dominate my worthless competition, I could be ranked pretty high in your poll because I would have pretty good numbers on offense and defense (from dominating cupcakes), and a high SoS (from playing those three really good teams), even though the SoS has proven to the humans that I’m not in the same league as the higher ranked opponents. Win % doesn’t reflect this, since it’s just an average and doesn’t take into account to whom you lost. Maybe if you weighted Win % with an S-curve factor based on the ranking of the opponent? Say, the average team is worth one win, the number one team in your rankings is worth 1.5 and the worst team in your rankings is worth 0.5, and the converse for losses? Then you’ll have an explicit function for how much you value a win which will help give teams credit for beating their tougher opponents, and punish them for losing to inferior opponents. Couple that with a Quality of Wins factor in your SoS and I think Kansas would move down.

by Horn Brain on Sep 29, 2008 10:21 AM EDT reply actions 0 recs

Interesting.

There’s still the problem of a seemingly faulty source for the SoS numbers themselves, but that would effectively create a Strength-of-Win and Strength-of-Loss system.

Definitely worth considering.

by hooper on Sep 29, 2008 10:41 AM EDT up reply actions 0 recs

Revisions

1: Drop-kick FSU out of the top 25. Seriously. They’re terrible. Don’t let the win over Colorado fool you. I’d say drop ‘em out of the top 35, but I suspect that won’t fly.
2: On a similar vein, let’s play roulette: move Kansas to LSU’s spot, LSU to FSU’s old spot, and shift everyone up 1 to make up for it.
3: TCU: way down. Don’t care where, but I think USC should be where UGA is, and UGA should be where TCU is. (Yes, TCU can be where USC is.)
4: Penn State and Boise State: swap.
5: What to do about Utah? I’d be in favor of a categorical drop to, oh, 15 or so, but since we’re playing the switch game, change ’em with the now-new Penn State ranking, then swap PSU and Alabama.
6: TTU should probably be swapped with Cal.

On SoS: drop 1-AA teams? Or have them as a default ranking of 121 or something. I suspect that’s the problem with your teams, and quick analysis shows:
Utah: 1 1-AA team
Kansas: 1 1-AA team (and FIU, who kind of counts but not for these purposes)
Florida State: 2 1-AA teams
Boise State: 1 1-AA team
TCU: 1 1-AA team
Duke: 1 1-AA team
Cal: 0 1-AA teams
Nebraska: 0 1-AA teams
Northwestern: 1 1-AA team

So it looks like a correction factor there could help.

by Graysnail on Sep 29, 2008 7:28 PM EDT reply actions 0 recs

SoS: Further Note

Really, what may help is applying a correction factor across the board for ALL stats generated against 1-AA competition to account for comparing across disparate difficulty levels. What I’ve done before is apply a .5 factor to this (i.e., 250 rushing yards offense = 125 “rushing yards”). Where it becomes useful is divide by the factor for defensive metrics. I don’t know what the factor should be – offhand, I’m thinking 0.85, but I don’t know.

by Graysnail on Sep 29, 2008 7:33 PM EDT up reply actions 0 recs

Hey, Graysnail

Welcome, and thanks for the input.

What do you think, hooper? And is it even doable with our spreadsheet without turning it into a full-time job?

Go Vols!

by Joel on Sep 29, 2008 8:06 PM EDT up reply actions 0 recs

Actually, now that I've thought about it....

What should probably be done is a rolling correction factor across all teams, going from 1 at the peak to 0.85 (or some such) as teams approach the 1-AA limit (121). Actually, it should probably be some kind of non-linear distribution, as it makes some sense to assume that teams concentrated in the “average team” region (I’d say ~50 – ~105) are similar. But that’s neither here nor there, and I’m not sure if that can be done in Excel.

If that can’t be done, maybe a staggered one across the rankings (1 for 1-50, .95 for 51-100, .9 for 101-120) could work. Basically, while I think this would help a lot, this’ll also correct the FIU issue – where you get full credit for playing a team that should rightfully be considered 1-AA for all intensive purposes.

by Graysnail on Sep 29, 2008 8:17 PM EDT up reply actions 0 recs

And yes,

thanks for the input.

by hooper on Sep 29, 2008 9:35 PM EDT up reply actions 0 recs

1-AA teams have always presented this problem.

There are several ways of dealing with it. Colley groups them as if several 1-AA teams were one team. (That allows him to group their results such that the 1-AA group has the same number of games played against 1-A opponents as a normal team.) Other people rank them (either separately or by performance v. 1-A) and place them behind all the 1-A teams. (That’s a little unfair to Appy State, but hey.)

Whether a fix like Graysnail’s is doable depends on how you get your SoS data. If it’s a list from, say, NCAA.com or ESPN, it’d be a lot of work to fix. If it’s something you’re already generating on your own, then it wouldn’t be as difficult. The lazy method would be to have it search the opponents name. If an opponent isn’t in the list of 1-A teams, then adjust the performance accordingly.

Another method would be to manually keep track of the number of 1-AA teams played by each team. Then there are several corrections available. But it all hinges on how you get your data in the first place.

by hooper on Sep 29, 2008 9:33 PM EDT up reply actions 0 recs

Hmm

It’s coming from here. It’s not automatic like the non-pdf pages, but it’s only a little more difficult in that you can actually grab and paste even though it is a pdf.

I don’t think that page has the data you’d need to do what you’re thinking, though, right?

Go Vols!

by Joel on Sep 29, 2008 10:12 PM EDT up reply actions 0 recs

Give me a sec.

by hooper on Sep 29, 2008 10:24 PM EDT up reply actions 0 recs

I suppose

That the inputs are assuming that you:

a) have the ability to track a team’s previous games played by team name.
b) are generating your own SoS numbers.

Really, I kind of slipped in the second one, and I would actually suggest that you not pull from the NCAA’s SoS rankings if possible. Unfortunately, I don’t know of any way to retroactively include SoS numbers unless you pull from another ranking system – Massey, perhaps – and base team SoS on a summation of the previous teams’ rankings or some such. More importantly, I suspect that’d be a headache and a half.

…while I’m at it, I freaking hate Colley as a ranking system. Hate it.

by Graysnail on Sep 29, 2008 10:30 PM EDT up reply actions 0 recs

Heh.

There’s no middle ground on Colley. I like it, if only for the mathematical curiosity. It’s not particularly useful at this time of year, but I don’t care much for any poll at this time of year, so there you go.

by hooper on Sep 29, 2008 10:34 PM EDT up reply actions 0 recs

The NCAA sheets already subtract the 1-AA games.

They eliminate those games entirely from both the team in question and the opponents of the team in question. For example, Utah has played 5 opponents. If you ignore Weber State, that leaves 4 1-A opponents. Of those 4 opponents, they have a total record of 9-8. Subtract the games played against Utah and the cumulative opponent record is 9-4. Subtract the one 1-AA game (Air Force v. Southern Utah – a win for USAFA) and you get a record of 8-4. The SoS score for Utah is 0.6666667, which corresponds to a 2-1 winning ratio, which is the same as 8-4.

So the 1-AA teams are already factored out.

by hooper on Sep 29, 2008 10:32 PM EDT up reply actions 0 recs

Oh, one more thing about the NCAA pdfs.

The “cumulative” SoS’s include strength of schedule data for future teams. You probably don’t want to use those, since future opposition doesn’t have anything to do with current resume. By the end of the year it won’t matter, but it’ll give you wonky results right now.

by hooper on Sep 29, 2008 10:51 PM EDT up reply actions 0 recs

Yeah

I’m only using the “past opposition” column. I use that link b/c it’s sorted by rank; the others are sorted by some bizarre code that’s almost, but not entirely alphabetical.

Go Vols!

by Joel on Sep 30, 2008 6:11 AM EDT up reply actions 0 recs

Ah.

That other code is the NCAA’s coding system that they use to keep track of teams in databases. It was alphabetical when they first instituted it, but the addition of teams to 1-A since then (whenever it was) caused a few to fall out of order.

by hooper on Sep 30, 2008 8:17 AM EDT up reply actions 0 recs

Again, thanks for the input

My tweaks based on the above:

*I’m comfortable with FSU at 22. That’s significantly lower than the No. 6 the computer spit out, and dropping them out altogether — just not comfortable with that unless there was some more compelling information.
*Move one-loss Kansas to LSU’s spot and LSU down to 22? Why? I don’t see it.
*I am skeptical of TCU’s numbers. The computer puts them at 10, and I have them at 15. I suspect you’re right, though, so I’ve bumped them down a few more spots.
*Agree on Southern Cal, mostly, but I think their loss was worse than Georgia’s so I put them just below Georgia.
*Penn St. and Boise St. I thought this sounded wrong until I looked at it, and I think you’re right. Maybe not an outright swap, as it would put Boise at No. 5. I moved them closer together and put Boise ahead of Penn.
*I’ve moved Utah down out of sheer human skepticism, but not down to 15 b/c the numbers are strong.
*Texas Tech — bumped down a couple of spots.

Thanks again for the great feedback.

Go Vols!

by Joel on Oct 1, 2008 6:32 AM EDT up reply actions 0 recs

Maybe I'm missing something with all the computer stuff above but...

USC’s out while Florida is in? I am curious as to the logic?

Look, SC is not off the hook for their terrible performance on Thursday against Oregon St. Far from it. They have a lot to show me (and everyone else who cares about college football) in the coming weeks if they expect to get to a major bowl game, but when compared to Florida’s loss AT HOME to Ole Miss, a team that went 0-7 in the SEC last year, or UGA’s loss AT HOME to a surging Alabama who stumbled in the 2nd half against UGA and who scored only one offensive TD against a lowly Tulane just a few short weeks ago.

Like I said maybe I am missing something…

Conquest Chronicles

by Paragon SC on Sep 29, 2008 8:23 PM EDT reply actions 0 recs

Looking at the spreadsheet:

Something is screwy with USC’s W/L. They have a score of 48, which places them behind 47 other teams. They only have 1 loss, but they’ve only played 3 games. that gives them a 0.667 winning percentage, which places them behind 47 teams that are either undefeated, have a record of 3-1, or have a record of 4-1. Unfortunately, the ranking system gives them a value closer to teams with a record of 3-2 (WL = 50) than teams with a record of 3-1 (WL = 20).

This is a fault of the early season byes that USC has had. Because of this, USC should probably be arbitrarily bumped via the manual override into a reasonable place until the bye weeks catch up for other teams. By the end of the season (when this actually matters), this won’t be a problem. Until now, blame the schedule.

by hooper on Sep 29, 2008 9:24 PM EDT up reply actions 0 recs