Bad Scyence

I'm an engineer. I have a degree in Materials Science and Engineering from one of the better such programs in the country (Lehigh University). This and $0.99 will get me a Coke. Sometimes. It doesn't qualify me to be a baseball analyst, but is is an indication that I have a pretty logical, analytical mind and that I have some idea of how to practice science and conduct experiments.

One of the first things that any analyst or scientist conducting an investigation must do is try to identify biases in his/her experiment, and either eliminate them or find a way to compensate for them, usually by including the analysis of an effectively neutral subject to go along with the analysis of the subject of interest. This, as you may know, is called a control, and it serves to put the data gleaned from the experiment into a context, so that meaningful conclusions can be drawn from them. Without such measures, the experiments are, well....bunk.

Jayson Stark is a sports writer. Not an engineer. He formerly wrote for the Philadelphia Enquirer, and was a pretty good writer, by most accounts, and certainly a heck of a lot better than Bill Conlin. Now he does a lot of "numbers" columns for ESPN, between his appearances on Baseball Tonight, finding weird and wacky statistics about which to muse, but rarely writing anything of substance. Often these stats are misleading, too often to take him to task every time he does so. But this particular case caught my attention because I heard it repeated during an ESPN broadcast of a game on Sunday and because it may reflect an intrinsic bias.

His column on who he thinks should win the major awards in each league included the following, with regard to the AL manager of the Year issue:

"...let's remember that Scioscia's team finished more games out of first place last year (41) than the Devil Rays. Biggest jump since the 1898-99 Brooklyn Bridegrooms."

He is, of course, not wrong. Last year's AL West standings finished up this way:

_Team___ W - L _ GB
Seattle __ 116 - 46 0
Oakland _ 102 - 60 _14
Anaheim _ 75 - 87 _ 41
Texas ____73 - 89 _ 43

This year's looked like this:

_Team____ W - L _ GB
Oakland _ 103 - 59 _ 0
Anaheim __99 - 63 _ 4
Seattle ___ 93 - 69 _10
Texas ____ 72 - 90 _31

Now certainly, an improvement from 41 games back to 4 games back is a significant improvement, but Stark points this out as though first place were a constant, and ist is most certainly a variable, an anyone with half a brain and one eye can see that the same team is not in first place this year as last, and that their respective records are quite different. Sure, Anaheim improved 37 games in the standings, but this was accompanied by the Seattle Mariners precipitous fall from the status of Ungodly Great to a more pedestrian Darn Good, a 23-game drop in their record. So really, the Angels improved by 24 games from their record last year, which is a big jump, but not as unprecedented as Jayson would have you believe.

As I see this, there are two major problems with Jayson Stark's argument:

The first is that he takes an accurate statistic and completely removes it from any relevant context, in order to give it more apparent weight than it actually should carry. Isn't it enough that the Angels improved their own record by 24 games? Do we have to pretend that the coincidence of this fact with the comparable decline of another team's record somehow makes it more of an accomplishment? This, in engineering terms, is an experimental measurement without any control group. Well, actually, Oakland is almost a Control, having finished within one game of their 2001 record. But Stark never mentions them.

The other problem is that Stark may have an inherent bias in that he works for ESPN, a company owned by Disney, who also owns ABC, and....you guessed it! The Anaheim Angels! So how seriously should we take the word of a man touting the merits of one of his fellow employees? I don't know. Perhaps a little more seriously than we would take Garret Anderson's word for it, but perhaps not as seriously as we might take that of someone like George Will.

This brings to the fray another question: What kind of integrity can/should we expect from an individual working for a company that is so closely attached to one of the organizations being analyzed? Or is it just the fact that Stark, like Peter Gammons before him, has kind of sold out what got him there in the first place, and stopped trying to really do tough, investigative and relevant journalism? I don't know.