Figure Making Methodology II

[TGJB]

Recently, Jason Litt posted some comments and questions about figure making. We think this is important stuff, so we\'re going to post it again, along with my response.

TGJB

Author: Jason L.
Date: 06-07-02 15:33

We keep mixing up \"science\" and \"statistics\". Science, as I am using it, consists of objectively verifiable facts. A horse going x wide means that it ran x further distance. A wind of x speed will impede the progress of horse weighing y pounds by z amount. A horse carrying x pounds is impeded by y amount. Statistics is a tool of science to take past outcomes and predict future outcomes. By definition, such analysis cannot be proved \"correct\" it can only be proved to be accurate withing a certain range of confidence. The more data you have, the more confident you can be.

Now, nobody makes a variant based upon science. That is because it is impossible to do so. You can only make assumptions based on science and engage in a predictive statisitical analsyis.

To the extent that I believe more in Ragozin\'s method than yours, it is that I believe their statistical analysis and assumptions are more rigorously derived. Their analysis is based upon more data points and is not adjusted for any \"anomoly.\"

Based on what you have said, you really do not make a track variant (at least sometimes). By a track variant, I mean that based on a statistical analysis of the races run that day, compared with past races, you come up with a number that normalizes any figures to some predtertimed baseline. That variant is than plugged into an algorithm with the raw number to get an adjusted number. If that variant is to change during the day, which should be rare, it should do so in a consistent manner.

You seem to think that it is unnecessary (at least sometimes), to even have a variant. That is because you look at the horses who ran in that particular race and determine what number they are most likely to run and then use that baseline to adjust the numbers in that race based on position. When this does not conform to what a consistent variant would tell you, you basically make the number consistent by changing each race based on your \"judgment.\" You then explain it by variations in the track, as that is the only way it could be explained, therefore it must be true. Math and statistics are thrown out the window. My quibble with you, is that I do not believe that to be an accurate way of determining numbers in the long run. In a particular race or a particular day, it certainly may be more accurate, particularly where the statisitical analysis is compromised by limited data (this is one reason why you always attack Ragozin on Triple Crown, Triple Crown prep and BC days because the normal claiming races are not being run and you have races being run at distances that are not normally run at the track, thus the data is usually limited on these days).

As to your issue of changing numbers, I really do not see a problem. All that is doing is recognizing that based on future events, the statistical analysis now shows that they were probably wrong. Again, they were probably wrong because of the limited available data, thus more data helps them get it correct. If it happened a lot, it would concern me because it would demonstrate that they were wrong a lot. When it happens once in a while, it confirms that the number maker is being honest and rigorous.

As for the tight ranges, the entire debate is a red herring. By definition, if you get rid of \"anomolies\" you are going to have a smoother pattern. If you want me to find a professor of statistics to prove this, I will. Now your point is that, if you were wrong, eventually your numbers would fall apart because the \"anomolies\" would turn out to be true and your future numbers would drift back toward the \"amomoly.\" David\'s point is, maybe that is true, but there would have to be a lot of anomolies and even then, it would take a long time to surface.

Now, saying that you got one race, the Peter Pan to provide a smooth pattern for all horses does not prove a damn thing about your methodology. At most, it proves you got the past numbers of those particular horses correct, and given that most of those horses had only run a few races, it doesn\'t even do that. If you turned over all of your sheets over the past decade, identified those races where you adjusted for \"anomolies\" and then had an expert statistician determine whether these number hold up, then maybe I would give your theory some credit.


Okay—

1)   I’m not mixing up anything. You came in late to this argument—I know you know the difference. My comments were aimed at David, who has referred to “factual data”, and that a good “methodologist, physicist or chemist could verify the rate of evaporation over a 2-3 hour period.” He also said “we’re talking physics and chemistry here, the effects of moisture are demonstrable and consistent,” and “given that they have a pretty good way of measuring the firmness of a course… you’d think that you could do better than just the assertion that you made and use some real data.”

    The point I was making to David (and again, I know you get it) is that there is no such thing as “factual data”, and going back to change numbers proves that Ragozin knows it, and doesn’t base his figures on “science”.

   By the way, some of the stuff you said was science is not. Both wind and weight adjustments are ultimately determined by trial and error, and the weight one is yet another average, since they won’t give us the weight of the horses. And wind adjustments are based on an ESTIMATE made before and/or after the race, and which assumes the wind is constant throughout the race, and only estimates the effect of Grandstand and other structures (watch the infield flags at AQU sometime—they point at each other).

2)    Your belief that Ragozin’s statistical analysis and assumptions are more rigorously derived is an assumption on your part—give me evidence. For starters, assumptions are not conclusions, they are assumptions—I use as few as possible because they are not provable. And basing an analysis on more data points is only constructive if the sampling actually is appropriate and the results applied correctly—if, for example, you were going to create a variant level for NWI allowance races at Belmont, you would be better off using a smaller sample (NYRA races) than a larger one (all Eastern tracks) because the NYRA horses are faster.

   What Ragozin does is rely heavily on averages, and I’ve explained before the flaws in that. You have to be very careful about correlating independent events—Mark Twain even wrote a story about it, the gist of which is:
       Scientists were testing a champion jumping frog. They said “Jump, frog”, and the frog jumped 4 feet. So they wrote down, “frog with 4 legs jumps 4 feet”.
    Then they cut off one of his legs, and said “Jump, frog”. He jumped 3 feet, so they wrote down, “frog with 3 legs jumps 3 feet”.
   They cut off another leg, said “Jump, frog”, he jumped 2 feet, and they wrote “frog with 2 legs jumps 2 feet”.
   They cut off another leg, said “Jump, frog”, he jumped 1 foot, they wrote “frog with 1 leg jumps 1 foot”.
   They cut off his last leg, and said “Jump, frog”. Nothing. They said it again. Nothing. So they wrote down, “frog without legs is deaf”.

   See if you can identify (or get them to articulate) the actual list of assumptions (working hypothese) Ragozin uses. Then let’s discuss them, one by one. I think this will be extremely productive.

3)   Paul is right—boy,  do you guys ever not understand what I or Ragozin or any other serious figure maker does. Whaddya think, before the first big race War Emblem ran (when he jumped big) I said, you know, I think this horse is ready to run a 1? Our choices in making variants are prescribed by the relationships between the horses, in figure terms. Once you adjust a race for weight and ground, the relationship between the horses in that race are fixed—i.e., the second horse has to get ½ point worse than the winner, but 2 points better than the next horse, etc.  I can’t say this horse figured to run a 6, this one an 8, this one a 13, unless the relationships between the 3 horses come out correctly. That’s why I keep sending everyone to look at the Peter Pan—it’s a solid block. If you change one, you have to change them all, and they all would come out funny (wrong). And again, the figures they ran in the Peter Pan go a long way in confirming the figures they ran back to—the same premise that Ragozin uses in going back to change earlier figures.

   So—as it happens, the Peter Pan fit tight with the day. But if it didn’t, if I had to add or subtract relative to the surrounding races I would have, BECAUSE IT IS SO OBVIOUSLY RIGHT. Again, look what happens if you do that race any other way. War Emblem’s first jump forward is another, slightly different example. You could either give him the jump and the others about what they run, or you could give him a slower number, and have the whole rest of the race collapsing badly. The first is much more likely. This situation goes on all the time, and that’s how you end up giving horses new tops.

   All of which is not to say I don’t look at surrounding races. I do, and most days fit, once you separate the apples and oranges (1 turn and 2 turn races)—not always as a steady variant, but sometimes as a steady identifiable slide, or clear split (this happens a lot in Chicago).

   But you are wrong when you say I explain it by variations in the track, “as that is the only way it could be explained”. Just off the top of my head, final time can be affected by pace, wind gusts during the race, humidity, shade, temperature, and the relationship between different soil compositions with several of the above. I urge you to read and think about my post “Changing Track Speeds”, 11/17/01, but to look at one mundane example, “ordinary” race to race maintenance can affect the track differently on different days. If the same amount of water is added between races, the track will dry out much more on a hot windy day than on a cool calm day. There is no formula for knowing what effect that will have on final time (which will differ with different soil compositions), so you have to determine the effect by looking at the figures of the horses that ran over the track.

   One of the differences between me and Ragozin is that he states out of hand that this is not true (which is an assumption, and obviously false, or at least arguable, to anyone who looks at it—other than by the horses, what would you base it on? And even if you had exact details of water, temperature, wind, soil composition, etc., and it didn’t have an effect, how could you know that another combination would not?). I don’t make any assumption about it. I use surrounding races (and to a lesser degree surrounding days and previous sprint/route relationships) as information bits (data points, in your terms, I guess)—but no more, no less.

     Incidentally, the reason I compare and call attention to big days is a) that’s when Ragozin posts whole cards, with the numbers they ran, and I see them, and b) even when he doesn’t (Wood Memorial etc.), it’s a race everyone has looked at, and from which important future decisions will be made. I hope you weren’t going where I think about the claiming races because if you were there would be deaf frogs all over the place, among other problems. If you want to state whatever that position is outright I’ll deal with it. I’m assuming you are not saying that without claiming races as a crutch Ragozin is at a disadvantage in assigning figures on big race days.

4)   For reasons I stated, I don’t think going back to change numbers is a great idea, but that’s not the point I was making. My point is that doing it confirms my premise of how figures should be made (fitting them together), taking it even further than I would.

5)  What you guys don’t understand—because you have never spent any time making serious figures—is that YOU CAN’T GET RID OF THE ANOMOLIES—YOU CAN JUST MOVE THEM AROUND. I can’t both give the horses behind War Emblem that day what they usually run and avoid giving him the jump—whatever I do, I’m going to have an anomaly. And if I screw up a race, either by doing what you think I do or by just blowing it, I’m going to have a lot of anomalies down the line, as those horses show up in races where they won’t fit as well. Do this often enough (let alone on the every race basis you guys think) and you will have nothing but anomalies—the situation will be the exact opposite of that which exists within the Thoro-Graph data base.

   Again, look at the Peter Pan. Those horses came out of different races, and different tracks. Think about the practical ramifications of what you suggest—if I had fudged the earlier figures, there ain’t no way I could have made it fit like that, and if it fits like that, there ain’t no way it’s wrong.


   If anyone is interested in learning more about these subjects, there are several good posts:

Figure Making Methodology (this site, 5/2/00)
Changing Track Speeds (this site, 11/17/01)
Split Variants, Changing Tracks: Bruno’s Take (this site, 1/3/02)

And one Alydar just pointed out:

An interview with Del Mar track superintendent Steve Wood at the Del Mar website.