GeoForecaster's Michael Kozuch and Lowell Whiteside don't charge much for their earthquake forecasts. It might be argued that $9.95 a month doesn't harm anyone. For their money, customers get long lists of little earthquakes most of the time. Kozuch and Whiteside aren't harming their scientific reputations—they don't have much of any. They aren't very smart Web marketers, hiding all of their methods and not allowing a free look at the product.
The Harm of Poor Predictions
But there are two kinds of harm in their approach. First, some day they might issue a forecast for a large, damaging quake (although they haven't yet), and the authorities will have to judge their claim somehow as the clock ticks and the press clamors at the door. The longer they do business, such a forecast would be inevitable, but after three years selling their forecasts (except in California, where they are not licensed to practice) they have not tried to warn officials. Perhaps they would rather avoid such a forecast—and what would that say about their confidence in their methods?
Second, the public is led to believe that GeoForecaster is using the rigorous tools of science. As we've seen in the case of evolution, science has a precious reputation at stake that is easily degraded by a public that believes what pleases it. Keep those points in mind as I spell out the faults I see with the structure of their statistics. (You can confirm what follows from the archives of their Web site.)
Five Flaws
The things I'm pointing out might seem subtle, but people with PhD's can point them out without a moment's thought. As Kozuch and Whiteside both have doctorates, these flaws can only be deliberate choices to use substandard methods.
- They count random events as successful forecasts. Consider that in the regions they cover (California, Japan and Taiwan), earthquakes of all sizes occur constantly. Forecasting small quakes there is like forecasting waves at the beach. It is rare that a forecast will not include something they can call a complete success, and we know this because they proclaim an overall success rate of 83 percent!
- Their criterion for a successful location is flawed. Instead of measuring the distance from an earthquake directly, as the radius of a circle, they use a longitude-latitude box. This allows them to call a "hit" an event as much as 1.4 times farther than the radius. (See the illustration.)
- They assess latitude and longitude separately. That way they can declare an "A" quality success for matching an event's latitude, even if the longitude is completely wrong. This procedure is not justified by scientific practice—or even common sense.
- There is a big loophole in the magnitude criterion: an event larger than the forecast can be counted as a success. Considering that all of southern California should be carpeted with forecasts for small quakes (GeoForecaster's go down to M2), magnitude-5 events like the quake of 22 February 2003 will always score as successes, even if they weren't predicted. Thus Kozuch and Whiteside claim over 90 percent success for earthquakes larger than magnitude 6.5, yet they don't say they actually issued forecasts for events of that size.
- They commit the basic statistical distortion of turning a semiquantitative measure of success (their 16-point ABCD metric) into a quantitative "AccuCast" index (0 to 100%), introducing false precision. It's the same error as reading the length of a car trip from your dashboard gauge, then reporting it in centimeters.
Still, if people think it's useful to get a weekly newsletter that claims to announce every magnitude-2 event in advance, within 5 days and 80 kilometers, then they can pay their money and enjoy the illusion as long as GeoForecaster stays in business. Not me. And the state of California agrees, forbidding the firm from serving its residents—look carefully for the discreet disclaimer.
PS: Lowell Whiteside wrote a stirring call to action a few years ago, in Seismological Research Letters: "A thousand years from now people will remember that scientists of our era solved the earthquake prediction problem," he said. "We can choose to be a part of that process or a hindrance to it, but in the end a solution will be found." Sometimes despite our best efforts, we choose wrong.
