False Positives & False Negatives
Why both types of error matter for earthquake prediction, and how Seismikon's evaluation handles them.
Definitions
In the context of Seismikon's evaluation protocol:
- True Positive (TP): a prediction that is matched by a real earthquake within the defined tolerances.
- False Positive (FP): a prediction that expires without a matching real event - an alert that did not correspond to an earthquake.
- False Negative (FN): a real earthquake that occurred without a matching prediction - a missed event.
- True Negative (TN): a time window in which no prediction was issued and no qualifying event occurred.
Why False Negatives Are Generally Costlier
In earthquake prediction, the asymmetry between error types is significant. A false positive means an unnecessary alert was issued - operational cost, potential alarm fatigue. A false negative means a significant earthquake occurred that the system did not predict - the core safety value of the system failed to materialise.
Seismikon's recall rate (the fraction of real qualifying earthquakes that were matched by a prediction) is therefore as important as its precision. Both metrics are reported on the Track Record page.
How They Appear in Seismikon's Record
Every prediction in Seismikon's history is publicly labelled with its outcome: TP, FP, or pending (prediction window not yet closed). Real earthquakes that do not match any active prediction are recorded as false negatives. The complete record is downloadable from the Data Downloads page.
No prediction outcome is removed from the record. A prediction is never retroactively re-scored to a TP by adjusting the evaluation protocol after the fact.
The Role of Tolerance Choice
The choice of tolerances (spatial, magnitude, depth, time) directly affects the TP/FP/FN balance. Wider tolerances increase recall (fewer FNs) but decrease precision (more FPs). Narrower tolerances do the reverse. The Seismikon protocol fixes tolerances in advance; the skill score is then computed against a random baseline that uses the same tolerances, so generous tolerances do not artificially inflate the reported skill.