How Seismikon Evaluates Predictions
Every prediction is scored against a frozen, pre-published set of rules. No retroactive adjustments.
The Evaluation Protocol
Seismikon publishes an Evaluation Protocol that defines the exact rules used to decide whether a prediction is a True Positive (TP) or False Positive (FP). The protocol is versioned, publicly available, and frozen before any prediction window closes. This means the scoring rules cannot be adjusted in light of results.
The Four Matching Criteria
A real earthquake from the USGS catalogue matches a Seismikon prediction - scoring a True Positive - if and only if it satisfies all four of the following simultaneously:
- Location: the real event's epicentre is within the published spatial tolerance (kilometres) of the predicted coordinates.
- Magnitude: the real event's magnitude falls within the published magnitude window (± Δ M) of the predicted magnitude.
- Depth: the real event's focal depth falls within the published depth tolerance (kilometres).
- Time: the real event occurs within the 48-hour prediction window.
False Positives and False Negatives
A prediction that expires without a matching real event is recorded as a False Positive (FP). A real earthquake that occurs without a matching prediction is recorded as a False Negative (FN). Both are published openly - neither is suppressed from the record.
The Track Record page shows the complete prediction history including FP and FN outcomes. See False Positives and False Negatives for why both matter and how they are reported.
Skill Above Chance
A prediction system is only useful if it performs better than random chance. The evaluation framework includes an empirical Monte Carlo null model: thousands of random forecasts are generated over the same geography and time period, and the Seismikon hit rate is compared against the distribution of random-baseline hit rates. This produces a skill score that cannot be inflated by choosing generous tolerances.