Limitations & Open Questions
An honest account of what the current model can and cannot do
Why We Publish This Page
Earthquake prediction is one of the most difficult problems in geoscience. Seismikon makes specific, falsifiable forecasts - we believe that is the right approach - but we are equally obligated to be explicit about the boundaries of those claims. This page describes every material limitation we are aware of. It is not exhaustive; science rarely is. If you are aware of a limitation not listed here, we want to hear from you at science@seismikon.com.
1. Geographic Scope: California Only
The current model is trained and validated exclusively on California seismic data. It operates within a fixed bounding box (lat 32–42°N, lon 114–125°W). Claims about performance in other regions would be unsupported.
The natural time / LSTM methodology (Bikos 2024) is in principle adaptable to other seismically active regions given sufficient catalogue data. We plan to extend geographic coverage but have not yet done so, and any future extension will be accompanied by a new validation study for that region.
2. Magnitude Scope: M ≥ 5.0 Target Events
Seismikon predicts significant earthquakes defined as M ≥ 5.0. Predictions for smaller events are not issued. The Bikos (2024) methodology was validated on events of comparable magnitude; performance at lower magnitude thresholds is unknown and would require a separate validation.
The upper end of the magnitude range - including M ≥ 7.0 great earthquakes - is statistically rare in the California training set. The model has limited empirical exposure to such events during training, and its performance for very large earthquakes should be regarded with additional caution.
3. 48-Hour Forecast Window
The model issues predictions with a 48-hour target window. It does not produce forecasts for longer time horizons (days, weeks, months) nor does it constrain event time to a narrow sub-hour window. The practical utility of a 48-hour earthquake prediction depends heavily on how that window aligns with emergency planning cycles.
4. Aftershock Ambiguity
The evaluation protocol does not filter aftershocks from the reference catalogue (see Evaluation Protocol §10). This means a prediction may be counted as a true positive by matching an aftershock of a large event rather than an independent earthquake. The natural time framework treats the full seismic sequence as meaningful signal - aftershocks are not noise in that model - so this is consistent with the methodology. Nonetheless, independent analysts who apply a declustering algorithm (e.g. Reasenberg, Gardner-Knopoff) may reach different TP/FP classifications.
5. False Negative Rate Cannot Be Rigorously Computed
Seismikon does not issue a prediction for every point in the operational region. When an M ≥ 5.0 earthquake occurs at a location for which no prediction was issued, it is ambiguous whether this reflects a model miss or a location the model correctly assessed as lower risk. The FN* figure shown on the Track Record page is a conservative upper bound - it counts all unmatched real events as misses. The true false negative rate is unknown and cannot be determined without a formal probability exceedance model for the null hypothesis.
6. Retrospective vs. Prospective Performance Gap
The 98% precision figure cited in the Bikos (2024) paper is from a retrospective back-test on historical data. Back-tests are susceptible to overfitting and do not account for distribution shift (the seismic environment at the time of testing may differ from future conditions). Live prospective performance - reported on the Track Record page - uses data that was entirely unseen during model development and is the only figure we treat as a genuine test of forward-looking predictive capability.
Prospective records are accumulated gradually. Early in the deployment, the sample size is small and confidence intervals on all metrics are wide. Statistical significance requires sustained evaluation over a long horizon.
7. EPS Score Calibration
The Earthquake Potential Score (EPS) is a relative index of seismic readiness, not a probabilistic forecast in the strict sense. It is not calibrated to a formal probability (e.g. "a prediction with EPS 80 has an 80% chance of a magnitude M event within distance D"). Research into formal calibration of the EPS against prospective outcomes is ongoing.
8. Not a Real-Time Hazard System
Seismikon is not a seismic monitoring or early-warning system. It does not detect P-waves, issue ShakeAlerts, or provide any warning of an earthquake that has already begun. Model runs occur on a schedule; there is no continuous real-time inference loop. Seismikon predictions should not be used to make immediate evacuation or safety decisions. Always follow official guidance from authorised seismic monitoring agencies (e.g. USGS ShakeAlert, CalOES).
Frequently Raised Scientific Questions
Q: Isn't earthquake prediction scientifically impossible?
The consensus view in mainstream seismology is that deterministic earthquake prediction - specifying exact time, location, and magnitude - is not achievable at useful accuracy. Seismikon does not claim deterministic prediction. The natural time / LSTM methodology produces probabilistic-style nowcasts with stated tolerances, analogous to weather forecasting. The prospective track record is the empirical test of those claims; we do not ask visitors to take performance on faith.
Q: How does this differ from operational probabilistic seismic hazard models (e.g. UCERF3)?
Models like UCERF3 are long-term (30-year or 50-year) hazard assessments based on fault mechanics, GPS strain, and historical seismicity. They do not issue 48-hour operational forecasts. Seismikon operates at a shorter time horizon using a data-driven approach (natural time statistics + LSTM). The two approaches are complementary, not competing; Seismikon's prospective results do not invalidate or contradict UCERF3-class models.
Q: How large does the tolerance window need to be for results to be meaningful?
This is a legitimate concern. A prediction with a 50 km spatial tolerance, ±4-hour time window, and ±1.5 magnitude band covers a finite but non-trivial fraction of California's seismic zone. The Track Record page computes a random baseline using empirical Monte Carlo permutation: predictions are randomly shuffled and re-matched against the same real-event catalogue. If Seismikon's hit rate exceeds the shuffled baseline at p < 0.05, the result is statistically significant. We have not yet accumulated enough prospective data to reach that threshold; we report the baseline alongside the live hit rate so visitors can form their own view.
Q: Are the predictions actually issued before the event, or backdated?
All predictions are stamped with a run_tag - the UTC timestamp at which
the model run completed and the record was ingested into the database. The
event_time (the forecast target time) is always in the future relative
to run_tag by definition: a prediction whose target time has already
elapsed at ingest is not added to the active set. The raw prediction records are
available for download, and independent analysts can verify the timestamp ordering.
Q: Why not use a skill score (e.g. Molchan diagram) for evaluation?
Molchan diagrams require a well-calibrated null hypothesis - typically a Poisson background rate derived from the long-term seismicity catalogue. Computing a valid Molchan tau requires precise definition of the forecast space (alarm fraction), which is ambiguous for a model that issues point predictions rather than areal probability maps. We use the empirical Monte Carlo permutation baseline as a more direct comparison because it uses the same prediction records and the same tolerance rules without requiring external rate assumptions. Adding formal Molchan evaluation is on the research roadmap.
Q: Has this methodology been peer-reviewed?
The underlying Bikos (2024) methodology was published in Geoinformatics & Geostatistics: An Overview (Vol. 12:4). The Seismikon implementation is an operational application of that methodology; a peer-reviewed paper specifically covering the prospective deployment and live track record is in preparation. Independent scientific review is welcomed - contact science@seismikon.com.