ShakeAlert Testing and Certification Platform: Point Source and Ground Motion Based Evaluations

Elizabeth S. Cochran, Monica D. Kohler, Douglas D. Given, Jennifer Andrews, Men-Andrin Meier, Egill Hauksson, Sarah E. Minson, Mohammad Ahmad, Jonathan DeLeon, & Stephan Guiwits

Submitted August 8, 2016, SCEC Contribution #6559, 2016 SCEC Annual Meeting Poster #187

An earthquake early warning system, ShakeAlert, has been developed over the last decade through a collaboration between the U.S. Geological Survey and its partners. ShakeAlert messages are composed of point source information generated by three seismic-data based algorithms. In Spring 2016, ShakeAlert transitioned from demonstration to production prototype in California, with a similar switch in the Pacific Northwest scheduled Fall 2016. The production prototype system incorporates several features that improve the robustness of the system, including a Testing and Certification Platform. The Testing and Certification Platform provides a quantitative measure of performance for evaluating modifications to existing algorithms and assessing new algorithms. The platform incorporates an in situ, real-time test and playback of a common suite of test events. The test suite includes almost 100 different test cases: historic earthquakes in northern and southern California, teleseismic events, and calibration and mass recentering records.

We developed a set of metrics to evaluate the algorithm solutions from the output of the test suite. We produce a count of matched, missed, and false alerts by assigning threshold values for the maximum allowable difference from the catalog value for magnitude (2.0 units), epicenter (100 km), and origin time (15 s). Additionally, we define a maximum time to first alert based on when the alert may no longer be considered useful, i.e. an alert is issued after S waves travel to a radius defined by a Modified Mercalli Intensity of IV. An overall performance measure is computed based on alert latency as well as accuracy of point source parameters for magnitude, epicenter, and origin time.

ShakeAlert will soon incorporate finite-fault algorithms that cannot be assessed using point source metrics. So, we are now developing metrics based on how accurately an algorithm predicts expected ground motion. To evaluate the current algorithms, we take the point source solution and apply a set of Ground Motion Prediction Equations (GMPEs) to estimate the peak ground acceleration (PGA) at a given location, which are then compared to those measured at seismic stations. For a given PGA threshold we categorize the results into: true positive (ground motions were correctly predicted to be above a given threshold), true negative (ground motions were correctly predicted to be below a given threshold), false (ground motions were incorrectly predicted to be above a given threshold), and missed (ground motions were incorrectly predicted to be below a given threshold). Next steps will include finalizing the GMPE, or suite of GMPEs, to be used in the assessments and adding a temporal component that evaluates how algorithms perform as an event evolves.

Key Words
Earthquake Early Warning, Ground Motion Prediction

Cochran, E. S., Kohler, M. D., Given, D. D., Andrews, J., Meier, M., Hauksson, E., Minson, S. E., Ahmad, M., DeLeon, J., & Guiwits, S. (2016, 08). ShakeAlert Testing and Certification Platform: Point Source and Ground Motion Based Evaluations. Poster Presentation at 2016 SCEC Annual Meeting.

Related Projects & Working Groups