Measuring Prediction Accuracy in a Maritime Early Warning System, in Early View at Production and Operations ManagementJune 1, 2022 - Merrick, J. R. W., Dorsey, C., Wang, B., Grabowski, M. and Harrald, J.
Citation: Merrick J. R. W., Dorsey, C., Wang, B., Grabowski, M. and Harrald, J., “Measuring Prediction Accuracy in a Maritime Early Warning System.” In Early View at Production and Operations Management (2021).
The authors created a tool for the U.S. Coast Guard to warn of elevated accident risks for vessels on the Lower Mississippi River. When comparing alternative machine learning algorithms to use in the tool, they found that traditional measures of accuracy did not correctly represent the operational problem. Instead, their new metrics showed that context is important, and one should measure whether the operator’s response to the warning is correct, not whether the algorithm classified the type of accident precisely. The study found that the top performing algorithm would perform well as an early warning system and the authors developed a prototype system.