Researchers have developed an algorithm that can identify people previously diagnosed with diabetes in Apple Watch heart data. The learning algorithm provides a strong model for identifying people with diabetes but has yet to prove itself against the tougher task of spotting undiagnosed patients without also racking up false positives.
Cardiogram, an Apple Watch app developer, ran the study (PDF) on 14,011 of its users in conjunction with researchers at UCSF. After using data on some of the participants to train a deep neural network, called DeepHeart, the team tested the algorithm on results from the remaining cohort of subjects. The best version of the algorithm recorded a c-statistic of 0.85 in diabetes, making it a strong model.
San Francisco-based Cardiogram is able to identify people with diabetes from heart data because of earlier work that spotted a correlation between variability in cardiovascular activity and the condition.
The study suggests the algorithm is pretty good at identifying people previously diagnosed with diabetes. That ability has limited clinical utility, though. The real prize is alerting undiagnosed patients that they may have the condition so they know to get tested and potentially treated. Cardiogram is working to show it can pull off this trickier task.
RELATED: Apple eyes FDA approval filing for investigational heart device
In this context, the algorithm’s ability to avoid false alarms is as important as the proportion of cases it accurately diagnoses. Each false positive will cause suffering to the individual and result in blood tests, mobile ECGs and other unnecessary healthcare procedures, wiping out many of the benefits of the algorithm’s accurate diagnoses. Cardiogram is aware of the problem but thinks the benefits will outweigh the costs once a certain specificity threshold is passed.
“With sufficiently high precision the cost of false positives is exceeded by the cost savings to the healthcare system of each true positive,” researchers at Cardiogram and UCSF wrote in the paper published this week.
The question is whether Cardiogram can tweak the algorithm so it can identify a meaningful number of undiagnosed diabetes patients without dialing the specificity down too far. The curves presented in the study this week show sensitivity rises above 0.8 at the point specificity falls toward 0.8. Ideally, a test would have a sensitivity of 1 and specificity of 1.