Neural network bests dermatologists in lesion classification test

Melanoma
Skin lesions like Melanoma may soon be easy to treat with a neural network-powered smartphone apps. (National Cancer Institute)

A deep learning network classified skin lesions more effectively than dermatologists in a recent test. The study builds on earlier research that delivered similar findings, raising hopes that smartphones can expand access to dermatologist-grade classification of lesions.

The latest study trained and validated a deep learning convolutional neural network using images of the skin and accompanying diagnoses. This neural network was then turned on 100 images that were also shared with 58 dermatologists, 30 of whom had more than five years of experience. The study looked at diagnostic accuracy when the images were analyzed with and without clinical information. 

Dermatologists classified potentially cancerous lesions with a sensitivity of 86.6% and specificity of 71.3% when given just the images. The figures increased by around two and four percentage points, respectively, when clinical information was provided. The 30 experienced dermatologists performed slightly better than the overall group.

FREE DAILY NEWSLETTER

Like this story? Subscribe to FierceBiotech!

Biopharma is a fast-growing world where big ideas come along every day. Our subscribers rely on FierceBiotech as their must-read source for the latest news, analysis and data in the world of biotech and pharma R&D. Sign up today to get biotech news and updates delivered to your inbox and read on the go.

The neural network outperformed the dermatologists. Using the dermatologists’ 86.6% sensitivity as the benchmark, the neural network achieved a specificity of 82.5%. That compared favorably to the 71.3% specificity of the dermatologists. A similar gap in performance was seen when dermatologists had access to clinical information.

The findings echo the results of a similar study that was published in Nature early last year. That study also trained a neural network and found it held its own against dermatologists. Collectively, the papers add to the impression that lesion classification may become an early test case for AI in healthcare.

As the papers note, the availability of lesion-classifying neural networks and the ubiquity of smartphones equipped with cameras could reshape how lesions are assessed initially. Technology could move lesion classification out of the clinic, thereby expanding access, driving down costs and increasing the frequency and timeliness of assessments. 

That rosy vision is undermined somewhat by the limitations of the latest study, which are noted in the paper. At 58, the number of dermatologists against whom the neural network was compared is fairly small. There is also a question mark over whether the dermatologists act differently in the real world, where the consequences of diagnostic errors are far more serious.

The readthrough from the study is further constrained by the limitations of the images used. The test set lacked certain types of lesion, such as pigmented basal cell carcinoma, and had less genetic and skin type diversity than is seen in the real world. 

Larger studies would address many of these issues but the truest test of the effectiveness of the tool in the real world will come from prospective trials. This is the setting that will start to show whether doctors and patients will accept neural networks and act on their advice. AI will underperform if doctors do not trust the technology and therefore do not follow its recommendations. 

Suggested Articles

By employing heart rate signals, physical activity and sleep quality, common Fitbit trackers may be able to predict the spread of the flu.

Nanox has raised $26 million to help fuel the development and commercialization of its Star Trek-inspired digital X-ray bed.

Oncology is clearly a major medical and societal issue, but one that sees too much focus from biopharmas at the expense of other killers.