Abstract

Significance: Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems.Aim: Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI.Approach: MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods.Results: Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy.Conclusions: The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.

View details for DOI 10.1117/1.JBO.28.1.016004

View details for PubMedID 36726664