Middle ear-acquired cholesteatoma diagnosis based on CT scan image mining using supervised machine learning models

Abstract

Background

Distinguishing between middle ear cholesteatoma and chronic suppurative otitis media (CSOM) is an ongoing challenge. While temporal bone computed tomography (CT) scan is highly accurate for diagnosing middle ear conditions, its specificity in discerning between cholesteatoma and CSOM is only moderate. To address this issue, we utilized trained machine learning models to enhance the specificity of temporal bone CT scan in diagnosing middle ear cholesteatoma. Our database consisted of temporal bone CT scan native images from 122 patients diagnosed with middle ear cholesteatoma and a control group of 115 patients diagnosed with CSOM, with both groups labeled based on surgical findings. We preprocessed the native images to isolate the region of interest and then utilized the Inception V3 convolutional neural network for image embedding into data vectors. Classification was performed using machine learning models including support vector machine (SVM), k-nearest neighbors (k-NN), random forest, and neural network. Statistical metrics employed to interpret the results included classification accuracy, precision, recall, F1 score, confusion matrix, area under the receiver operating characteristic curve (AUC), and FreeViz diagram.

Results

Our training dataset comprised 5390 images, and the testing dataset included 125 different images. The neural network, k-NN, and SVM models demonstrated significantly higher relevance in terms of classification accuracy, precision, and recall compared to the random forest model. For instance, the F1 scores were 0.974, 0.987, and 0.897, respectively, for the former three models, in contrast to 0.661 for the random forest model.

Conclusion

The performance metrics of the presented trained machine learning models hold promising prospects as potentially clinically useful aids.

1 Background

Chronic otitis media (COM) is considered the primary cause of preventable hearing loss in developing countries [1]. Another significant issue is that COM consists of a broad spectrum of pathologies, encompassing relatively benign entities such as chronic suppurative otitis media (CSOM) and more serious, life-threatening conditions such as cholesteatoma. The latter carries an osteolytic capacity due to the inflammatory mediators of its matrix. Therefore, in addition to permanent hearing loss, other serious complications can occur, as cholesteatoma may extend into the brain, leading to conditions such as meningitis, sinus thrombosis, facial nerve palsy, vestibulitis, and brain abscess [1, 2].

The joint consensus paper by the European Academy of Otology and Neurotology and the Japanese Otological Society addresses the complex endeavor of defining, classifying, and staging middle ear cholesteatoma [3]. Furthermore, it is difficult to eradicate such conditions, as demonstrated by the numerous surgical techniques described for the management of cholesteatoma and the reported residual disease rates, particularly in children, where rates range from 10 to 40%, implying extended monitoring for up to 5 years [4]. On the other hand, there are a few reported cases of cholesteatoma associated with CSOM in the same ear [5]. Consequently, achieving a precise preoperative diagnosis is critical for optimal management.

Relying solely on clinical ear examination may not consistently provide the precision required to differentiate between cholesteatoma and other categories of chronic otitis media. Therefore, clinicians frequently request imaging techniques to enhance preoperative assessment [3,4,5,6]. Ayache et al. recommend the use of a high-resolution preoperative computed tomography scan (HRCT) not only for its unequivocal utility in assessing disease extension and anatomical variations but also as an effective diagnostic modality [4].

Indeed, temporal bone computed tomography (CT) scans with 1-mm slices provide high spatial resolution for evaluating ear anatomy, making them a valuable preoperative tool. Nevertheless, their sensitivity for diagnosing cholesteatoma fluctuates from 72.73% to 88% [6,7,8], while specificity is only moderate, ranging from 66 to 77.22% [7, 8].

Data mining is a field that combines computer science and statistics to extract valuable information from extensive databases. Originally employed for alphanumeric databases, the increase in storage capacity has resulted in the accumulation of substantial volumes of non-standard data, including audio, images, and videos. Therefore, the mining of medical data has demonstrated its potency and precision in extracting information to establish classifications and discover patterns, generating crucial information for decision-making [9,10,11,12].

Only six manuscripts discuss the differential diagnosis of middle ear-acquired cholesteatoma [13,14,15,16,17,18], with two studies based on otoendoscopy images [16, 17] and four manuscripts using HRCT [1, 13,14,15].

Our study suggests an alternative approach for the automated diagnosis of CSOM versus cholesteatoma based on trained machine learning models using HRCT scan native images. Additionally, we compared our results with those in the existing literature and discussed the methodology and the classifier's performance.

2 Methods

2.1 Study concept

To enhance the specificity of temporal bone HRCT scans for the diagnosis of middle ear cholesteatoma versus CSOM, we trained several machine learning models on a dataset of native HRCT scans of both conditions. First, we used a convolutional neural network (CNN) to extract features for image embedding. Then, we trained several machine learning classifiers to statistically categorize the HRCT scan images into either the cholesteatoma group or the CSOM group. We compared the classifiers’ performance based on statistical indicators. The prediction step of the trained machine learning classifiers was performed on a different dataset.

2.2 Study design

This case–control study includes 122 patients with middle ear-acquired cholesteatoma, and as a control group, 115 patients diagnosed with CSOM. Both groups were labeled based on intraoperative findings. Our institution review board has reviewed and approved the study and granted us an informed consent waiver.

• For this study, we included:

  • Preoperative temporal bone HRCT scans performed for patients diagnosed with cholesteatoma based on endoscopic or microscopic examination. The diagnosis of cholesteatoma was also confirmed by surgical findings.

  • Temporal bone CT scans of patients operated for CSOM for whom surgery findings confirmed the presence of cholesteatoma.

  • A control group of preoperative CT scans of patients operated for CSOM without cholesteatoma.

• We excluded the patients operated on for middle ear cholesteatoma who exhibited residual disease, according to conventional imaging, during the follow-up period.

2.3 Database

The first dataset used for training and validation included 5390 images, with 2742 images of middle ear-acquired cholesteatoma and 2648 images of CSOM. This dataset was then divided into two subsets: 80% for training and 20% for validation. An independent dataset consisting of 125 images was used for testing purposes. All images were obtained from the Otolaryngology-Head and Neck Surgery and Radiology Departments of a single university hospital.

2.3.1 Imaging materials

We used one-millimeter CT scan native slices of the temporal bone. These images were acquired as Digital Imaging and Communication in Medicine files (DICOM) from the hospital's Picture Archiving and Communication System (PACS) and were anonymized for the study. Axial views were obtained using a 32-channel multidetector BrightSpeed CT scanner (GE Medical Systems) with the following specifications: an exposure time per rotation of 0.8 s, generator power of 800 mA with a standard convolution kernel, scanning length of 250.00 mm, nominal single collimation width of 1.25 mm, nominal total collimation width of 250.0 mm, voltage of 120.0 kV, pitch factor of 1.0, and a maximum X-ray tube power of 160.0 mA. The machine performed scanning from the tip of the mastoid process to the top margin of the petrous bone in each patient.

2.3.2 Labeling

Relevant slices from the temporal bone CT scan series were manually selected and annotated by two specialist doctors, an otolaryngologist and a radiologist. The doctors independently classified the images into either the cholesteatoma or CSOM group based on radiological features. This classification was subsequently validated against the surgical findings documented in the patients' medical records. Finally, the images were saved in JPEG format. Subsequently, we cropped the images to focus on the region of interest, eliminating any unnecessary data related to the patient or the equipment and imaging techniques. Following this, the database was categorized into four groups: right ear cholesteatoma, left ear cholesteatoma, right ear CSOM, and left ear CSOM.

2.4 Image mining

2.4.1 Image embedding

An image embedding serves as a concise representation of an image within a lower-dimensional space. The process of generating image embeddings entails feature extraction using a model conceptualized as a mapping function, denoted as Φ. This function takes an image X as input and extracts its pertinent features, producing the resulting output known as the embedding of X: Φ(X) [19]. For our study, we utilized a pre-trained Inception V3 model as a feature extractor.

Inception V3 (Φ(X)) is a sophisticated 42-layer CNN classifier that has been pre-trained on the extensive ImageNet dataset. It can also function as a feature extractor by excluding the final fully connected layer [10]. The intermediate outputs from the last pooling layer can then be employed to generate a feature embedding, which is a vector of size 2048 that summarizes the key features of the input image. These 2048 abstract values effectively capture the salient characteristics of the image, even when it comprises millions of pixels. The feature extractor's capability to condense information about the image is powerful as it reduces the number of dimensions per image, facilitating efficient image comparisons. Subsequently, classification is conducted by comparing the similarities between embeddings to identify similar images [9,10,11].

We used the FreeViz diagram to exhibit the distribution of descriptive data vectors generated by Inception V3. The FreeViz diagram is an optimization method that identifies linear projections and corresponding scatter plots, effectively segregating instances of different classes. This visualization method displays the distribution of descriptive vectors for each image based on the four classes of our classification. Each class has its own set of unique vectors.

2.4.2 Classification models

The image classification process involves evaluating extracted features, which holds more significance than comparing images at the pixel level. Several methods can be employed to compare feature vectors, including subtraction, calculating the Euclidean distance or cosine similarity between vectors, or treating the two sets of images to be classified as distributions and measuring the distance between them [18, 19]. We trained four models for the classification of our database:

• k-nearest Neighbors model:

The k-nearest neighbors (k-NN) algorithm is a widely used and straightforward machine learning model for identifying patterns in classification and regression problems. It determines the distance of a test observation from all observations in the training dataset and identifies the 'k'-nearest neighbors for each test observation. The algorithm relies on distance metrics, with the Euclidean distance being the most commonly used, although other metrics such as Manhattan, Maximum, or Mahalanobis can also be applied. The weight assigned to neighbors in the classification process may be uniform or defined based on distance, giving closer neighbors greater influence."

• Support vector machine model:

Support vector machine (SVM) maps inputs to higher-dimensional feature spaces and is employed as a supervised machine learning model for classification and regression. It optimizes the gap width between two categories by mapping training examples to points in space. New examples are then mapped into the same space, predicting their category based on their position relative to the gap. This technique often yields superior predictive performance.

• Random Forest model:

Random forest is an ensemble learning method utilized for classification, regression, and various other tasks. It constructs a set of decision trees, each developed from a bootstrap sample of the training data. In building individual trees, a random subset of attributes is chosen, and the best attribute for the split is determined. The final model is based on the majority vote from the individually developed trees in the forest.

• Neural network model:

The neural network used is scikit-learn (sklearn), a multilayer perceptron algorithm with backpropagation, capable of learning both linear and nonlinear models. The rectified linear unit (ReLU) function serves as the activation function for the perceptron. We applied the Adam stochastic gradient-based optimizer to adjust the algorithm's learning ability and reduce the error rate.

2.4.3 Training

The performance of each classification model is assessed using various parameters, including classification accuracy, recall (sensitivity), precision, F1 score, AUC (area under the receiver operating characteristic (ROC) curve), and the confusion matrix. The F1 score, which combines precision and recall through their harmonic mean, is particularly valuable as it implies the simultaneous maximization of both precision and recall. Consequently, the F1 score has become the preferred metric in conjunction with accuracy when evaluating our models.

2.4.4 Prediction

For prediction purposes, we utilized a new dataset of 125 temporal bone CT scan images. Subsequently, we compared the classification performance of our trained models. Figure 1 summarizes the image mining approach used in this research.

Fig. 1

Methodology of the research protocol

3 Results

Figure 2 illustrates the distribution of data vectors generated by Inception V3 image embedding using the FreeViz diagram. It demonstrates that certain vectors are associated with a specific diagnosis, while others are common to more than one class. For example, vectors n948 and n716 belong to the right ear cholesteatoma group, while vectors n713 and n664 belong to the right ear CSOM group.

Fig. 2

FreeViz diagram depicting the distribution of feature vectors based on the diagnosis

Among all models used for the four-item classification task (right ear cholesteatoma, left ear cholesteatoma, right ear CSOM, and left ear CSOM), the neural network, k-NN, and SVM demonstrated superior performance compared to the random forest model. The AUC values at the optimal cutoff probability determined by the curve of the three former models (neural network, k-NN, and SVM) were higher (0.999, 0.999, and 0.987 respectively) than the performance of the random forest model (0.865).

Figures 3A, B and 4A, B provide a detailed display of the models' AUC performance. Similarly, the neural network, k-NN, and SVM models showed superior performance in terms of classification accuracy, precision, and recall compared to the Random Forest model. For example, the classification accuracy was 0.974, 0.987, and 0.896, respectively, for the three former models compared to 0.665 for the random forest model. On the other hand, the F1 score was 0.974, 0.987, and 0.897, respectively, for the three former models compared to 0.661 for the random forest (Table 1).

Fig. 3

A The receiver operating characteristic curve for left ear cholesteatoma. Target probability 27.0%. B The receiver operating characteristic curve for right ear cholesteatoma. Target probability 24.0%

Fig. 4

A The receiver operating characteristic curve for left ear Chronic Suppurative Otitis Media. Target probability 31.0%. B The receiver operating characteristic curve for right ear Chronic Suppurative Otitis Media. Target probability 18.0%

Table 1 The statistic metrics of each classification model after fivefold cross-validation

Figure 5A, B displays the confusion matrix of each model regarding the classification of the dataset into the four predetermined categories.

Fig. 5

A Confusion matrix for neural network model (upper), and k-nearest neighbors model (lower). B Confusion matrix for support vector machine model (upper), and random forest model (lower). Our database is available on: https://github.com/naouar2023/database-cholesteatome.git/README.md

Regarding the testing dataset, the neural network and SVM outperformed the other models, with only 3/125 and 2/125 misclassified images, respectively.

Table 1 displays the statistical metrics of each model after undergoing fivefold cross-validation.

4 Discussion

To distinguish between CSOM and middle ear cholesteatoma using image mining of temporal bone CT scans, we utilized the Inception V3 CNN for image embedding. This method represented each image as a 2048-dimensional data vector, which was then analyzed and classified by various models. Our approach achieved an F1 score over 95% for neural network and k-NN models, and nearly 90% for the SVM model, whereas random forest achieved less satisfactory performance with an F1 score of only 66.1%. Furthermore, our models achieved high classification accuracy, with an area under the curve (AUC) of 0.99 for the neural network and k-NN models, 0.98 for SVM models, and 0.86 for the random forest model at the optimal cutoff probability determined by the curve.

Artificial intelligence has made remarkable progress in diagnosing and classifying human diseases, utilizing various imaging techniques such as standard radiography [20], echocardiography [21], CT scans [1, 11,12,13,14,15,16], and MRI [22]. It has demonstrated exceptional performance in medical fields including COVID-19 diagnosis [23], intracranial hemorrhage [24], critical findings in head CT scans [25], pulmonary tuberculosis [26], liver tumors [27], pathology, genetic mutation classification [28], and otolaryngology malignancies [29, 30], among others. A recent review focused on the application of deep learning in otolaryngology found that out of 458 articles examined, 38 focused on image-based analysis using artificial intelligence [31]. The study identified several techniques for image retrieval and recovery, with many classification models achieving accuracy rates between 85 and 90%, including CNN perceptor V3, auto machine learning (auto ML), latent Dirichlet allocation (LDA), statistic shape model (SSM), random forest (RF), and support vector machine (SVM) [31].

In various studies, the performance of specific models, notably SVM, RF, and CNN, exhibited variability, potentially attributable to differences in preprocessing techniques applied to datasets or variations in database size. It is noteworthy that larger databases tend to yield more accurate classification in automated models.

Six studies investigated the differential diagnosis between middle ear cholesteatoma and CSOM [1, 13,14,15,16,17], with only four focusing on image mining of temporal bone CT scans [1, 13,14,15]. Exploration was limited to three studies, as Su et al.'s [14] manuscript was exclusively in Chinese. Wang et al. employed a segmentation framework based on a CNN for region-of-interest extraction, followed by data augmentation. Using Inception V3, they achieved an overall accuracy of 76.7%, with a recall rate of 75% for the CSOM group and 76% for the cholesteatoma group [1]. However, the observed performance difference between Wang et al.'s study and our series may be attributed to the limited and unbalanced dataset in their study. Notably, their training and validation set included 146 cholesteatoma cases and 505 CSOM cases, utilizing automated data augmentation, which may not necessarily enhance pertinent information for classifications. A similar issue arose in Takahashi et al.'s research [15]. In this study, a distinct classification model based on deep neural network (DNN) was utilized, necessitating extensive and very large datasets. The dataset comprised 4950 images, with only 2425 slides including middle ear lesions. Takahashi et al. also employed data augmentation to address this limitation.

Finally, Eroglu et al. adopted a methodology closely similar to our research. They utilized multiple pre-trained CNNs (AlexNet, GoogleNet, and DenseNet201) as feature extractors and a single trained machine learning model as a classifier (SVM). With a database of 3039 images, including 1844 significant slides, they refrained from using data augmentation and achieved satisfactory performance (recall ~ 93% and precision ~ 97%) [13].

Table 2 summarizes protocols and results of studies addressing the diagnosis of cholesteatoma based on the automatic analysis of temporal bone CT scans.

Table 2 Protocols and results of studies addressing the diagnosis of cholesteatoma based on automatic analysis of the temporal bone computed tomography scan

5 Conclusion

Automated classification performance metrics of machine learning models suggest their potential utility in assisting radiologists in daily practice or as part of resident specialist training. However, a primary obstacle to implementing these techniques in routine medical care lies in the limited understanding of the rationale behind classifications, as emphasized by Tachibana et al. [32].