Few-Shot Image Generation and Novel Deep Learning Model for Enhancing Classification of Microscopic Single-Cell Images Obtained from Peripheral Blood Smears

Abstract

The human body is composed of 1.2 to 1.5 gallons of blood which is approximately 7%-10% of the weight of an adult. The blood constitutes of plasma and blood cells, and the ratio is 55% and 45%. The plasma in the blood is responsible for carrying proteins, hormones and nutrients that help the blood clot and remove waste components from the human body. The blood cells in the body carry oxygen to the tissues, fight infections and also helps blood to clot. An excessive amount of blood cells or deficiency of blood cells can indicate various health problems like anaemia, leukaemia, thalassemia, etc. The most common form of malignancy in kids is leukaemia which represents 30% of all pediatric cancers. Classification of cells procured from bone marrow aspirate smears and cell differential count is important for hematologic disease diagnosis. But the process of classification and cell counting requires thorough manual intervention and is error-prone and tedious. Machine learning and deep learning have successfully generated accurate and excellent results in the medical domain, especially for automating the medical field's diagnosis proc ess. The only drawback is the requirement of ample data and a balanced dataset to develop an efficient model. The structure and pattern of data also play an important role in enhancing the performance of a model. In our research work, we take advantage of generative adversarial networks (GAN) and deep learning models to enhance the classification of microscopic single-cell images obtained from peripheral blood smears. To enhance classification, our primary goal was to balance and normalize the dataset. We use GAN for three reason in our proposed work. First, we utilize GAN for stain normalization. Second, we use GAN to generate synthetic images of blood cells that contained more than thousand images but less than six thousand images, per cell type in the dataset. Third, for cell types having less than hundred images we propose few-shot image generation based on GAN architecture and meta-learning framework. We combine the original and the synthetic data to form a balanced dataset. After stain normalization and image generation, we obtain a balanced and normalized dataset which is used by the classification model. For classification, we proposed a novel deep learning model, SENet-154-GE which uses the original data and the balanced dataset individually to demonstrate how normalizing and balancing a dataset through proper models and algorithms can enhance the performance of a classification model and improve the classification accuracy. For stain normalization, we have proposed a modified version of cycle consistency GAN (CycleGAN). We have incorporated Wasserstein GAN with gradient penalty (WGAN-GP) for the CycleGAN training and concentrated on the adversarial loss, cycle consistency loss and identity loss for building our model. Also, for the generator, instead of an encoder, transformer, and decoder network, we have used a U-Net architecture. For generating images of cell types with a moderate number of images, we have built a three-network GAN architecture consisting of a classifier, generator and discriminator. We named the model C-WGAN-GP since it’s a classifierbased GAN architecture which incorporates the loss function of WGAN-GP. For cell types with very less images, we have proposed a meta learning based GAN framework that is trained on a larger dataset and, through learning the parameters, can generate images with fewer examples. We have incorporated the squeeze-and-excitation networks (SE) into the aggregated residual transformations (ResNeXt) architecture. We have implemented SENet-154 model and combined the gather excite model with SENet-154 for crucial and detailed feature extraction. We named the novel deep learning classification model SENet-154-GE. For evaluation, we have used various evaluation metrics such as structural similarity index measurement (SSIM), Frechet inception distance (FID), and inception score (IS) for stain normalization. FID, precision, recall, F1-score, SSIM, L1 and L2 error, IS and learned perceptual image patch similarity (LPIPS) were used for evaluating C-WGAN-GP. We used FID, LPIPS and IS for evaluating few-shot image generation. We assessed the performance of the SENet-154-GE classification model through accuracy, specificity and sensitivity. We performed an ablation study to demonstrate the importance of each module in our proposed work, and the results show that our proposed approach can significantly enhance the performance of classifying microscopic singlecell images obtained from peripheral blood smears.|The human body is composed of 1.2 to 1.5 gallons of blood which is approximately 7%-10% of the weight of an adult. The blood constitutes of plasma and blood cells, and the ratio is 55% and 45%. The plasma in the blood is responsible for carrying proteins, hormones and nutrients that help the blood clot and remove waste components from the human body. The blood cells in the body carry oxygen to the tissues, fight infections and also helps blood to clot. An excessive amount of blood cells or deficiency of blood cells can indicate various health problems like anaemia, leukaemia, thalassemia, etc. The most common form of malignancy in kids is leukaemia which represents 30% of all pediatric cancers. Classification of cells procured from bone marrow aspirate smears and cell differential count is important for hematologic disease diagnosis. But the process of classification and cell counting requires thorough manual intervention and is error-prone and tedious. Machine learning and deep learning have successfully generated accurate and excellent results in the medical domain, especially for automating the medical field's diagnosis proc ess. The only drawback is the requirement of ample data and a balanced dataset to develop an efficient model. The structure and pattern of data also play an important role in enhancing the performance of a model. In our research work, we take advantage of generative adversarial networks (GAN) and deep learning models to enhance the classification of microscopic single-cell images obtained from peripheral blood smears. To enhance classification, our primary goal was to balance and normalize the dataset. We use GAN for three reason in our proposed work. First, we utilize GAN for stain normalization. Second, we use GAN to generate synthetic images of blood cells that contained more than thousand images but less than six thousand images, per cell type in the dataset. Third, for cell types having less than hundred images we propose few-shot image generation based on GAN architecture and meta-learning framework. We combine the original and the synthetic data to form a balanced dataset. After stain normalization and image generation, we obtain a balanced and normalized dataset which is used by the classification model. For classification, we proposed a novel deep learning model, SENet-154-GE which uses the original data and the balanced dataset individually to demonstrate how normalizing and balancing a dataset through proper models and algorithms can enhance the performance of a classification model and improve the classification accuracy. For stain normalization, we have proposed a modified version of cycle consistency GAN (CycleGAN). We have incorporated Wasserstein GAN with gradient penalty (WGAN-GP) for the CycleGAN training and concentrated on the adversarial loss, cycle consistency loss and identity loss for building our model. Also, for the generator, instead of an encoder, transformer, and decoder network, we have used a U-Net architecture. For generating images of cell types with a moderate number of images, we have built a three-network GAN architecture consisting of a classifier, generator and discriminator. We named the model C-WGAN-GP since it’s a classifierbased GAN architecture which incorporates the loss function of WGAN-GP. For cell types with very less images, we have proposed a meta learning based GAN framework that is trained on a larger dataset and, through learning the parameters, can generate images with fewer examples. We have incorporated the squeeze-and-excitation networks (SE) into the aggregated residual transformations (ResNeXt) architecture. We have implemented SENet-154 model and combined the gather excite model with SENet-154 for crucial and detailed feature extraction. We named the novel deep learning classification model SENet-154-GE. For evaluation, we have used various evaluation metrics such as structural similarity index measurement (SSIM), Frechet inception distance (FID), and inception score (IS) for stain normalization. FID, precision, recall, F1-score, SSIM, L1 and L2 error, IS and learned perceptual image patch similarity (LPIPS) were used for evaluating C-WGAN-GP. We used FID, LPIPS and IS for evaluating few-shot image generation. We assessed the performance of the SENet-154-GE classification model through accuracy, specificity and sensitivity. We performed an ablation study to demonstrate the importance of each module in our proposed work, and the results show that our proposed approach can significantly enhance the performance of classifying microscopic singlecell images obtained from peripheral blood smears.