Interpretable Deep Learning for Multimodal Super-Resolution of Medical Images

Abstract

In medical image acquisition, hardware limitations and scanning time constraints result in degraded images. Super-resolution (SR) is a post-processing approach aiming to reconstruct a high-resolution image from its low-resolution counterpart. Recent advances in medical image SR include the application of deep neural networks, which can improve image quality at a low computational cost. When dealing with medical data, accuracy is important for discovery and diagnosis, therefore, interpretable neural network models are of significant interest as they enable a theoretical study and increase trustworthiness needed in clinical practice. While several interpretable deep learning designs have been proposed to treat unimodal images, to the best of our knowledge, there is no multimodal SR approach applied for medical images. In this paper, we present an interpretable neural network model that exploits information from multiple modalities to super-resolve an image of a target modality. Experiments with simulated and real MRI data show the performance of the proposed approach in terms of numerical and visual results.