Noise2Average: An iterative residual learning strategy for image denoising without clean data

Abstract Magnetic resonance imaging (MRI) is a widely adopted non-invasive imaging tool for both clinical diagnosis and neuroscientific research. Nonetheless, the quality of MRI is often hampered by noise. Supervised deep learning-based denoising has proven to outperform conventional methods but requires high-signal-to-noise ratio (SNR) reference data for supervising the training, which considerably reduces its practical feasibility. To address this challenge, we propose a new iterative residual learning strategy entitled “Noise2Average” for denoising MRI data with multiple repetitions, which can be combined with transfer learning for subject-specific self-supervised training. Noise2Average learns to map each noisy repetition to the average of all noisy repetitions by fine-tuning parameters of a pre-trained convolutional neural network (CNN) and recovers higher SNR by averaging all denoised results at the first iteration, and performs this supervised residual learning-based denoising process repeatedly with the denoising results from the previous iteration as the training target for several iterations. The efficacy of Noise2Average is systematically and comprehensively demonstrated on four types of commonly acquired MRI data, including two or more consecutively acquired highly accelerated T1-weighted (T1w) image volumes, two T1w image volumes acquired with different echo times, two diffusion-weighted image (DWI) volumes acquired with opposite phase encoding directions, and two DWI volumes synthesized using different sets of DWI volumes from a diffusion tensor imaging (DTI) scan. Quantitative evaluations show that Noise2Average preserves more image sharpness and textural details and produces more accurate quantitative microstructural metrics from DTI signal modeling than the classic Noise2Noise method and conventional benchmark denoising methods BM4D and AONLM, with denoising performance slightly inferior to that of supervised learning-based denoising method. By reducing the requirement for training data and scan time, Noise2Average substantially increases the feasibility and accessibility of deep learning-based denoising methods for MRI and potentially benefits a wider range of clinical and neuroscientific studies.