W21 EECS 556 project abstracts # Denoising % allanzhu-junyyang-wangchy-yzhiheng Guided Image Filtering: Theory, Improvements and Application Jingying Wang, Junyuan Yang, Zhiheng Yin, Yilun Zhu Edge-preserving image filtering is an important topic in image processing and has also been serving as the foundation for many high-level computer vision and graphics applications. In this project, we investigated a highly-cited edge-preserving denoising method, guided image filter (GIF). In this joint image processing scenario, we are given two images: (noisy) input and guidance. The goal is to use information from both to provide a noise-free output with sharp edges. Inspired by several other methods, like rolling bilateral filter, weighted GIF, and static-dynamic filter, we further developed rolling GIF (RGIF), edge-aware GIF (WGIF), adaptive static-dynamic GIF (A-SDGIF), and augmented guidance static-dynamic GIF (AG-SDGIF). These extensions all improve GIF to some extent, and AG-SDGIF achieves the best performance both visually and in the percentage of bad batching pixels (PBP). https://github.com/wjymonica/WGIF-and-GIF % dongyaoz-yanbhliu-yangcao-ychenn Image denoising via sparse and redundant dictionary learning and non-local means filter Yang Cao, Yang Chen, Yanbaihui Liu, Dongyao Zhang K-SVD algorithm is a classical tool used for image denoising. However, the traditional K-SVD dictionary cannot reflect all the detailed information of a noisy image. We designed a combined dictionary algorithm, which is based on the classification of training samples and non-local regularized sparse representation. First, we divided patches on a noisy image into the smooth, edge and texture categories based on the variance of pixels and PCA algorithm, and then trained an overcomplete dictionary of three categories by K-SVD algorithm. To further improve the image reconstruction result, a non-local regularization was introduced to the cost function to modify the sparse coefficients. We obtained the optimization result by the ADMM algorithm. The experimental results show that the peak signal-to-noise ratio of the combined method is close to the original K-SVD denoising algorithm. https://github.com/dongyaoz/EECS-556-project % jashu-jiachenj-mingshzh-zctchn Hyperspectral Images Denoising via a Tensor Dictionary Jason Hu, Jiachen Jiang, Chengtian Zhang, Mingshuang Zhang As 3rd order tensors, Hyperspectral images (HSIs) can deliver more authentic representations for real scenes, enhancing the performance of many computer vision tasks when compared with traditional RGB or gray-scale images. In practice, the quality of an HSI is always affected by various noises. In this paper we propose an effective HSI denoising approach based on the Tensor Dictionary Model by considering two intrinsic characteristics underlying an HSI: the nonlocal similarity over space and the global correlation across spectrum. We modify the original method of grouping similar patches into clusters and optimizing over each cluster by designing an optimization problem with an objective function targeted at representing the HSI with a sparse global dictionary representation. Iteration over variables is used to solve the nonconvex Lagrange dual problem. Experimental results show that our method outperforms some basic state-of-the-art HSI denoising methods under comprehensive quantitative performance measures due to convergence issues. At the end we analyze our results and explore other possibly better methods. # Restoration % huwei-nisargtr-ycpan-zonyul Algorithms for Blind Image Deblurring Wei Hu, Zongyu Li, Preston Pan, Nisarg Trivedi Blind image deblurring aims to restore a blurred image without knowledge of the blurring kernel. This report discusses three algorithms based on alternating minimization routines of a maximum a posteriori (MAP) model. We first re-implemented the method proposed by Dong et al., where an iteratively re-weighted least squares algorithm (IRLS) was used to optimize the proposed novel data-fidelity function with hyper-Laplacian image prior and vector 1- norm as kernel prior. However, because IRLS suffers drawbacks of division by zero and is sensitive to initialization, we investigated the majorize-minimize algorithm (MM) with Huber's majorizer and applied proximal gradient methods for optimization. Additionally, we implemented an image gradient-based method for latent image estimation, as suggested by Prof. Qing Qu. Furthermore, to potentially improve the deblurred results, we explored initializing our algorithms with images estimated by a blind deep learning method (Ren et al.) which out-performed all other methods and initialization schemes. All algorithms discussed in this report were tested on a well-known data set (consisting of 4 real images and 8 blurring kernels) and evaluated with Normalized Root Mean Square Error (NRMSE) and Peak Signal to Noise Ratio (PSNR). With a comprehensive comparison and analysis, we could observe that our deblurring algorithms met the expectations of quantitative performance prediction (QPP) stated in our proposal. https://github.com/TrNi/Image_Processing_Project % dnaihao-linguo-zhaoyq Graph-Based Blind Image Deblurring From a Single Photograph Naihao Deng, Lingyun Guo, Yiqing Zhao Blind image deblurring, which is the deblurring without knowledge of the blur kernel, is a highly ill-posed problem. The problem can be solved in two parts: 1) estimate a blur kernel from the blurry image, and 2) deconvolve the blurry input to restore the target image based on an estimated blur kernel. In this project, we re-implement the graph-based blind image deblurring algorithm by interpreting an image patch as a signal on a weighted graph based on. Specifically, we first state that a skeleton image–a piece wise smooth (PWS) proxy that retains the strong gradients of the tar- get image but smooths out the details–is sufficient to estimate the blur kernel. Then, we propose a graph-based image re-weighted graph total variation prior (RGTV) that promotes a bi-modal weight distribution to reconstruct a skeleton patch from a blurry observation. Further, we design a graph weight function so that RGTV can be expressed as a graph l1-Laplacian regularizer. The prior can then be interpreted as a low-pass graph filter with desirable spectral properties. Based on the spectral interpretation of RGTV, we implement the algorithm to solve the non-convex non-differentiable optimization problem alternately, where the two sub-problems to solve for the skeleton image and the blur kernel have closed-form solutions. Finally, we apply recent non-blind image deblurring algorithms to restore the target image with the computed blur kernel. Experimental results demonstrate that our re-implemented algorithm successfully restores latent sharp images and achieves similar performances to the protected code provided by quantitatively and qualitatively. https://github.com/YunyyYY/eecs556-graph-based-blind-deblur % jyzhong-senweim-sunmessi-ziboyan Outlier Identifying and Discarding in Blind Image Deblurring Senwei Ma, Jiawei Sun, Zibo Yan, Jiayang Zhong Blind deblurring methods are sensitive to outliers, such as saturated pixels and non-Gaussian noise. Although recent state-of-the art outlier handling methods have developed effective ways to deal with them, their limitations are obvious, like the complexity of edge-selecting skills and some specific fidelity terms that may not be suitable for fitting additive noise. What’s more, these sophisticated methods may fail when edges are severely polluted by high-density outliers. Our group try to re-implement the main idea from a new published outlier-handling framework named outlier identifying and discarding (OID) which addresses these problems efficiently and directly. Unlike previous indirect outlier processing methods, OID tackles outliers by explicitly re-weighting them during both latent image and blur kernel updating steps. We show that OID method can perfectly handle different types of outliers. Experimental results demonstrate that OID method outperforms other outlier-handling framework substantially in PSNR quantitative metric but seems fail to recover the original sharp edges. # Reconstruction % acshikh-avrotsos-cumminge-nalucido Plug-and-Play Methods for Compressive MRI Reconstruction A. Haggenmiller, A. Vrotsosa, E. Cummings, N. Lucido No abstract provided https://github.com/acshi/eecs556_mri_pnp % dnikolov-efvega-echeek-gfortman Plug n' Play Denoising Methods to Reconstruct Sparse SAR Data Eric Cheek, Geoff Fortman, Denislav Nikolov, Erick Vega We investigated the use of various denoising algorithms in the context of high-bandwidth synthetic aperture radar (SAR) imaging. As is typical of airborne SAR imaging, the data used in our experiments had limited angular extent, but with the added challenge of a sparse measurement model. The non-exhaustive angular support and random under-sampling of the data leads to aliasing noise in the SAR image when using conventional Fourier or backprojection-based imaging methods. We demonstrate that the use of plug n' play denoising algorithms is effective in reducing the appearance of these artifacts. We tested our algorithms using the AFRL 2D Backhoe dataset. We argue that when used as a post-processing technique, this procedure yields more interpretable images with qualitatively better characteristics. % jiarenz-makur-ninglu-yuhangz Learning-Based Optimization for Under-Sampling MRI KLA runner up team prize! Anyatama Makur, Jiaren Zou, Ning Lu, Yuhang Zhang Compressed sensing MRI (CS-MRI) recovers the images with under-sampled k-space measurements to accelerate scan times. The two fundamental components in CS- MRI are the under-sampling pattern and the reconstruction model. In this paper, we acquire both components simultaneously using an end-to-end learning framework named LOUPE (Learning-based Optimization of the Under-sampling PattErn). For a given sparsity constraint, this method trains a neural network model on full-resolution data that are under-sampled retrospectively, yielding a data-driven optimized sub-sampling pattern and a reconstruction model that is customized to the type of images represented in the training data. We adapt the original LOUPE algorithm in Pytorch and implement three different neural networks using both magnitude and complex MRI data. We also extended LOUPE from the specific case of 2D Cartesian sampling to the non-Cartesian scheme. Our experiments with single-coil knee MRI data show that the optimized sub-sampling pattern can offer significantly more accurate reconstructions compared to standard random uniform under-sampling schemes. https://github.com/NingLuUM/EECS556proj-LOUPEplus # Segmentation % alecsoc-amakuch-higginss-najariac Medical Image Segmentation with Gaussian Mixture Models Sean Higgins, Alexander Makuch, Cyrus Najarian, Alec Socha In this report, we describe several image segmentation models and present our results from the implementation and evaluation of the Double Gaussian mixture model described in [1] for the segmentation of various image sets (natural, synthetic, and medical images). We compare this model's performance to that of a standard GMM model, a standard k-means clustering model, and a convolutional neural network (CNN) segmentation model trained on our medical images data set [2]. Our performance metrics applied to evaluate these methods were the Probabilistic Rand (PR) index and the Dice Score, and we found that the CNN method generally performed much better than the other segmentation models, and that the DGMM performed better than the GMM and k-means for the natural and synthetic images but had poor performance for the medical images. https://github.com/AlecS19/EECS556W21_final_project # Registration % dinankg-dkucher-dmanwill-yeatsem KLA top team prize! Multimodal Image Registration: Comparison of Methods for 3D MRI to 3D Ultrasound Image Registration with Classical and Deep-Learning Accelerated Approaches Dinank Gupta, David Kucher, Daniel Manwiller, Ellen Yeats Multimodal image registration is the task of mapping images from different coordinate systems and different imaging modalities into a common coordinate system. We investigated Magnetic Resonance Imaging (MRI) to 3D Ultrasound (US) image registration, typically desired in neurosurgery applications, where intra-operative US provides real-time feedback to the surgeon but pre- operative MRI provides much better soft tissue contrast. We compared two state-of-the-art image processing based methods: Linear Correlation of Linear Combinations (LC2), Self-Similarity Context (SSC), and a U-Net based deep- learning approach to compare their registration accuracy. LC2 assumes that US intensities are linearly related to the MRI intensity and gradient and tries to learn a transformation to minimize error in this relation. SSC, on the other hand, assumes similarities in relationships of image neighborhoods in both images and tries to learn a transformation to make the neighborhood relations similar. Deep learning tries to minimize a loss function based on hand-matched landmarks in both images. We also tested whether the more roughly aligned output from the U-Net model could then be fed to a classical method for a more fine-tuned registration to potentially provide an overall improved accuracy and an accelerated convergence while maintaining its model robustness. https://github.com/mrdkucher/eecs556_MMIR # Compression % nilinyu-theroy-xuzhou-ztr Image Compression Optimization Based on Graph Transform Transform Tianrong Zhang, Linyu Ni, Xuzhou Li, Yi Liu In this project, we studied and implemented an input structure-aware Graph Fourier Transform (GFT) based image compression method, introduced by Giulia et. al. GFT based compression is known to be potentially more informative than Fourier Transform (FT) based ones due to the freedom to choose bases according to the image distribution. Such freedom comes with the extra cost to encode and transmit the graph description.The proposed method attempts to find the optimum of such trade-off by formulating it as a rate-distortion problem and solving it using convex optimizer. We successfully rebuilt the codes to compress both natural and piece-wise images using this proposed optimization GFT method. Comparing with classical discrete cosine transform (DCT) compression, this method can achieve higher peak signal-to-noise ratio (PSNR). In addition, this optimization GFT compression performs better on piece-wise images than natural images with higher average PSNR gain of 0.25 dB. https://github.com/nilinyu/EECS556-Project_-GFT-Compression # Enhancement % hdo-johnge-kikkeria-krausch KLA top team prize! Sentinel-2 Sharpening Using a Reduced-Rank Method With Modified Roughness Regularization Hyeonsu Do, John Gearig, Anusha Kikkeri, Konrad Rauscher Pansharpening, the process of improving low-resolution bands of an image through information from high-resolution bands, is important to construct high-fidelity satellite images. In the following paper, we propose improvements to S2Sharp, an optimization-based sharpening method, which sharpens 60-meter bands of images collected from the Sentinel-2 Satellite. We observe modest improvements in error metrics, convert S2Sharp from Matlab to Julia, and mathematically describe possible changes to the regularization used for the cost function. https://github.com/konradrauscher/eecs556-sharp