EECS 556 W21 Class Topics - Jeff Fessler todo: possibly Mike McCann guest lecture near wavelets This list will be updated regularly (online) over the course of the semester. Class# topic 1 1/19 0.1 course policies *** Image processing overview (I: n-00-intro.pdf) 0.2 intro 0.3 pictorial overview 2 1/21 *** 2D CONTINUOUS-SPACE SIGNALS/SYSTEMS (CS: n-01-cont.pdf) 1.0 intro / overview 1.1 2D signals signal classes (symmetry, periodic) simple signal transformations 2D Dirac impulse / properties [video] sifting property comb nonlinear arguments [skim] 1.2 2D systems, input-output relationship [optional in W21] system classifications A-2 stability A-3 invertibility S-1 causality S-1' separability S-2 memory S-3 shift-invariance S-4 rotation-invariance A-1 linearity point spread function (PSF)/ impulse response impulse representation superposition integral (for linear systems) 1D pinhole camera 1.3 convolution (for LSI systems) [optional in W21] convolution properties system properties in terms of PSF: causality, memory, stability, invertibility, rot. inv. magnification convolution example: tri, chat 3 1/26 *** 2D FOURIER TRANSFORMS (FT: n-02-four.pdf) 2.1 orthogonal representation of signals generalized Fourier series ideas behind Fourier coefficient formula (finite approximation) Parseval's theorem separable bases 1D FS example (Harr) eigenfunctions of LSI systems 2D Fourier series convergence, completeness and Gibbs phenomenon (skim) properties 2D FS example FS of impulse train (comb) and "bed of nails" FS of finite-support images 4 1/28 h01 2.2 2D Fourier transform relate to 1D FT related to 2D FS (skim) existence / disconinuities / null functions (skim) Properties of 2D FT (same as 1D) linearity convolution correlation magnification (scaling) shift duality differentiation parseval's theorem symmetry properties (even, real) generalized FT (distributions) frequency response, OTF/MTF 2D FT: new properties rotation rotational symmetry separability circular symmetry Fourier transforming property of lenses 2D-FT of periodic functions (relation to 2D FS) Hankel transform 2D-FT Examples rms bandwidth, rms time duration (students read) time-bandwidth product, gaussian example 5 2/02 *** Sampling (SA: n-04-sample.pdf) 4.1 ideal rectilinear sampling nyquist sampling rate / sampling theorem sub-nyquist sampling 4.2 signal reconstruction Fourier approach sinc interpolation jinc interpolation 6 2/04 h02 4.3 aliasing Anti-aliasing property of lenses (PSF, frequency response) video displays 7 2/09 *** 2D discrete-space signals/systems (DS: n-05-disc.pdf) [quick (?) version of following topics] 5.1 2D Kronecker impulse function / properties notation, coordinate systems signal classes (symmetry, periodic, separable) circular symmetry 5.2 2D systems, input-output relationship examples: mov. avg., thresholding, down-sampling, up-sampling system classifications A-2 stability A-3 invertibility S-1 causality S-1' separability S-2 memory S-3 shift-invariance S-4 rotation-invariance A-1 linearity impulse response h[m,n] 8 2/11 h03 5.3 2D convolution convolution sum, finite-support considerations graphical convolution separable convolution edge effects convolution properties in 2D LSI system properties in terms of PSF: causality, memory, stability, invertibility, rotation invariance 2D correlation 9 2/16 [pre-proposal meetings parallel to class, steven whitaker covers class] 5.4 2D discrete-space Fourier transform (DSFT) eigenfunctions existence/convergence of FT periodic signals properties DSFT examples: discrete rect, impulse, sinusoids, periodic sampling revisited: DSFT vs CSFT inverse DSFT magnitude vs phase phase-only reconstruction / magnitude retrieval (brief) LSI systems in frequency domain Digital processing of analog images rotation invariance revisited 5.5 Introduction to filter design separability vs rotation invariance via examples *** Filters (FI: n-06-filt.pdf) [optional in W21 !!] 6 overview 6.1 ideal filter specifications ideal filter impulse response separable filters zero-phase filters filter symmetries and computation 6.2 IIR filters in 2D / Z transform - not 10 2/18 h04 *** 2D Discrete Fourier Transform (DFT: n-07-dft.pdf) 7.1 2D DS orthogonal representation 7.2 2D DFS Discrete Fourier series (just skim) properties circular convolution circular convolution example 11 2/23 7.3 2D DFT perspectives: orthonormal basis, periodization, sampling DSFT derivation relationship to DSFT properties circular convolution filtering using FFT zero padding overlap-add method sampling dsft sampling H(omega) example matrix representation of DFT (brief) 7.4 2D FFT row column decomposition 7.5 FT family relations 2D-DSFT / 2D-FT relationship 2D DFT/FFT, relationships to 2D-FT [2/24 wellness day] 12 2/25 [no hw!] 7.6 Numerical evaluation of CS FT using 2D FFT and fftshift 7.7 frequency-sampling methods for FIR filter design 7.8 DCT motivation derivation properties [2/26] proposal due 13 3/02 *** Interpolation (IN: n-08-interp.pdf) Image interpolation 8.0 sinc, separable, linear 8.1 polynomial: nearest ("rect"), linear bilinear, griddata interpolator properties Lagrange interpolation quadratic, cubic 8.2 shift-invariant subspaces / prefilter equivalent impulse response equivalent frequency response basis kernels supported on (-2,2) implementation using IIR filters end conditions B-spline interpolation 14 3/04 [h05] 8.3 applications image registration image rotation with separable operations FFT-based image zooming (up-sampling) interpft issues as a review 8.4 Motion estimation [SKIP] Motion-compensated interpolation region matching methods space-time constraint equation temporal interpolation 15 3/09 *** Image analysis (IA: n-09-analyze.pdf) 9.1 Edge detection basics gradient-based methods derivatives from discrete images Canny's method (brief) derivative of Gaussian (DoG) 16 3/11 h06 9.2 Edge detection using 2nd derivatives and beyond Laplacian-based methods Marr and Hildreth methods: Laplacian of Gaussian (LoG) parametric edge-detection methods texture image segmentation (brief) 9.3 Corner detection - Harris' method 17 3/16 *** Image enhancement (IE: n-10-enhance.pdf) 10.1 Contrast adjustment [optional] piecewise linear contrast adjustment gamma correction histograms histogram transformation histogram equalization pseudo-color / false color (skip) 10.2 Image sharpening [optional] 10.3 Image denoising: basic methods linear vs median filters median statistics 10.4 Image denoising: adaptive methods adaptive smoothing [skip] bilateral filter Non-local means (NLM): patch-based denoising 10.5 Image denoising: CNN methods Supervised methods DnCNN, DIDN, Noise2Noise Self-supervised methods SURE 18 3/18 h07 blind-spot networks: noise2void, noise2self *** Wiener Filter / Spectral estimation / random proc. *** (WF: n-11-wiener.pdf) [optional for 498-556 students] random vectors (skip) 11.1 random processes, WSS autocorrelation, properties power spectral density nonnegative definiteness of covariance matrices nonnegativity of power spectra (brief) 11.2 pairs of random processes cross-correlation, properties jointly WSS processes RP's through LSI systems synthesizing WSS random processes via filtering auto-correlation functions for filtered IID processes 3/23 [no class - wellness day] 19 3/25 11.3 Noncausal Wiener filter (MMSE denoising) deconvwnr and claims of optimality 11.4 Wiener for deblurring [read - didn't get to in class] 11.5 Spectral estimation [Optional in W21] periodogram, asymptotic unbiasedness cross-correlation and windowing fractal processes [skip] 11.6 Markov random field models [Optional in W21] Markov chains / random walk neighborhood / clique / potential functions Ising model Metropolis sampler example ising1.m demo [skip] 11.7 Image segmentation [Optional in W21] ML methods priors Ising model MRF methods: iterated conditional modes (ICM) [3/26 progress] 20 3/30 *** C1 Image restoration ([n-12] c-restore.pdf) 1.1 overview 1.2 conventional discrete convolution model 1.3 continuous to discrete modeling (skip) 1.4 matrix-vector representations of convolution diagonalization of circulant matrices vs DFT eigenvalues of circulant matrix 1.5 inverse filter / deconvolution (in both SP and matrix forms) 21 4/01 h08 1.6 statistical formulations: LS, ML 1.7 MAP estimation (skip MMSE) MAP with (IID) gaussian prior (cf Wiener filter) circulant analysis overview of model-based image restoration / reconstruction 1.8 regularized/penalized least squares roughness penalties differencing matrix C circulant analysis invertibility of A'A + C'C 1.9 mean and variance analysis PLS case, general linear case circulant analysis resolution-noise trade-off c-restore/fig/demo_res_wiener.m 22 4/06 1.8 and 1.9 with roughness penalty invertibility of A'A + C'C 1.10 nonquadratic PLS oracle edge-preserver in 1D edge-preserving penalty functions gradient of cost function nonlinear estimation as adaptive penalty weighting 1.11 algorithms for NPLS (skipped due to time) X optimization transfer / majorize-minimize X majorizers / surrogate functions X quadratic majorizer D >= H X paraboloidal surrogate algorithm (diagonal Hessian) X DePierro's trick for separating (skipped) X separable paraboloidal surrogate (SPS) algorithm X Huber's iteration preconditioned gradient descent (PGD) X diagonally PGD with diagonal majorizer GD with step-size based on spectral radius of Hessian X Lipschitz constant matrix 1-norm of A'A + C'D'C 2D finite differences npls_sps.m explanation X sparse matrix representation (brief) X uniqueness of PLS minimizer (convexity) iterative soft thresholding algorithm (ISTA) (brief) X FISTA 1.12 sparsity models? [4/7 exam] 23 4/08 [exam team work] 24 4/13 [isbi] *** Sparsity and wavelets (SP: n-13-sparse.pdf) sparsity after various transforms Haar wavelets and filter banks orthonormal haar wavelet transform in 2D 2D wavelets: separable vs matlab [did not cover due to time]: analysis vs synthesis approach l0 "norm" denoising using sparsity regularization hard thresholding using l0 soft threshold using l1 X demos of hard thresholding using DCT, Haar X over-complete / redundant dictionaries X examples of denoising, restoration ------- below here is from W18 (but with updated dates) 25 4/15 h09? [isbi] - Michael McCann ?? [4/16 report due] 26 4/20 [last class] X penalty approach to analysis-form regularization X alternating minimization X augmented Lagrangian minimization method X ADMM X super-resolution (multi-frame) X phase-retrieval (Gerchberg-Saxton) X super-resolution (Gerchberg-Papoulis) ------- dates below here from W19 27 4/?? Shrinkage for Fair potential 28 4/?? *** Image coding (IC: n-14-code.pdf) [all brief] quantization scalar quantization (uniform) quantizer design (nonuniform) companding high-rate scalar quantization bit allocation vector quantization codebook design k-means (LBG) algorithm X codeword assignment (brief) X bit allocation X uniform length X variable length X joint optimization thereof X Huffman, entropy waveform coding PCM Robert's pseudonoise Delta modulation X diff PCM, two-channel pyramid coding X analysis of Laplacian pyramid X adaptive coding transform coding KL transform KL derivation (brief) KLT vs DCT example Hadamard transform (brief) JPEG standard (brief) X other transforms practical issues subimages zones / bit allocation X artifacts hybrid transform coding adaptive coding X model coding (brief) X interframe coding (brief) 4/?? ?? ?-?pm final exam time [project presentations 4-?pm] --- other topics we could cover if there is time... X recent issue of T-IP X [project proposal feedback] *** C9 Image restoration overview degradation estimation motion blur shape statistics binary morphology Dec. 1999 T-IP issue wavelet image coding image analysis deformable templates object recognition