EECS 556 W21 Class Topics - Jeff Fessler
todo: possibly Mike McCann guest lecture near wavelets
This list will be updated regularly (online) over the course of the semester.
Class# topic
1 1/19
0.1 course policies
*** Image processing overview (I: n-00-intro.pdf)
0.2 intro
0.3 pictorial overview
2 1/21
*** 2D CONTINUOUS-SPACE SIGNALS/SYSTEMS (CS: n-01-cont.pdf)
1.0 intro / overview
1.1 2D signals
signal classes (symmetry, periodic)
simple signal transformations
2D Dirac impulse / properties [video]
sifting property
comb
nonlinear arguments [skim]
1.2 2D systems, input-output relationship [optional in W21]
system classifications
A-2 stability
A-3 invertibility
S-1 causality
S-1' separability
S-2 memory
S-3 shift-invariance
S-4 rotation-invariance
A-1 linearity
point spread function (PSF)/ impulse response
impulse representation
superposition integral (for linear systems)
1D pinhole camera
1.3 convolution (for LSI systems) [optional in W21]
convolution properties
system properties in terms of PSF:
causality, memory, stability, invertibility, rot. inv.
magnification
convolution example: tri, chat
3 1/26
*** 2D FOURIER TRANSFORMS (FT: n-02-four.pdf)
2.1 orthogonal representation of signals
generalized Fourier series
ideas behind Fourier coefficient formula (finite approximation)
Parseval's theorem
separable bases
1D FS example (Harr)
eigenfunctions of LSI systems
2D Fourier series
convergence, completeness and Gibbs phenomenon (skim)
properties
2D FS example
FS of impulse train (comb) and "bed of nails"
FS of finite-support images
4 1/28 h01
2.2 2D Fourier transform
relate to 1D FT
related to 2D FS (skim)
existence / disconinuities / null functions (skim)
Properties of 2D FT (same as 1D)
linearity
convolution
correlation
magnification (scaling)
shift
duality
differentiation
parseval's theorem
symmetry properties (even, real)
generalized FT (distributions)
frequency response, OTF/MTF
2D FT: new properties
rotation
rotational symmetry
separability
circular symmetry
Fourier transforming property of lenses
2D-FT of periodic functions (relation to 2D FS)
Hankel transform
2D-FT Examples
rms bandwidth, rms time duration (students read)
time-bandwidth product, gaussian example
5 2/02
*** Sampling (SA: n-04-sample.pdf)
4.1 ideal rectilinear sampling
nyquist sampling rate / sampling theorem
sub-nyquist sampling
4.2 signal reconstruction
Fourier approach
sinc interpolation
jinc interpolation
6 2/04 h02
4.3 aliasing
Anti-aliasing property of lenses
(PSF, frequency response)
video displays
7 2/09
*** 2D discrete-space signals/systems (DS: n-05-disc.pdf)
[quick (?) version of following topics]
5.1 2D Kronecker impulse function / properties
notation, coordinate systems
signal classes (symmetry, periodic, separable)
circular symmetry
5.2 2D systems, input-output relationship
examples: mov. avg., thresholding, down-sampling, up-sampling
system classifications
A-2 stability
A-3 invertibility
S-1 causality
S-1' separability
S-2 memory
S-3 shift-invariance
S-4 rotation-invariance
A-1 linearity
impulse response h[m,n]
8 2/11 h03
5.3 2D convolution
convolution sum, finite-support considerations
graphical convolution
separable convolution
edge effects
convolution properties in 2D
LSI system properties in terms of PSF:
causality, memory, stability, invertibility, rotation invariance
2D correlation
9 2/16 [pre-proposal meetings parallel to class, steven whitaker covers class]
5.4 2D discrete-space Fourier transform (DSFT)
eigenfunctions
existence/convergence of FT
periodic signals
properties
DSFT examples: discrete rect, impulse, sinusoids, periodic
sampling revisited: DSFT vs CSFT
inverse DSFT
magnitude vs phase
phase-only reconstruction / magnitude retrieval (brief)
LSI systems in frequency domain
Digital processing of analog images
rotation invariance revisited
5.5 Introduction to filter design
separability vs rotation invariance via examples
*** Filters (FI: n-06-filt.pdf) [optional in W21 !!]
6 overview
6.1 ideal filter specifications
ideal filter impulse response
separable filters
zero-phase filters
filter symmetries and computation
6.2 IIR filters in 2D / Z transform - not
10 2/18 h04
*** 2D Discrete Fourier Transform (DFT: n-07-dft.pdf)
7.1 2D DS orthogonal representation
7.2 2D DFS Discrete Fourier series (just skim)
properties
circular convolution
circular convolution example
11 2/23
7.3 2D DFT
perspectives:
orthonormal basis, periodization, sampling DSFT
derivation
relationship to DSFT
properties
circular convolution
filtering using FFT
zero padding
overlap-add method
sampling dsft
sampling H(omega) example
matrix representation of DFT (brief)
7.4 2D FFT
row column decomposition
7.5 FT family relations
2D-DSFT / 2D-FT relationship
2D DFT/FFT, relationships to 2D-FT
[2/24 wellness day]
12 2/25 [no hw!]
7.6 Numerical evaluation of CS FT using 2D FFT and fftshift
7.7 frequency-sampling methods for FIR filter design
7.8 DCT
motivation
derivation
properties
[2/26] proposal due
13 3/02
*** Interpolation (IN: n-08-interp.pdf)
Image interpolation
8.0 sinc, separable, linear
8.1 polynomial: nearest ("rect"), linear
bilinear, griddata
interpolator properties
Lagrange interpolation
quadratic, cubic
8.2 shift-invariant subspaces / prefilter
equivalent impulse response
equivalent frequency response
basis kernels supported on (-2,2)
implementation using IIR filters
end conditions
B-spline interpolation
14 3/04 [h05]
8.3 applications
image registration
image rotation with separable operations
FFT-based image zooming (up-sampling)
interpft issues as a review
8.4 Motion estimation [SKIP]
Motion-compensated interpolation
region matching methods
space-time constraint equation
temporal interpolation
15 3/09
*** Image analysis (IA: n-09-analyze.pdf)
9.1 Edge detection basics
gradient-based methods
derivatives from discrete images
Canny's method (brief) derivative of Gaussian (DoG)
16 3/11 h06
9.2 Edge detection using 2nd derivatives and beyond
Laplacian-based methods
Marr and Hildreth methods: Laplacian of Gaussian (LoG)
parametric edge-detection methods
texture image segmentation (brief)
9.3 Corner detection - Harris' method
17 3/16
*** Image enhancement (IE: n-10-enhance.pdf)
10.1 Contrast adjustment [optional]
piecewise linear contrast adjustment
gamma correction
histograms
histogram transformation
histogram equalization
pseudo-color / false color (skip)
10.2 Image sharpening [optional]
10.3 Image denoising: basic methods
linear vs median filters
median statistics
10.4 Image denoising: adaptive methods
adaptive smoothing [skip]
bilateral filter
Non-local means (NLM): patch-based denoising
10.5 Image denoising: CNN methods
Supervised methods
DnCNN, DIDN, Noise2Noise
Self-supervised methods
SURE
18 3/18 h07
blind-spot networks: noise2void, noise2self
*** Wiener Filter / Spectral estimation / random proc.
*** (WF: n-11-wiener.pdf) [optional for 498-556 students]
random vectors (skip)
11.1 random processes, WSS
autocorrelation, properties
power spectral density
nonnegative definiteness of covariance matrices
nonnegativity of power spectra (brief)
11.2 pairs of random processes
cross-correlation, properties
jointly WSS processes
RP's through LSI systems
synthesizing WSS random processes via filtering
auto-correlation functions for filtered IID processes
3/23 [no class - wellness day]
19 3/25
11.3 Noncausal Wiener filter (MMSE denoising)
deconvwnr and claims of optimality
11.4 Wiener for deblurring [read - didn't get to in class]
11.5 Spectral estimation [Optional in W21]
periodogram, asymptotic unbiasedness
cross-correlation and windowing
fractal processes
[skip] 11.6 Markov random field models [Optional in W21]
Markov chains / random walk
neighborhood / clique / potential functions
Ising model
Metropolis sampler example
ising1.m demo
[skip] 11.7 Image segmentation [Optional in W21]
ML methods
priors
Ising model
MRF methods: iterated conditional modes (ICM)
[3/26 progress]
20 3/30
*** C1 Image restoration ([n-12] c-restore.pdf)
1.1 overview
1.2 conventional discrete convolution model
1.3 continuous to discrete modeling (skip)
1.4 matrix-vector representations of convolution
diagonalization of circulant matrices vs DFT
eigenvalues of circulant matrix
1.5 inverse filter / deconvolution (in both SP and matrix forms)
21 4/01 h08
1.6 statistical formulations:
LS, ML
1.7 MAP estimation (skip MMSE)
MAP with (IID) gaussian prior (cf Wiener filter)
circulant analysis
overview of model-based image restoration / reconstruction
1.8 regularized/penalized least squares
roughness penalties
differencing matrix C
circulant analysis
invertibility of A'A + C'C
1.9 mean and variance analysis
PLS case, general linear case
circulant analysis
resolution-noise trade-off
c-restore/fig/demo_res_wiener.m
22 4/06
1.8 and 1.9 with roughness penalty
invertibility of A'A + C'C
1.10 nonquadratic PLS
oracle edge-preserver in 1D
edge-preserving penalty functions
gradient of cost function
nonlinear estimation as adaptive penalty weighting
1.11 algorithms for NPLS (skipped due to time)
X optimization transfer / majorize-minimize
X majorizers / surrogate functions
X quadratic majorizer D >= H
X paraboloidal surrogate algorithm (diagonal Hessian)
X DePierro's trick for separating (skipped)
X separable paraboloidal surrogate (SPS) algorithm
X Huber's iteration
preconditioned gradient descent (PGD)
X diagonally PGD with diagonal majorizer
GD with step-size based on
spectral radius of Hessian
X Lipschitz constant
matrix 1-norm of A'A + C'D'C
2D finite differences
npls_sps.m explanation
X sparse matrix representation (brief)
X uniqueness of PLS minimizer (convexity)
iterative soft thresholding algorithm (ISTA) (brief)
X FISTA
1.12 sparsity models?
[4/7 exam]
23 4/08 [exam team work]
24 4/13 [isbi]
*** Sparsity and wavelets (SP: n-13-sparse.pdf)
sparsity after various transforms
Haar wavelets and filter banks
orthonormal haar wavelet transform in 2D
2D wavelets: separable vs matlab
[did not cover due to time]:
analysis vs synthesis approach
l0 "norm"
denoising using sparsity regularization
hard thresholding using l0
soft threshold using l1
X demos of hard thresholding using DCT, Haar
X over-complete / redundant dictionaries
X examples of denoising, restoration
------- below here is from W18 (but with updated dates)
25 4/15 h09? [isbi] - Michael McCann ??
[4/16 report due]
26 4/20 [last class]
X penalty approach to analysis-form regularization
X alternating minimization
X augmented Lagrangian minimization method
X ADMM
X super-resolution (multi-frame)
X phase-retrieval (Gerchberg-Saxton)
X super-resolution (Gerchberg-Papoulis)
------- dates below here from W19
27 4/??
Shrinkage for Fair potential
28 4/??
*** Image coding (IC: n-14-code.pdf) [all brief]
quantization
scalar quantization (uniform)
quantizer design (nonuniform)
companding
high-rate scalar quantization
bit allocation
vector quantization
codebook design
k-means (LBG) algorithm
X codeword assignment (brief)
X bit allocation
X uniform length
X variable length
X joint optimization thereof
X Huffman, entropy
waveform coding
PCM
Robert's pseudonoise
Delta modulation
X diff PCM, two-channel
pyramid coding
X analysis of Laplacian pyramid
X adaptive coding
transform coding
KL transform
KL derivation (brief)
KLT vs DCT example
Hadamard transform (brief)
JPEG standard (brief)
X other transforms
practical issues
subimages
zones / bit allocation
X artifacts
hybrid transform coding
adaptive coding
X model coding (brief)
X interframe coding (brief)
4/?? ?? ?-?pm final exam time [project presentations 4-?pm]
--- other topics we could cover if there is time...
X recent issue of T-IP
X [project proposal feedback]
*** C9 Image restoration
overview
degradation estimation
motion blur
shape statistics
binary morphology
Dec. 1999 T-IP issue
wavelet image coding
image analysis
deformable templates
object recognition