CSE 559A: Computer Vision Fall 2020: T-R: 11:30-12:50pm @ Wrighton 300 / Zoom Instructor: Ayan Chakrabarti ([email protected]). Course Staff: Adith Boloor, Patrick Williams Sep 24, 2020 http://www.cse.wustl.edu/~ayan/courses/cse559a/ 1 ADMINISTRIVIA ADMINISTRIVIA REMINDER: Office Hours start this Friday. Zoom link available through Canvas. Received your problem set 0 solutions. Will be giving individual feedback Soon. Meanwhile, a few points: Make sure your write-up is named solution.pdf, and not something else. Make sure you git add the solution.pdf file before commit & push. We need the PDF, not just the .tex. Don’t rename the .py files. Your submitted code should be added in to the original prob?.py files. Be a little careful about googling for answers to specific questions. Problem set 0 was pretty basic, so both were the same. But in problem set 1, don’t google: how to implement histogram equalization! Stack overflow type web-sites can especially get problematic sometimes. Do not use OpenCV! You have to implement everything using numpy / scipy. The point is to know how computer vision algorithms work and are implemented. 2 LAST TIME LAST TIME Talked about point-wise operations. Began talking about efficient implementations in numpy (with parallel elementwise operations, reshapes, and broadcasting). 3 POINTWISE OPERATIONS IN NUMPY POINTWISE OPERATIONS IN NUMPY Images themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul does matrix multiplication). But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determined order, as rows. Represent as: matrix: color images vector: grayscale images. 4
Transcript
CSE 559A: Computer Vision
Fall 2020: T-R: 11:30-12:50pm @ Wrighton 300 / Zoom
Instructor: Ayan Chakrabarti ([email protected]).Course Staff: Adith Boloor, Patrick Williams
Sep 24, 2020
http://www.cse.wustl.edu/~ayan/courses/cse559a/
1
ADMINISTRIVIAADMINISTRIVIAREMINDER: Office Hours start this Friday. Zoom link available through Canvas.
Received your problem set 0 solutions. Will be giving individual feedback Soon.Meanwhile, a few points:
Make sure your write-up is named solution.pdf, and not something else.Make sure you git add the solution.pdf file before commit & push. We need the PDF, not just the .tex.Don’t rename the .py files. Your submitted code should be added in to the original prob?.py files.Be a little careful about googling for answers to specific questions. Problem set 0 was pretty basic, so bothwere the same. But in problem set 1, don’t google: how to implement histogram equalization!
Stack overflow type web-sites can especially get problematic sometimes.Do not use OpenCV! You have to implement everything using numpy / scipy. The point is to know howcomputer vision algorithms work and are implemented.
2
LAST TIMELAST TIMETalked about point-wise operations.Began talking about efficient implementations in numpy (with parallel elementwise operations, reshapes,and broadcasting).
3
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:
matrix: color images
vector: grayscale images.
X
(HW) × 3
(HW) × 1
Y[n] = C X[n] ⇒ Y = ?
4
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:
matrix: color images
vector: grayscale images.
X
(HW) × 3
(HW) × 1
Y[n] = C X[n] ⇒ Y = X CT
# Begin with X as (H,W,3) array
Xflt = np.reshape(X,(-1,3)) # Flatten X to a (H*W, 3) matrix
Yflt = np.matmul(Xflt,C.T) # Post-multiply by C
Y = np.reshape(Yflt,X.shape) # Turn Y back to an image array
5
POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYOne last trick: what if your pointwise function is a lookup table (for grayscale images).Assume that you are trying to map all intensities in an array X, which is of type integer (uint8, etc.), and youhave a lookup table as a vector H, where you want to map an intensity f to H[f].So what you want to do is: Y[i,j] = H[X[i,j]]You can do this simply as Y = H[X]. Y will be the same shape as X (and not H).
6
CONVOLUTIONCONVOLUTION
7
CONVOLUTIONCONVOLUTION
8
CONVOLUTIONCONVOLUTION
9
CONVOLUTIONCONVOLUTION
10
CONVOLUTIONCONVOLUTION
11
CONVOLUTIONCONVOLUTION
12
CONVOLUTIONCONVOLUTION
13
CONVOLUTIONCONVOLUTION
14
CONVOLUTIONCONVOLUTION
15
CONVOLUTIONCONVOLUTION
Double summation over the support / size of the kernel
We assume is scalar vaued.
If is scalar, so is .
If is a color image, each channel convolved with independently.
To go from m to n channels in a “conv layer”: is matrix valued, and is a matrix-vector product.
Notation: Y = X ∗ k
Y[n] = k[ ] X[n − ]∑n′
n′
n′
Y[ , ] = k[ , ] X[( − ), ( − )]nx ny ∑n′x
∑n′y
n′x n′y nx n′x ny n′y
k
k[n] ∈ ℝ
X[n] Y[n]
X k
k[n] ∈ ℝn×mk[ ] X[n − ]n
′n
′
16
CONVOLUTIONCONVOLUTION
17
CONVOLUTIONCONVOLUTION
18
CONVOLUTIONCONVOLUTION
19
CONVOLUTIONCONVOLUTION
20
CONVOLUTIONCONVOLUTION
21
CONVOLUTIONCONVOLUTION
22
CONVOLUTIONCONVOLUTION
23
CONVOLUTIONCONVOLUTION
24
CONVOLUTIONCONVOLUTION
25
CONVOLUTIONCONVOLUTION
26
CONVOLUTIONCONVOLUTION
27
CONVOLUTIONCONVOLUTION
28
CONVOLUTIONCONVOLUTION
29
CONVOLUTIONCONVOLUTION
30
CONVOLUTIONCONVOLUTION
31
CONVOLUTION: PROPERTIESCONVOLUTION: PROPERTIESLet , , and denote full, valid, and same convolution (with zero padding for full and same)
Linear / Distributive: For scalars ;
If , then:
If and , ( , same size):
If and , ( , same size):
X k∗full
X k∗val
X k∗same
α, β
Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY
= X ∗Y1 k1 = X ∗Y2 k2 k1 k2 X ∗ (α + β ) = α + βk1 k2 Y1 Y2
