Signal and Image Processing

2D images and formation

from matplotlib import pyplot as p
import numpy.random as r
I = r.rand(256,256)*0.5;
p.imshow(I ,cmap='gray', vmin=0.0,vmax=1.0);
p.show();
Tip
  • rand(256,256) 生成一个 256x256 的随机矩阵, 值在 0-1 之间
  • cmap = 'gray' 表示使用灰度图显示
  • vmin 和 vmax 分别表示最小值和最大值
  • matplotlib.pyplot.imshow() 函数用于显示图像, 一般可以自动数值normalize到 0-1 之间
  • vmin 和 vmax 可以指定normalize的范围

Dimensionality of an Image

8 bit image: 256 gray levels

  • e.g. 256x256 = 65,536 pixels
  • 256 values ^ 65,536 pixels = 256^65,536

Image: 2D sampling of signal

  • Signal: function depending on some variable with physical meaning.
  • Image: sampling of that function.
    • 2 variables: xy coordinates
    • 3 variables: xy + time (video)
    • ‘Brightness’ is the value of the function for visible light
  • Can be other physical values too: temperature, pressure, depth …
    • 信号不仅限于亮度，还可以表示其他物理量。例如，红外图像可以表示温度，海洋探测图像可以表示深度，气象图像可以表示压力等。

Sampling

Sampling in 1D

Sampling in 1D

  • Sampling in 1D takes a function and returns a vector whose elements are values of that function at the sample points.

等间隔采样

Sampling in 2D

  • Sampling in 2D takes a function and returns a matrix.

Sampling in 2D

Quantization

  • Digital values – integer values
    • Quantization is lossy!! After quantization, the original signal cannot be reconstructed anymore
    • In contrast to sampling, as a sampled but not quantized signal can be reconstructed.
  • Simple quantization uses equally spaced levels with k intervals
    量化是一种将连续信号转换为离散值的过程。在简单量化中，整个信号的范围会被均匀地划分为 k 个区间（或称为级别）。这些区间之间的间隔是相同的，因此称为“等间隔”。每个信号的值都会被映射到其对应的区间，最终得到的是有限的离散值，而不是连续的信号。

Quantization Effects – Radiometric Resolution

Radiometric Resolution

Spatial Linear Filters

Four views of filtering

  • Image filters in spatial domain
  • Image filters in the frequency domain
  • Image filters in the temporal domain
  • Image pyramids

Image filters in spatial domain

  • Filter is a mathematical operation of a grid of numbers
  • Smoothing, sharpening, measuring texture
    • 平滑、锐化、测量纹理

Box filter

系数 用于保证整体亮度不发生变化

  • Usages
    • Enhance images
      • Denoise, resize, increase contrast, etc.
    • Extract information from images
      • Texture, edges, distinctive points, etc.
    • Detect patterns
      • Template matching

Sobel filter

  • 用于边缘检测

  • Check horizontal edge 横边

  • Check Vertical edge 竖边

相当于计算两侧的差值，差值越大，当前位置是边缘

Correlation and Convolution

Correlation and Convolution

Correlation

h=scipy.signal.correlated2d(f,I)
Convolution

h=scipy.signal.convolve2d(f,I)
  • Convolution is the same as correlation with a 180° rotated filter kernel.
  • Correlation and convolution are identical when the filter kernel is symmetric.

卷积与相关的关系：

  • 卷积可以看作是相关的一种变体，唯一的区别是卷积中的滤波器会先进行180度的旋转。
  • 当滤波器是对称的时，卷积和相关是相同的。这是因为旋转180度不会改变滤波器的形状。

Properties of linear filters

Linearity

线性滤波器的线性性意味着，对于输入图像 I 以及两个滤波器 f_1 和 f_2 ，对图像应用滤波器的结果等于先将滤波器相加，再应用一次滤波，或者分别应用每个滤波器，再将结果相加。
换句话说，滤波操作对滤波器的加法是可交换的，可以直接作用于滤波器的线性组合。

Shift invariance

平移不变性指的是滤波器的输出与像素位置无关，意思是无论图像中的像素在什么位置，滤波器对强度的操作结果都是相同的。也就是说，滤波器的应用不会因为图像的某个区域平移而改变输出。
在公式中，如果我们先将滤波器 f 平移，再对图像 I 进行滤波，这与先对图像 I 进行滤波，然后再平移滤波结果是等价的。

Any linear, shift-invariant operator can be represented as a convolution.

Convolution properties

1. Commutative

  • Conceptually no difference between filter and signal
  • But particular filtering implementations might break this equality
2. Associative

  • Often apply several filters one after another:
  • This is equivalent to applying one filter:
3. Distributes over addition

4. Scalars factor out

5. .Identity unit impulse

Gaussian filter

Gaussian filter

Property
  • Remove high-frequency components from the image (a low-pass filter)
    • Images become more smooth
  • Gaussian convolved with Gaussian is another Gaussian
  • Separable kernel
    • Factors into product of two 1D Gaussians

Match Object

Subtract the median

f = D[ 57:117, 107:167 ]
f2 = f – np.mean(f)
D2 = D – np.mean(D)
I2 = correlate2d( D2, f2, 'same' )

Now zero centered.
Score is higher only when dark parts
match and when light parts match.

通过将图像和子区域的均值归零，相关得分在模板的暗区和亮区与图像相应区域匹配时才会较高。这避免了只匹配亮度（而不考虑结构）的高得分情况。

Non-linear filters

Median filter & Mean filter

  • Operates over a window by selecting the median intensity in the window.
  • Rank filter as based on ordering of gray levels
    • E.G., min, max, range filters

选择窗口内的中位数/平均数

中值滤波器：

  • 能够保留图像的边缘和细节。
  • 有效去除盐噪声和胡椒噪声。
  • 对异常值有很好的鲁棒性。

均值滤波器：

  • 会模糊图像边缘，平滑噪声和细节。
  • 对极端噪声和异常值处理效果较差。
Table Of Contents
Signal and Image Processing
2D images and formation
Dimensionality of an Image
Image: 2D sampling of signal
Sampling
Sampling in 1D
Sampling in 2D
Quantization
Quantization Effects – Radiometric Resolution
Spatial Linear Filters
Image filters in spatial domain
Box filter
Sobel filter
Correlation and Convolution
Correlation
Convolution
卷积与相关的关系：
Properties of linear filters
Linearity
Shift invariance
Convolution properties
1. Commutative
2. Associative
3. Distributes over addition
4. Scalars factor out
5. .Identity unit impulse
Gaussian filter
Property
Match Object
Subtract the median
Non-linear filters
Median filter & Mean filter