Skip to main content

Motion 运动

Why is Vision Interesting? 为什么视觉有趣？​

  • Psychology 心理學
    • ~ 35% of cerebral cortex is for vision. 大約 35％的大腦皮層是用於視覺。
    • Vision is how we experience the world. 視覺是我們體驗世界的方式。
  • Engineering 工程
    • Want machines to interact with world. 想要機器與世界互動。
    • Digital images are everywhere. 數字影像無處不在。

Vision is inferential: Light 視覺是推理：光​

1

2

http://www-bcs.mit.edu/people/adelson/checkershadow_illusion.html

Vision is Inferential: Prior Knowledge 視覺是推理：先前的知識​

3

4

Boundary Detection 邊界檢測​

5

http://www.robots.ox.ac.uk/~vdg/dynamics.html

6

7

Tracking 追蹤​

8 9 10 11 12

Optical flow 光流​

13

14

Video Mosaics 影片拼貼​

15

16

17

Video Compression 影片壓縮​

18

Geo Registration 地理註冊​

19

Aims 宗旨​

  • How to detect what changes have occurred 如何檢測發生了什麼變化
  • How to determine what motions have induced those changes 如何確定哪些動作引起了這些變化
  • We will learn some other useful things on the way: 我們將在這個過程中學習一些其他有用的東西：
    • Notion of 4 and 8 neighbours 4 和 8 個鄰居的概念
    • Notion of 4 and 8 connectedness 4 和 8 連接的概念
    • Using these for noise elimination 使用這些來消除噪音
    • Moravec Interest Operator Moravec 關注運算符

Dynamic Vision 活動視覺​

  • FOUR possibilities for dynamic nature of camera and world: 相機和世界的動態性有四種可能性：
    • Stationary Camera, stationary Objects 靜止相機，靜止物體
    • Stationary Camera, moving Objects 靜止相機，移動物體
    • Moving Camera, stationary Objects 移動相機，靜止物體
    • Moving Camera, moving Objects 移動相機，移動物體

Detecting a change 檢測變化​

  • Where has an image changed? 圖像發生了變化嗎？

20

Detecting a change 檢測變化​

  • Where has an image changed? 圖像發生了變化嗎？
  • F(x,y,i) is the intensity of the image at time i at point x,y F(x,y,i)是時間 i 點 x,y 的圖像強度
  • Difference picture, DP is the absolute difference between the two images: 差異圖 DP 是兩個圖像之間的絕對差異：
    DP12(x,y)={1 if ∣F(x,y,1)−F(x,y,2)∣>τ0 otherwise D P_{12}(x, y)= \begin{cases}1 & \text { if }|F(x, y, 1)-F(x, y, 2)|>\tau \\ 0 & \text { otherwise }\end{cases}

21

  • If we have 'random' noise in image the difference picture will include all the noise points in both images 如果圖像中有"隨機"噪聲，則差異圖將包含兩個圖像中的所有噪聲點

22

  • If we have 'random' noise in image the difference picture will include all the noise points in both images 如果圖像中有"隨機"噪聲，則差異圖將包含兩個圖像中的所有噪聲點 23
  • Filter out all pixels that are not part of a larger structure--- how? 濾除所有不是較大結構的一部分的像素

Connectedness 連接​

  • To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

    • Two pixels are 4-neighbours if they share a common boundary 4-neighbours 是共享一個公共邊界的兩個像素

    • Two pixels are 8-neighbours if they share at least a common corner 8-neighbours 是至少共享一個公共角的兩個像素

    • Two pixels are 8 connected if we can create a path of 8-neighbours from one to the other 兩個像素是 8 連接的，如果我們可以從一個像素到另一個像素創建一條 8-neighbours 的路徑

Connectedness 連接​

To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

  • Two pixels are 4-neighbours if they share a common boundary 4-neighbours 是共享一個公共邊界的兩個像素 24

Connectedness 連接​

To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

  • Two pixels are 4-neighbours if they share a common boundary 4-neighbours 是共享一個公共邊界的兩個像素 25

Connectedness 連接​

To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

  • Two pixels are 8-neighbours if they share at least a common corner 8-neighbours 是至少共享一個公共角的兩個像素 26

Connectedness 連接​

To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

  • Two pixels are 8-neighbours if they share at least a common corner 8-neighbours 是至少共享一個公共角的兩個像素 27

Connectedness 連接​

To filter out noise, we can use the idea of 8 or 4 connectedness: 為了消除噪聲，我們可以使用 8 或 4 連接的概念：

  • Suppose that P and Q are any two pixels (not necessarily adjacent), and suppose P and Q can be joined by a sequence of pixels as shown 假設 P 和 Q 是任意兩個像素（不一定相鄰），並假設 P 和 Q 可以通過一系列像素連接

If the path contains 8-adjacent pixels, then P and Q are 8- connected. 如果路徑包含 8 個相鄰像素，則 P 和 Q 是 8 個連通的。 28

Removing Noise 移除噪聲​

  • Pixels not in a connected cluster of a certain size are removed from the difference image 未連接到一個特定大小的連接群集的像素從差異圖中移除 29
  • In MATLAB type ' help bwlabel '

Determining motion 確定運動​

  • We want to determine the motion of bodies in the image 我們想要確定圖像中物體的運動
  • There is not enough information in the local intensity changes in an image to determine the motion of the objects in the image 在圖像中的局部強度變化中沒有足夠的信息來確定圖像中物體的運動

30

The aperture problem 光圈問題​

  • The grating appears to be moving down and to the right 這個格條似乎向下和向右移動
  • But it could be moving in many other directions 但它可以向許多其他方向移動
    • such as only down, or only to the right. 31 例如，只向下或只向右。

The aperture problem 光圈問題​

32

  • Locally we only see the horizontal component of the motion (the grating is moving to the right) 在局部，我們只能看到運動的水平分量（格條向右移動）
  • What ways round this are there? 有什麼方法可以解決這個問題？

Motion correspondence 運動對應​

  • One way is to pick a bunch of interesting points in each image 一種方法是在每個圖像中選擇一堆有趣的點
  • And then match them (hoping they can be matched) 33
  • e.g. pick corner points using a corner detector 例如，使用角點檢測器選擇角點
  • Or use a Moravec operator 或者使用 Moravec 運算符

Moravec OperatorMoravec 算子​

  • The Moravec operator is considered a corner detector Moravec 運算符被認為是一個角點檢測器
    • it defines interest points as points where there is a large intensity variation in every direction. 它定義了興趣點為在每個方向上強度變化很大的點。
    • This is the case at corners. 這是角點的情況。
    • It may be sensitive to detecting isolated pixels as corners. 它可能對檢測孤立像素作為角點很敏感。 34

Moravec Operator Moravec 算子​

  • measure the intensity variation by placing a small square window (typically, 3x3, 5x5, or 7x7 pixels) centered at P 通過在 P 的中心放置一個小方形窗口（通常是 3x3，5x5 或 7x7 像素）來衡量強度變化
    • then shifting this window by one pixel in each of the eight principle directions (horizontally, vertically, and four diagonals). 然後在八個主要方向（水平，垂直和四個對角線）中將此窗口平移一個像素。 35

The Moravec Interest Operator Moravec 興趣算子​

  • We now apply the Moravec operator to every point (i,j) in the image 我們現在將 Moravec 運算符應用於圖像中的每個點（i，j）
  • This produces a new interest image 這產生了一個新的興趣圖像
  • We keep only the local maxima of this image 我們只保留這個圖像的局部最大值
    • (i.e. points that are locally most interesting) 也就是說，局部最有趣的點 36

Motion Correspondence 運動對應​

  • Now we have our points of interest 現在我們有了我們的興趣點 37
  • We can try to match points in one image with those in another 我們可以嘗試將一個圖像中的點與另一個圖像中的點匹配 38
  • By doing this we are able toestimate the motion of the objects in the image 通過這樣做，我們能夠估計圖像中物體的運動

Motion Correspondence 運動對應​

  • Motion correspondence (matching) is guided by three principles 運動對應（匹配）受三個原則的指導
    1. Discreteness: a measure of the distinctiveness of individual points Discreteness：個別點的獨特性的度量
    2. Similarity: a measure of how closely two points resemble one another Similarity：兩個點相似程度的度量
    3. Consistency: a measure of how well a match conforms with nearby matches Consistency：一個匹配與附近匹配的程度的度量

Motion correspondence 運動對應​

  • A sketch of the algorithm 運動對應算法的草圖
    • Get the interest points in each image 在每個圖像中獲取興趣點
    • Pair each feature point in the first image with every feature point in the other image within some distance of it 將第一個圖像中的每個特徵點與其他圖像中的每個特徵點配對，並在其附近的某個距離內
    • Calculate the degree of similarity between the images for each possible match 計算每個可能匹配的圖像之間的相似度
    • Use this to calculate the likelihood of each match 使用這個來計算每個匹配的可能性
    • Now revise the match likelihood for each point using the nearby matches 現在使用附近的匹配來修訂每個點的匹配可能性 39

Calculating the degree of similarity between two patches 計算兩個補丁之間的相似度​

  • To calculate the degree of similarity between two patches take the sum of the differences in the pixel values of the two patches 要計算兩個補丁之間的相似度，請將兩個補丁的像素值之間的差異之和 40

i(t) is the patch in the first image j(t+1) is a patch in the second image sijs_{ij} is the similarity between i and j wijw_{ij} is a weight

wij=11+αsijw_{i j}=\frac{1}{1+\alpha s_{i j}}

Calculating the likelihood of a match 計算匹配的可能性​

  • Now we calculate the weights for all the possible matches for patch i(t) 現在我們計算所有可能的匹配的權重

41

42

43

The likelihood of a match 匹配的可能性​

  • Assume α = 1 44

Adjusting probabilities 調整概率​

  • So now we have a probability for each match 現在我們有每個匹配的概率

45

Summary 摘要​

  • Change detection
  • Making change detection more robust to noise 使變化檢測更能抵抗噪聲
  • Determining motion of objects 確定物體的運動
  • Aperture problem 光圈問題
  • Motion Correspondence 運動對應