Optical Flow

Motion Vector (u,v)

A motion vector represents the displacement of pixels between consecutive frames in a video sequence. Specifically:

• u = dx/dt: Represents the horizontal component (x-direction) of the motion
• v = dy/dt: Represents the vertical component (y-direction) of the motion[^4]

These vectors are derived from the optical flow equation:

$$\nabla I\cdot \mathbf{v}+I_t=0$$

where $\nabla I$ is the spatial gradient of the image intensity, $\mathbf{v}$ is the motion vector (u,v), and $I_t$ is the temporal derivative[^4].

Image Displacement

Image displacement refers to the change in position of pixels or objects between consecutive frames. This displacement is measured by:

1. Mean Absolute Difference (MAD): A common metric used to quantify displacement between macroblocks in consecutive frames[^3]
2. Sum of Squared Differences (SSD) or Sum of Absolute Differences (SAD): Used to compare image patches and find corresponding features between frames[^9]

The displacement vector from one point to another is represented as an arrow with its tail at the first point and its tip at the second, with magnitude equal to the distance between the points[^7].

Motion Flow Estimation Techniques

There are two primary approaches to optical flow estimation:

Sparse Optical Flow

• Calculates flow vectors for specific “interesting features” (like corners or edges)
• Example: Lucas-Kanade method, which assumes neighboring pixels have similar motion[^1][^4]
• More computationally efficient but less detailed

Dense Optical Flow

• Calculates flow vectors for every pixel in the image
• Example: Farneback algorithm, which uses polynomial expansion to model intensity changes[^5][^11]
• More accurate but computationally intensive[^4][^5]

Disparity

Disparity refers to the difference in image location of an object seen by the left and right eyes or cameras, resulting from horizontal separation[^9]. In stereo vision:

• Disparity (d) is inversely proportional to the distance (z) from the observer
• Objects far away have small disparity; nearby objects have large disparity[^8]
• Can be computed using methods like:
  • Block Matching (comparing SAD of pixel blocks)
  • Semi-Global Matching (adding neighborhood constraints)
  • Deep learning approaches (PSMNet and other volumetric methods)[^12][^13]

The relationship between disparity and depth is given by:

$$d=\frac{f\times b}{z}$$

where f is focal length, b is baseline (distance between cameras), and z is depth[^8].

Applications

Optical flow and disparity estimation have numerous applications:

• Object tracking and detection
• Video stabilization
• Autonomous navigation
• Motion analysis
• Depth estimation in computer vision systems[^5][^11]