Image Alignment and Stitching

Image Alignment Applications

Local Alignment

• Tracking
• Stereo vision

Global Alignment:

• Camera jitter elimination
• Image enhancement (e.g., denoising)
• Panoramic mosaicing (stitching multiple images to create panoramas)

Image Enhancement

• Process of improving image quality by removing noise and artifacts.

Panoramic Mosaics

• Combining multiple overlapping images into a single wide-angle view.

Gigapixel Images

• Extremely high-resolution images created by stitching many smaller images.

Applications

• Historical views: Align historical images with current ones.
• Streetside imagery (e.g., Google Street View).

Image Alignment Approaches

Direct Alignment

• Find image transformation that maximizes similarity in overlapping regions.
• Typically performed in a coarse-to-fine manner.

Feature-Based Alignment

• Uses keypoints and descriptors (e.g., SIFT) to align images.

Steps for Feature-Based Alignment:

1. Detect keypoints and compute SIFT(Scale-Invariant Feature Transform) descriptors.
2. Generate candidate keypoint matches.
3. Use RANSAC to select inliers from candidate matches.
4. Fit the best transformation using inliers.
5. Warp images according to the computed transformation.
6. Blend overlapping regions smoothly.

SIFT Descriptors

• Divide image patch into $4\times 4$ sub-patches.
• Compute histogram of gradient orientations (8 angles) per sub-patch.
• Result: 128-dimensional descriptor ($4\times 4\times 8=128$).

Candidate Matches:

• Match each descriptor from image A to the closest descriptor in image B using Euclidean distance:

$$\text{best\_match}(x)=\arg \underset{x^{\prime }}{\min }\Vert x-x^{\prime }{\Vert }^2$$

• Accelerate matching using k-d trees.
• Refinement strategies:
  • Mutual best match (both keypoints mutually closest).
  • Ratio test (ratio of second-best to best match distances, threshold ~0.8).

Problem: Ambiguous Correspondences

• Occurs when multiple similar matches exist; resolved using mutual best match or ratio test.

RANSAC Algorithm

• Robust method for selecting correct matches among outliers.

RANSAC Steps:

1. Randomly select minimal set of matches.
2. Compute transformation from selected matches.
3. Identify inliers consistent with the computed transformation.
4. Repeat multiple times; choose transformation with most inliers.
5. Re-estimate final transformation using all identified inliers.

2D Transformation Models

Fitting as Optimization:

Find transformation $$ T

T^* = \arg \min_T \sum_i L(T(x_i), x’_i)

## Warping Techniques **Forward Warping:** - Maps pixels from source to destination directly:

g(x’) = f(h(x))

$$\ast \ast InverseWarping:\ast \ast -Mapspixelsfromdestinationbacktosource;usedmorecommonlyduetofewerartifacts:$$

g(x’) = f(h^{-1}(x’))

**Interpolation:** Required when pixel values fall between discrete locations; typically bilinear interpolation is used. ## Blending Images - After warping, blend overlapping regions smoothly. ### Blending Techniques: 1. **Pyramid Blending:** - Gaussian Pyramid: progressively blurred versions of the image. - Laplacian Pyramid: captures details at different scales; computed as:

L_i = G_i - expand(G_{i+1})

2. **Multiband Blending:** - High frequencies blended over small regions to avoid ghosting. - Low frequencies blended over larger regions to avoid visible seams. 3. **Minimum-Cost Cut:** - Finds boundary between overlapping regions minimizing visual artifacts and blur. 4. **Poisson Blending:** - Blends images by matching gradients while satisfying boundary conditions set by destination image. ## Homography (Projective Transformation) - Transforms points between two planes or images taken from cameras sharing a center: General form:

x’ = \frac{ax + by + c}{gx + hy + i}, \quad y’ = \frac{dx + ey + f}{gx + hy + i}

$$Matrixform:$$

H = \begin{bmatrix} a & b & c\ d & e & f\ g & h & i \end{array}

**Homography Estimation:** Solve linear system $$ Ax=0 $$, typically solved via Singular Value Decomposition (SVD):

A x=0, \quad x=\text{eigenvector corresponding to smallest eigenvalue of } A^TA.

## Real-world Panoramic Stitching Challenges **Drift Problem:** - Aligning each frame sequentially can cause cumulative errors ("drift"). **Solution: Bundle Adjustment** - Optimizes all transformations simultaneously across multiple frames for global consistency. **Wide Field-of-view Problem:** - Cylindrical projection used for extremely wide panoramas to minimize distortion and preserve straight lines. --- These structured notes cover essential concepts and methods related to image alignment, stitching, feature-based alignment techniques, transformations, warping methods, and blending strategies—crucial topics likely to appear in your exam. <div style="text-align: center">⁂</div> [^1]: https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/52995484/f57fda47-397e-4fa5-8aa3-1c2384141553/cos429_f16_lecture06_alignment.pdf