Computer Vision - Texture

1. What is Texture?

• Definition: Texture is easy to recognize visually but difficult to define precisely. It generally refers to:
  • Views of a large number of small objects (e.g., grass, foliage, pebbles).
  • Surfaces exhibiting patterns (e.g., spots, stripes, wood grain, skin).
• Key Characteristics: Texture consists of organized patterns of regular sub-elements.
• Scale Dependence: Whether a visual pattern is considered “texture” depends on the scale at which it’s viewed. A single leaf is not texture, but a field of leaves is.

2. Texture-Related Problems

Several computer vision tasks involve texture:

• Texture Analysis: Representing and modeling texture. The goal is to find a mathematical description of a texture.
• Texture Segmentation: Dividing an image into regions where the texture is consistent. This is like object segmentation, but based on texture instead of, say, color.
• Texture Synthesis: Generating large areas of texture from small example patches. Think of this like “texture cloning.”
• Shape from Texture: Inferring the 3D shape or orientation of a surface from the texture in the image. Texture gradients (changes in texture) provide cues about surface orientation.

3. Representing Texture: Textons

• The Goal: To represent texture, we look for the regular sub-elements that compose it. These sub-elements are sometimes called “textons.”
• Approach:
  1. Find the sub-elements: Identify the repeating patterns.
  2. Represent their statistics: Describe the characteristics of the sub-elements (e.g., size, shape, orientation).
  3. Reason about their spatial layout: How are the sub-elements arranged relative to each other?
• Challenge: There is no universally agreed-upon, definitive set of textons.

4. Texture Analysis Approaches

Several approaches are used for texture analysis:

• Co-occurrence Matrices (Classical): A statistical method based on the spatial relationships between pixel intensities.
• Spatial Filtering: Applying filters (like edge detectors or Gabor filters) to extract texture features.
• Random Field Models: Statistical models that describe the probability of pixel values given their neighbors. (Not covered in detail in these slides).

5. Co-occurrence Matrix Features

• Objective: To capture the spatial relationships between pixel intensities in a texture.

• Co-occurrence Matrix (C): A 2D array where:

  • Rows and columns represent the possible image intensity values.
  • $C_d(i,j)$ counts how many times intensity value i occurs next to intensity value j with a specific spatial relationship d.
  • The spatial relationship d is defined by a vector $d=(dr,dc)$, representing the offset in rows (dr) and columns (dc) between the two pixels.

Example:

Consider a small grayscale image and a displacement vector $d=(3,1)$:

1 1 0 0
1 1 0 0
0 0 2 2
0 0 2 2
0 0 2 2
0 0 2 2

The co-occurrence matrix $C_d$ would be:

   0 1 2
0 [1 0 3]
1 [2 0 2]
2 [0 0 1]

• Normalized Co-occurrence Matrix ($N_d$): Each entry in $C_d$ divided by sum of entire matrix. This makes co-occurrence statistics less sensitive to number of pixels.

7.Quantitative Features from Co-occurrence Matrices:

• Energy: $Energy={\sum }_i{\sum }_j{N}_d(i,j{)}^2$ (Measures uniformity. Higher for homogeneous textures.)

  • What it measures: The uniformity or homogeneity of the texture.
  • Intuition:
    • High energy = The texture is very uniform (only a few gray-level pairs are common). The GLCM will have a few large values, possibly close to the diagonal if the pixel pairs often have similar gray level values.
    • Low energy = The texture is less uniform, with many different gray-level pairs present.

• Entropy: $Entropy=-{\sum }_i{\sum }_j{N}_d(i,j){\log }_2{N}_d(i,j)$ (Measures disorder. Higher for more random textures.)

  • What it measures: The randomness or disorder in the texture.
  • Intuition:
    • High entropy = Many different gray-level pairs occur with similar probability (like a noisy or very complex texture). The GLCM will have values spread out.
    • Low entropy = Only a few gray-level pairs dominate (like a very regular or uniform texture). The GLCM will have a few large values.

• Contrast: $Contrast={\sum }_i{\sum }_j(i-j{)}^2{N}_d(i,j)$ (Measures local variations. Higher for textures with large intensity differences.)

  • What it measures: The amount of local variation in gray levels in the texture.
  • Intuition:
    • High contrast = Large differences between gray levels of neighboring pixels are common (like a texture with sharp edges or large variations between light and dark areas). Values further away from the diagonal of the GLCM will be larger.
    • Low contrast = Neighboring pixels tend to have similar gray levels (like a smooth texture). Values along the diagonal will be larger.

• Homogeneity: $Homogeneity={\sum }_i{\sum }_j\frac{N_d(i,j)}{1+|i-j|}$ (Measures similarity. Higher for textures with similar pixel values.)

  • What it measures: How close the distribution of elements in the GLCM is to the GLCM diagonal. This indicates how similar the pixel pairs are in terms of their gray level values.
  • Intuition:
    • High homogeneity = Pixel pairs tend to have very similar gray level values (the texture is locally smooth). Values are concentrated along the diagonal of the GLCM.
    • Low homogeneity = Pixel pairs often have different gray level values (the texture has more local variations).

• Correlation: $Correlation=\frac{{\sum }_i{\sum }_j(i-{\mu }_i)(j-{\mu }_j)N_d(i,j)}{{\sigma }_i{\sigma }_j}$ (Measures linear dependencies. ${\mu }_i$, ${\mu }_j$ are means, and ${\sigma }_i$, ${\sigma }_j$ are standard deviations of rows and columns.)

• Disadvantages of Co-occurrence Matrices:

  • Computationally expensive, especially for large images and many intensity levels.
  • Sensitive to grayscale distortion (since they directly depend on pixel values).
  • More suitable for fine-grained textures than for textures with large-scale structures.