temo

Okay, let’s break down these two steps in detail. The goal here is to transform the non-linear projection equation into a system of linear equations that we can solve to find the unknown projection matrix $\mathbf{M}$.

Step 1: Formulate Linear Equations (for a single point i)

1. Starting Point: The fundamental relationship between a 3D world point ${\mathbf{P}}_{w,i}$ and its 2D image projection ${\mathbf{p}}_i$ is given by the perspective projection equation: $\lambda {\mathbf{p}}_i=\mathbf{M}{\mathbf{P}}_{w,i}$

   • ${\mathbf{P}}_{w,i}=[X_i,Y_i,Z_i,1{]}^T$: Homogeneous coordinates of the known 3D point.
   • ${\mathbf{p}}_i=[u_i,v_i,1{]}^T$: Homogeneous coordinates of the measured 2D image point. $u_i$ and $v_i$ are the pixel coordinates we observe.
   • $\mathbf{M}$: The $3\times 4$ projection matrix we want to find. Its 12 elements are the unknowns.
   • $\lambda$: An unknown, non-zero scalar representing the projective depth. It varies for each point $i$.

2. Expand the Matrix Equation: Let’s represent the rows of the unknown matrix $\mathbf{M}$ as ${\mathbf{m}}_1^T$, ${\mathbf{m}}_2^T$, and ${\mathbf{m}}_3^T$. Each ${\mathbf{m}}_j^T$ is a $1\times 4$ row vector (or ${\mathbf{m}}_j$ is a $4\times 1$ column vector).

   $$=\left[\begin{array}{c}{\mathbf{m}}_1^T\\ {\mathbf{m}}_2^T\\ {\mathbf{m}}_3^T\end{array}\right]\left[\begin{array}{c}{X}_i\\ Y_i\\ Z_i\\ 1\end{array}\right]=\left[\begin{array}{c}{\mathbf{m}}_1^T{\mathbf{P}}_{w,i}\\ {\mathbf{m}}_2^T{\mathbf{P}}_{w,i}\\ {\mathbf{m}}_3^T{\mathbf{P}}_{w,i}\end{array}\right]$$

   The terms ${\mathbf{m}}_j^T{\mathbf{P}}_{w,i}$ are just dot products. For example, ${\mathbf{m}}_1^T{\mathbf{P}}_{w,i}=m_{11}{X}_i+m_{12}{Y}_i+m_{13}{Z}_i+m_{14}$.

3. Write out Scalar Equations: This matrix equation gives us three separate scalar equations:

   • (1) $\lambda u_i={\mathbf{m}}_1^T{\mathbf{P}}_{w,i}$
   • (2) $\lambda v_i={\mathbf{m}}_2^T{\mathbf{P}}_{w,i}$
   • (3) $\lambda ={\mathbf{m}}_3^T{\mathbf{P}}_{w,i}$

4. Eliminate the Unknown Scale $\lambda$: Our goal is to find the elements of $\mathbf{M}$. The scale factor $\lambda$ is different for every point and is also unknown. We can eliminate it using substitution. Substitute equation (3) into equations (1) and (2):

   • Substitute into (1): $({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})u_i={\mathbf{m}}_1^T{\mathbf{P}}_{w,i}$
   • Substitute into (2): $({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})v_i={\mathbf{m}}_2^T{\mathbf{P}}_{w,i}$

5. Rearrange into Linear Homogeneous Form: Move all terms to one side to get equations equal to zero. This form ($A\mathbf{x}=0$) is suitable for solving with methods like SVD.

   • From (1): $u_i({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})-({\mathbf{m}}_1^T{\mathbf{P}}_{w,i})=0$
   • From (2): $v_i({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})-({\mathbf{m}}_2^T{\mathbf{P}}_{w,i})=0$

   Crucially, these two equations are linear in the unknown elements of $\mathbf{M}$ (the $m_{jk}$ values that make up the row vectors ${\mathbf{m}}_1^T,{\mathbf{m}}_2^T,{\mathbf{m}}_3^T$). All the terms involving the known 3D point ${\mathbf{P}}_{w,i}$ and the measured 2D point $(u_i,v_i)$ act as known coefficients multiplying the unknown matrix elements.

Step 2: Set up Homogeneous Linear System (for n points)

1. The Goal: We have two linear equations for each point correspondence. If we have $n$ correspondences (where $n\ge 6$), we have a total of $2n$ linear equations. We want to write this entire system in a single matrix equation form: $\mathcal{P}\mathbf{m}=0$

2. The Unknown Vector $\mathbf{m}$: We need to arrange the 12 unknown elements of $\mathbf{M}$ into a single column vector $\mathbf{m}$. A standard way is to stack the row vectors (transposed):

   $$\mathbf{m}=\left[\begin{array}{c}{\mathbf{m}}_1\\ {\mathbf{m}}_2\\ {\mathbf{m}}_3\end{array}\right]=\left[\begin{array}{c}{m}_{11}\\ m_{12}\\ m_{13}\\ m_{14}\\ m_{21}\\ m_{22}\\ m_{23}\\ m_{24}\\ m_{31}\\ m_{32}\\ m_{33}\\ m_{34}\end{array}\right](\text{a }12\times 1\text{ vector})$$

3. The Coefficient Matrix $\mathcal{P}$: We need to construct the large matrix $\mathcal{P}$ such that when multiplied by $\mathbf{m}$, it reproduces the $2n$ equations we derived. Let’s look at the two equations for a single point $i$ again:

   • Equation 1: $-{\mathbf{m}}_1^T{\mathbf{P}}_{w,i}+0\cdot ({\mathbf{m}}_2^T{\mathbf{P}}_{w,i})+u_i({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})=0$
   • Equation 2: $0\cdot ({\mathbf{m}}_1^T{\mathbf{P}}_{w,i})-{\mathbf{m}}_2^T{\mathbf{P}}_{w,i}+v_i({\mathbf{m}}_3^T{\mathbf{P}}_{w,i})=0$

   Now, let’s express the dot products using the vector ${\mathbf{P}}_{w,i}^T=[X_i,Y_i,Z_i,1]$ and the blocks of $\mathbf{m}$:

   • ${\mathbf{m}}_1^T{\mathbf{P}}_{w,i}={\mathbf{P}}_{w,i}^T{\mathbf{m}}_1$
   • ${\mathbf{m}}_2^T{\mathbf{P}}_{w,i}={\mathbf{P}}_{w,i}^T{\mathbf{m}}_2$
   • ${\mathbf{m}}_3^T{\mathbf{P}}_{w,i}={\mathbf{P}}_{w,i}^T{\mathbf{m}}_3$

   Substituting back:

   • Equation 1: $-{\mathbf{P}}_{w,i}^T{\mathbf{m}}_1+0^T{\mathbf{m}}_2+u_i{\mathbf{P}}_{w,i}^T{\mathbf{m}}_3=0$
   • Equation 2: $0^T{\mathbf{m}}_1-{\mathbf{P}}_{w,i}^T{\mathbf{m}}_2+v_i{\mathbf{P}}_{w,i}^T{\mathbf{m}}_3=0$

   These two equations correspond to two rows in the matrix $\mathcal{P}$. The structure provided in the notes matches these equations (possibly with an overall sign flip on one or both, which doesn’t affect the solution of $\mathcal{P}\mathbf{m}=0$):

   • Row for Eq 2: $[0^T-{\mathbf{P}}_{w,i}^T{v}_i{\mathbf{P}}_{w,i}^T]$. This is a $1\times 12$ row vector. When multiplied by $\mathbf{m}=[{\mathbf{m}}_1^T,{\mathbf{m}}_2^T,{\mathbf{m}}_3^T{]}^T$, it gives Eq 2. This corresponds to row1 in the provided structure.
   • Row for Eq 1: $[{\mathbf{P}}_{w,i}^T{0}^T-u_i{\mathbf{P}}_{w,i}^T]$. This is a $1\times 12$ row vector. When multiplied by $\mathbf{m}$, it gives Eq 1 (with a sign flip). This corresponds to row2 in the provided structure.

   (Note: The naming row1/row2 in the notes might seem reversed relative to the $u,v$ equations, but the structure is correct for generating the necessary constraints.)

   The full $\mathcal{P}$ matrix is constructed by stacking these two rows for each point $i$ from $1$ to $n$:

   $$\mathcal{P}=\left[\begin{array}{ccc}{0}^T& -{\mathbf{P}}_{w,1}^T& v_1{\mathbf{P}}_{w,1}^T\\ {\mathbf{P}}_{w,1}^T& 0^T& -u_1{\mathbf{P}}_{w,1}^T\\ 0^T& -{\mathbf{P}}_{w,2}^T& v_2{\mathbf{P}}_{w,2}^T\\ {\mathbf{P}}_{w,2}^T& 0^T& -u_2{\mathbf{P}}_{w,2}^T\\ ⋮& ⋮& ⋮\\ 0^T& -{\mathbf{P}}_{w,n}^T& v_n{\mathbf{P}}_{w,n}^T\\ {\mathbf{P}}_{w,n}^T& 0^T& -u_n{\mathbf{P}}_{w,n}^T\end{array}\right](\text{a }2n\times 12\text{ matrix})$$

   Each block like ${\mathbf{P}}_{w,i}^T$ is $1\times 4$, and $0^T$ is also $1\times 4$.

4. The Result: We now have the system $\mathcal{P}\mathbf{m}=0$. The matrix $\mathcal{P}$ contains only known quantities (from the 3D points and 2D measurements). The vector $\mathbf{m}$ contains the 12 unknowns we want to find. This system can be solved using Singular Value Decomposition (SVD) to find the non-trivial solution for $\mathbf{m}$ (which corresponds to the vector associated with the smallest singular value).