## Where are the inverse equations hiding?

A common problem in computer vision is modelling lens distortion.  Lens distortion distort the shape of objects and introduce large errors in structure from motion applications.  Techniques and tools for calibrating cameras and removing lens distortion are widely available.  While books and papers readily provide the forward distortion equations (given an ideal undistorted coordinate, compute the distorted coordinate) inverse equations are much harder to come by.

Turns out there is no analytic equation for inverse radial distortion.  Which might explain the mysterious absence of inverse equations, but still it would be nice if this issue was discussed in common references. First a brief summary of background equations is given, followed by the solution to the inverse distortion problem.

Inverse Distortion Problem: Given a distorted image coordinate and distortion parameters, determined the coordinate of the ideal undistorted coordinate.

## Background

Unlike the idealized pin-hole camera model, real world cameras have lens distortions.  Radial distortion is a common model for lens distortion and is summarized by the following equations: $\hat{x} = x + x[k_1 r^2 + k_2 r^4]$ $\hat{u} = u + (u-u_0)[k_1 r^2 + k_2 r^4]$ $r^2=u_x^2 + u_y^2$

where $\hat{u}=[\hat{u}_x,\hat{u}_y]$ and $u$ are the observed distorted and ideal undistorted pixel coordinates, $\hat{x}$ and $x$ are the observed and ideal undistorted normalized pixel coordinates, and $k_1, k_2$ are radial distortion coefficients.  The relationship between normalized pixel and unnormalized pixel coordinates is determined by the camera calibration matrix, $u=C x$, where $C$ is the 3×3 calibration matrix.

## Solution

While there is no analytic solution there is an iterative solution.  The iterative solution works by first estimating the radial distortion’s magnitude at the distorted point and then refining the estimate until it converges.

1. $x=C^{-1}\hat{u}$
2. do until converge:
1. $r=||x||^2$
2. $m=k_1 r^2 + k_2 r^4$
3. $x=\frac{C^{-1}\hat{u}}{1+m}$
3. $u = \frac{x+x_0 m}{1+m}$

Where $x_0$ is the principle point/image center, and $||x||$ is the Euclidean norm. Only a few iterations are required before convergence.  For added speed when applying to an entire image the results can be cached.  A similar solution can be found by adding in terms for tangential distortion.

1. Joe Jacobs says:

Any chance you could point me towards a few (academic) reference articles?

2. peter says:

Just about any 3D computer vision book (e.g. ) should provide the equations for radial distortion. As mentioned above they don’t seem to discuss the inverse problem, that I am aware of at least. The popular matlab camera calibration toolbox found http://www.vision.caltech.edu/bouguetj/calib_doc/ uses a similar formula when it computes the inverse radial distortion.

 Richard Hartley and Andrew Zisserman, “Multiple View Geometry in Computer Vision” 2003

• Joe Jacobs says:

Brilliant…thanks!

3. Libor says:

Unfortunately there can be multiple solutions to the inverse problem.
The iteration will then stick to nearest minimum, which may not be the original (undistorted) value.

I know this does not pose a problem with simple applications, but estimating lens distortion from points (e.g. camera calibration, image registration, bundle adjustment), with some noise added may cause such issue.

I tried to solve this by splitting the distortion function in in part that is monotonic to a certain point and then is replaced by linear function. This ensures the function is differentiable and invertible everywhere.

However, the solution is quite messy and the “breaking point” itself depends on kappa parameters. This causes my unit tests failing because partial derivatives w.r.t. kappa are badly defined.

Any ideas?

• peter says:

I’ve never bothered to go into that much detail into trying to find the global optimal solution. Basically during calibration you use the forward equations (just double checked that statement by looking at code) and RANSAC or some other robust technique removes outliers (which would include poor convergence) when reconstructing a scene.

How much of an error are you seeing caused by poor convergence? Are these highly distorted lenses?

• Josh says:

Thank you peter for this great post! I did some additional analysis on the problem and posted it on Math SE (https://math.stackexchange.com/a/3638968/209710). It discusses the problem of multiple solutions and provides a strict bound for when multiple solutions will exist.

4. Ash says:

Hi Peter,

Wonderful explanation. Thank you!

I am a little confused about ‘adding’ the tangential distortion components. The tangential component has the corrected component in it.Based on the expression here – https://en.wikipedia.org/wiki/Distortion_(optics)#Software_correction

Could you please refer me to an article or paper that explains the addition of tangential component terms in the iterative method?

Thanks!
Ash