Inverse Radial Distortion Formula

by peter

Where are the inverse equations hiding?

A common problem in computer vision is modelling lens distortion.  Lens distortion distort the shape of objects and introduce large errors in structure from motion applications.  Techniques and tools for calibrating cameras and removing lens distortion are widely available.  While books and papers readily provide the forward distortion equations (given an ideal undistorted coordinate, compute the distorted coordinate) inverse equations are much harder to come by.

Turns out there is no analytic equation for inverse radial distortion.  Which might explain the mysterious absence of inverse equations, but still it would be nice if this issue was discussed in common references. First a brief summary of background equations is given, followed by the solution to the inverse distortion problem.

Inverse Distortion Problem: Given a distorted image coordinate and distortion parameters, determined the coordinate of the ideal undistorted coordinate.

Background

Unlike the idealized pin-hole camera model, real world cameras have lens distortions.  Radial distortion is a common model for lens distortion and is summarized by the following equations:

\hat{x} = x + x[k_1 r^2 + k_2 r^4]
\hat{u} = u + (u-u_0)[k_1 r^2 + k_2 r^4]
r^2=u_x^2 + u_y^2

where \hat{u}=[\hat{u}_x,\hat{u}_y] and u are the observed distorted and ideal undistorted pixel coordinates, \hat{x} and x are the observed and ideal undistorted normalized pixel coordinates, and k_1, k_2 are radial distortion coefficients.  The relationship between normalized pixel and unnormalized pixel coordinates is determined by the camera calibration matrix, u=C x, where C is the 3×3 calibration matrix.

Solution

While there is no analytic solution there is an iterative solution.  The iterative solution works by first estimating the radial distortion’s magnitude at the distorted point and then refining the estimate until it converges.

  1. x=C^{-1}\hat{u}
  2. do until converge:
    1. r=||x||^2
    2. m=k_1 r^2 + k_2 r^4
    3. x=\frac{C^{-1}\hat{u}}{1+m}
  3. u = \frac{x+x_0 m}{1+m}

Where x_0 is the principle point/image center, and ||x|| is the Euclidean norm. Only a few iterations are required before convergence.  For added speed when applying to an entire image the results can be cached.  A similar solution can be found by adding in terms for tangential distortion.

 

7 comments on “Inverse Radial Distortion Formula”

  1. Joe Jacobs says:
    March 13, 2012 at 1:23 pm

    Any chance you could point me towards a few (academic) reference articles?

    Reply
  2. peter says:
    March 14, 2012 at 10:39 am

    Just about any 3D computer vision book (e.g. [1]) should provide the equations for radial distortion. As mentioned above they don’t seem to discuss the inverse problem, that I am aware of at least. The popular matlab camera calibration toolbox found http://www.vision.caltech.edu/bouguetj/calib_doc/ uses a similar formula when it computes the inverse radial distortion.

    [1] Richard Hartley and Andrew Zisserman, “Multiple View Geometry in Computer Vision” 2003

    Reply
  3. Libor says:
    September 14, 2016 at 4:08 pm

    Unfortunately there can be multiple solutions to the inverse problem.
    The iteration will then stick to nearest minimum, which may not be the original (undistorted) value.

    I know this does not pose a problem with simple applications, but estimating lens distortion from points (e.g. camera calibration, image registration, bundle adjustment), with some noise added may cause such issue.

    I tried to solve this by splitting the distortion function in in part that is monotonic to a certain point and then is replaced by linear function. This ensures the function is differentiable and invertible everywhere.

    However, the solution is quite messy and the “breaking point” itself depends on kappa parameters. This causes my unit tests failing because partial derivatives w.r.t. kappa are badly defined.

    Any ideas?

    Reply
    • peter says:
      December 5, 2016 at 9:48 pm

      I’ve never bothered to go into that much detail into trying to find the global optimal solution. Basically during calibration you use the forward equations (just double checked that statement by looking at code) and RANSAC or some other robust technique removes outliers (which would include poor convergence) when reconstructing a scene.

      How much of an error are you seeing caused by poor convergence? Are these highly distorted lenses?

      Reply
    • Josh says:
      April 22, 2020 at 8:43 pm

      Thank you peter for this great post! I did some additional analysis on the problem and posted it on Math SE (https://math.stackexchange.com/a/3638968/209710). It discusses the problem of multiple solutions and provides a strict bound for when multiple solutions will exist.

      Reply
  4. Ash says:
    April 7, 2017 at 7:40 am

    Hi Peter,

    Wonderful explanation. Thank you!

    I am a little confused about ‘adding’ the tangential distortion components. The tangential component has the corrected component in it.Based on the expression here – https://en.wikipedia.org/wiki/Distortion_(optics)#Software_correction

    Could you please refer me to an article or paper that explains the addition of tangential component terms in the iterative method?

    Thanks!
    Ash

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *