Image processing 透视变形矩形的比例_Image Processing_Geometry_Computer Vision_Reverseprojection_Projective Geometry

Image processing 透视变形矩形的比例

image-processing geometry computer-vision

Image processing 透视变形矩形的比例,image-processing,geometry,computer-vision,reverseprojection,projective-geometry,Image Processing,Geometry,Computer Vision,Reverseprojection,Projective Geometry,给定一个被透视扭曲的矩形的2d图片：我知道这个形状原来是一个长方形，但我不知道它原来的大小如果我知道这张图片中角点的像素坐标，我如何计算原始比例，即矩形的商（宽度/高度）（背景：目标是自动消除矩形文档的照片失真，边缘检测可能使用hough变换完成）更新：关于是否有可能根据给定的信息确定宽高比，已经进行了一些讨论。我天真的想法是，这一定是可能的，因为我想不出任何方法，例如，将一个1:4的矩形投影到上面描述的四边形上。这个比例显然接近1:1，因此应该有一种方法从数学上确定它。然而，我没有

给定一个被透视扭曲的矩形的2d图片：

我知道这个形状原来是一个长方形，但我不知道它原来的大小

如果我知道这张图片中角点的像素坐标，我如何计算原始比例，即矩形的商（宽度/高度）

（背景：目标是自动消除矩形文档的照片失真，边缘检测可能使用hough变换完成）

更新：关于是否有可能根据给定的信息确定宽高比，已经进行了一些讨论。我天真的想法是，这一定是可能的，因为我想不出任何方法，例如，将一个1:4的矩形投影到上面描述的四边形上。这个比例显然接近1:1，因此应该有一种方法从数学上确定它。然而，我没有超出直觉猜测的证据

我还没有完全理解下面提出的论点，但我认为一定有某种隐含的假设，即我们在这里遗漏了，这是不同的解释

然而，经过几个小时的搜索，我终于找到了一些与这个问题相关的论文。我正在努力理解其中使用的数学，但迄今为止没有成功。特别是第一篇论文似乎完全讨论了我想做什么，不幸的是没有代码示例和非常密集的数学

张正友，何李伟，“白板扫描与图像增强” p、十一,
由于透视失真，矩形的图像看起来是一个四边形。然而，由于我们知道它在空间中是一个矩形，我们可以估计相机的焦距和矩形的纵横比
ROBERT M.HARALICK“从矩形的透视投影确定相机参数”
“我们将演示如何使用三维空间中未知大小和位置的矩形的二维透视投影来确定相对于矩形平面的相机视角参数。”

编辑：

R=[（0,0）、（1,0）、（1,R）、（0,R）]

T=[（tx0，ty0），（tx1，ty1），（tx2，ty2），（tx3，ty3）]

（Rxi，Ryi，1）*M=wi（txi，tyi，1）

M=[[m00，m01，m02]，[m10，m11，m12]，[m20，m21，m22]

R_0

m02-tx0*w0=m12-ty0*w0=m22-w0=0

R_1

m00-tx1*w1=m10-ty1*w1=m20+m22-w1=0

R_2

m00+R*m01-tx2*w2=m10+R*m11-ty2*w2=m20+R*m21+m22-w2=0

R_3

m00+R*m01-tx3*w3=m10+R*m11-ty3*w3=m20+R*m21+m22-w3=0

wi

txi

tyi

mi0

更新
在阅读了您的更新，并查看了第一个参考（白板扫描和图像增强）之后，我看到了缺失的地方
问题的输入数据是投影图像的中心O的四倍（a、B、C、D），和。在本文中，它对应于假设u0=v0=0。加上这一点，问题将受到足够的约束，以获得矩形的纵横比
然后将问题重述如下：给定Z=0平面上的四重体（a，B，C，D），求
// in case it matters: licensed under GPLv2 or later // legend: // sqr(x) = x*x // sqrt(x) = square root of x // let m1x,m1y ... m4x,m4y be the (x,y) pixel coordinates // of the 4 corners of the detected quadrangle // i.e. (m1x, m1y) are the cordinates of the first corner, // (m2x, m2y) of the second corner and so on. // let u0, v0 be the pixel coordinates of the principal point of the image // for a normal camera this will be the center of the image, // i.e. u0=IMAGEWIDTH/2; v0 =IMAGEHEIGHT/2 // This assumption does not hold if the image has been cropped asymmetrically // first, transform the image so the principal point is at (0,0) // this makes the following equations much easier m1x = m1x - u0; m1y = m1y - v0; m2x = m2x - u0; m2y = m2y - v0; m3x = m3x - u0; m3y = m3y - v0; m4x = m4x - u0; m4y = m4y - v0; // temporary variables k2, k3 double k2 = ((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x) / ((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) ; double k3 = ((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x) / ((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) ; // f_squared is the focal length of the camera, squared // if k2==1 OR k3==1 then this equation is not solvable // if the focal length is known, then this equation is not needed // in that case assign f_squared= sqr(focal_length) double f_squared = -((k3*m3y - m1y)*(k2*m2y - m1y) + (k3*m3x - m1x)*(k2*m2x - m1x)) / ((k3 - 1)*(k2 - 1)) ; //The width/height ratio of the original rectangle double whRatio = sqrt( (sqr(k2 - 1) + sqr(k2*m2y - m1y)/f_squared + sqr(k2*m2x - m1x)/f_squared) / (sqr(k3 - 1) + sqr(k3*m3y - m1y)/f_squared + sqr(k3*m3x - m1x)/f_squared) ) ; // if k2==1 AND k3==1, then the focal length equation is not solvable // but the focal length is not needed to calculate the ratio. // I am still trying to figure out under which circumstances k2 and k3 become 1 // but it seems to be when the rectangle is not distorted by perspective, // i.e. viewed straight on. Then the equation is obvious: if (k2==1 && k3==1) whRatio = sqrt( (sqr(m2y-m1y) + sqr(m2x-m1x)) / (sqr(m3y-m1y) + sqr(m3x-m1x)) // After testing, I found that the above equations // actually give the height/width ratio of the rectangle, // not the width/height ratio. // If someone can find the error that caused this, // I would be most grateful. // until then: whRatio = 1/whRatio;

# CALCULATING THE ASPECT RATIO OF A RECTANGLE DISTORTED BY PERSPECTIVE # # BIBLIOGRAPHY: # [zhang-single]: "Single-View Geometry of A Rectangle # With Application to Whiteboard Image Rectification" # by Zhenggyou Zhang # http://research.microsoft.com/users/zhang/Papers/WhiteboardRectification.pdf # pixel coordinates of the 4 corners of the quadrangle (m1, m2, m3, m4) # see [zhang-single] figure 1 m1x = var('m1x') m1y = var('m1y') m2x = var('m2x') m2y = var('m2y') m3x = var('m3x') m3y = var('m3y') m4x = var('m4x') m4y = var('m4y') # pixel coordinates of the principal point of the image # for a normal camera this will be the center of the image, # i.e. u0=IMAGEWIDTH/2; v0 =IMAGEHEIGHT/2 # This assumption does not hold if the image has been cropped asymmetrically u0 = var('u0') v0 = var('v0') # pixel aspect ratio; for a normal camera pixels are square, so s=1 s = var('s') # homogenous coordinates of the quadrangle m1 = vector ([m1x,m1y,1]) m2 = vector ([m2x,m2y,1]) m3 = vector ([m3x,m3y,1]) m4 = vector ([m4x,m4y,1]) # the following equations are later used in calculating the the focal length # and the rectangle's aspect ratio. # temporary variables: k2, k3, n2, n3 # see [zhang-single] Equation 11, 12 k2_ = m1.cross_product(m4).dot_product(m3) / m2.cross_product(m4).dot_product(m3) k3_ = m1.cross_product(m4).dot_product(m2) / m3.cross_product(m4).dot_product(m2) k2 = var('k2') k3 = var('k3') # see [zhang-single] Equation 14,16 n2 = k2 * m2 - m1 n3 = k3 * m3 - m1 # the focal length of the camera. f = var('f') # see [zhang-single] Equation 21 f_ = sqrt( -1 / ( n2[2]*n3[2]*s^2 ) * ( ( n2[0]*n3[0] - (n2[0]*n3[2]+n2[2]*n3[0])*u0 + n2[2]*n3[2]*u0^2 )*s^2 + ( n2[1]*n3[1] - (n2[1]*n3[2]+n2[2]*n3[1])*v0 + n2[2]*n3[2]*v0^2 ) ) ) # standard pinhole camera matrix # see [zhang-single] Equation 1 A = matrix([[f,0,u0],[0,s*f,v0],[0,0,1]]) #the width/height ratio of the original rectangle # see [zhang-single] Equation 20 whRatio = sqrt ( (n2*A.transpose()^(-1) * A^(-1)*n2.transpose()) / (n3*A.transpose()^(-1) * A^(-1)*n3.transpose()) )

print "simplified equations, assuming u0=0, v0=0, s=1" print "k2 := ", k2_ print "k3 := ", k3_ print "f := ", f_(u0=0,v0=0,s=1) print "whRatio := ", whRatio(u0=0,v0=0,s=1) simplified equations, assuming u0=0, v0=0, s=1 k2 := ((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) k3 := ((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) f := sqrt(-((k3*m3y - m1y)*(k2*m2y - m1y) + (k3*m3x - m1x)*(k2*m2x - m1x))/((k3 - 1)*(k2 - 1))) whRatio := sqrt(((k2 - 1)^2 + (k2*m2y - m1y)^2/f^2 + (k2*m2x - m1x)^2/f^2)/((k3 - 1)^2 + (k3*m3y - m1y)^2/f^2 + (k3*m3x - m1x)^2/f^2)) print "Everything in one equation:" print "whRatio := ", whRatio(f=f_)(k2=k2_,k3=k3_)(u0=0,v0=0,s=1) Everything in one equation: whRatio := sqrt(((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2y/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1y)^2/((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3y/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1y)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2y/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1y) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3x/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1x)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2x/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1x)) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2x/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1x)^2/((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3y/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1y)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2y/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1y) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3x/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1x)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2x/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1x)) - (((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - 1)^2)/((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - 1)*(((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3y/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1y)^2/((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3y/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1y)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2y/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1y) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3x/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1x)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2x/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1x)) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - 1)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - 1)*(((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3x/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1x)^2/((((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3y/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1y)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2y/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1y) + (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)*m3x/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - m1x)*(((m1y - m4y)*m3x - (m1x - m4x)*m3y + m1x*m4y - m1y*m4x)*m2x/((m2y - m4y)*m3x - (m2x - m4x)*m3y + m2x*m4y - m2y*m4x) - m1x)) - (((m1y - m4y)*m2x - (m1x - m4x)*m2y + m1x*m4y - m1y*m4x)/((m3y - m4y)*m2x - (m3x - m4x)*m2y + m3x*m4y - m3y*m4x) - 1)^2))

# some testing: # - choose a random rectangle, # - project it onto a random plane, # - insert the corners in the above equations, # - check if the aspect ratio is correct. from sage.plot.plot3d.transform import rotate_arbitrary #redundandly random rotation matrix rand_rotMatrix = \ rotate_arbitrary((uniform(-5,5),uniform(-5,5),uniform(-5,5)),uniform(-5,5)) *\ rotate_arbitrary((uniform(-5,5),uniform(-5,5),uniform(-5,5)),uniform(-5,5)) *\ rotate_arbitrary((uniform(-5,5),uniform(-5,5),uniform(-5,5)),uniform(-5,5)) #random translation vector rand_transVector = vector((uniform(-10,10),uniform(-10,10),uniform(-10,10))).transpose() #random rectangle parameters rand_width =uniform(0.1,10) rand_height=uniform(0.1,10) rand_left =uniform(-10,10) rand_top =uniform(-10,10) #random focal length and principal point rand_f = uniform(0.1,100) rand_u0 = uniform(-100,100) rand_v0 = uniform(-100,100) # homogenous standard pinhole projection, see [zhang-single] Equation 1 hom_projection = A * rand_rotMatrix.augment(rand_transVector) # construct a random rectangle in the plane z=0, then project it randomly rand_m1hom = hom_projection*vector((rand_left ,rand_top ,0,1)).transpose() rand_m2hom = hom_projection*vector((rand_left ,rand_top+rand_height,0,1)).transpose() rand_m3hom = hom_projection*vector((rand_left+rand_width,rand_top ,0,1)).transpose() rand_m4hom = hom_projection*vector((rand_left+rand_width,rand_top+rand_height,0,1)).transpose() #change type from 1x3 matrix to vector rand_m1hom = rand_m1hom.column(0) rand_m2hom = rand_m2hom.column(0) rand_m3hom = rand_m3hom.column(0) rand_m4hom = rand_m4hom.column(0) #normalize rand_m1hom = rand_m1hom/rand_m1hom[2] rand_m2hom = rand_m2hom/rand_m2hom[2] rand_m3hom = rand_m3hom/rand_m3hom[2] rand_m4hom = rand_m4hom/rand_m4hom[2] #substitute random values for f, u0, v0 rand_m1hom = rand_m1hom(f=rand_f,s=1,u0=rand_u0,v0=rand_v0) rand_m2hom = rand_m2hom(f=rand_f,s=1,u0=rand_u0,v0=rand_v0) rand_m3hom = rand_m3hom(f=rand_f,s=1,u0=rand_u0,v0=rand_v0) rand_m4hom = rand_m4hom(f=rand_f,s=1,u0=rand_u0,v0=rand_v0) # printing the randomly choosen values print "ground truth: f=", rand_f, "; ratio=", rand_width/rand_height # substitute all the variables in the equations: print "calculated: f= ",\ f_(k2=k2_,k3=k3_)(s=1,u0=rand_u0,v0=rand_v0)( m1x=rand_m1hom[0],m1y=rand_m1hom[1], m2x=rand_m2hom[0],m2y=rand_m2hom[1], m3x=rand_m3hom[0],m3y=rand_m3hom[1], m4x=rand_m4hom[0],m4y=rand_m4hom[1], ),"; 1/ratio=", \ 1/whRatio(f=f_)(k2=k2_,k3=k3_)(s=1,u0=rand_u0,v0=rand_v0)( m1x=rand_m1hom[0],m1y=rand_m1hom[1], m2x=rand_m2hom[0],m2y=rand_m2hom[1], m3x=rand_m3hom[0],m3y=rand_m3hom[1], m4x=rand_m4hom[0],m4y=rand_m4hom[1], ) print "k2 = ", k2_( m1x=rand_m1hom[0],m1y=rand_m1hom[1], m2x=rand_m2hom[0],m2y=rand_m2hom[1], m3x=rand_m3hom[0],m3y=rand_m3hom[1], m4x=rand_m4hom[0],m4y=rand_m4hom[1], ), "; k3 = ", k3_( m1x=rand_m1hom[0],m1y=rand_m1hom[1], m2x=rand_m2hom[0],m2y=rand_m2hom[1], m3x=rand_m3hom[0],m3y=rand_m3hom[1], m4x=rand_m4hom[0],m4y=rand_m4hom[1], ) # ATTENTION: testing revealed, that the whRatio # is actually the height/width ratio, # not the width/height ratio # This contradicts [zhang-single] # if anyone can find the error that caused this, I'd be grateful ground truth: f= 72.1045134124554 ; ratio= 3.46538779959142 calculated: f= 72.1045134125 ; 1/ratio= 3.46538779959 k2 = 0.99114614987 ; k3 = 1.57376280159

whRatio = sqrt ( (n2*A.transpose()^(-1) * A^(-1)*n2.transpose()) / (n3*A.transpose()^(-1) * A^(-1)*n3.transpose()) )

whRatio = sqrt ( (n2.transpose()*A.transpose()^(-1) * A^(-1)*n2) / (n3.transpose()*A.transpose()^(-1) * A^(-1)*n3) )