将鼠标转换为屏幕位置

Question

我正在使用Open GL制作具有三角投影的游戏。我有以下作为视图投影变换：

float aspect = 1024.f/768.f;
glm::ortho<float>(-aspect*5, aspect*5, -5, 5, 0, 20) * glm::lookAt(glm::vec3(-std::sin(M_PI*36/180.f)*10,sin(M_PI*34/180.f)*10,10), glm::vec3(), glm::vec3(0,1,0));

屏幕分辨率为1024x768。我想将鼠标映射到屏幕上的位置。例如，我游戏中的牌是1x1。当我将鼠标悬停在第一个图块（源自屏幕中心）时，我希望鼠标的位置在（0,0）和（1,1）之间。我不知道该怎么做。到目前为止我得到的是通过这样做转换为屏幕的线性视图（即正交投影）：

glm::vec4 mousePos(x/768*10-apsect*5, 0, y/768*10-5, 0);

但是我不知道从哪里去。

Answer 1

您有屏幕空间坐标，并希望将其转换为模型空间。

您必须应用反转以从屏幕空间返回模型空间。

GLM有一个很好的函数叫做unProject，你可以看到示例代码1。

您遇到的问题是您将屏幕空间中的鼠标坐标视为一个点。鼠标的位置应该被看作是一条正朝着光标前进的光线。如果它只是一个点，你实际上只有很少的信息。

因此，如果您的鼠标有2d坐标作为屏幕空间，则需要将该2d点转换为具有不同z值的两个不同的3d坐标。）

然后，您取消投影这些3d点以获得两个模型空间点。这两个点就是代表鼠标的光线。参见示例代码2。

然后，通过简单地执行光线与平面平面的交叉，您可以从三维光线返回到平铺坐标。参见示例代码3。

示例代码1

我在此示例中添加了完整代码，以便您可以使用值来查看会发生什么。

#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>
#include <iostream>
#include <cmath>

using namespace glm;
using namespace std;

int main()
{
  //Our Matrices
  float aspect = 1024.f/ 768;
  vec4 viewPort = vec4(0.f,0.f,1024.f,768.f);
  mat4 projectionMatrix =  ortho<float>(0, 1024, 0, 768, 0, 20);
  mat4 modelWorldMatrix = lookAt(vec3(-sin(M_PI*36/180.f)*10,sin(M_PI*34/180.f)*10,10), vec3(), vec3(0,1,0)); 

  //Our position
  vec3 pos = vec3(1.0f,2.0f,3.0f);

  //Tranform it into screen space
  vec3 transformed = project(pos, modelWorldMatrix, projectionMatrix, viewPort);
  cout << transformed.x << " " << transformed.y << " " << transformed.z << endl;

  //Transform it back
  vec3 untransformed = unProject(transformed, modelWorldMatrix, projectionMatrix, viewPort);
  cout << untransformed.x << " " << untransformed.y << " " << untransformed.z << endl;

  //You'll see how the glm's unproject works

  return 0;
}

示例代码2

  //You have your screen space coordinates as x and y
  vec3 screenPoint1 = vec3(x, y, 0.f);
  vec3 screenPoint2 = vec3(x, y, 20.f);

  //Unproject both these points
  vec3 modelPoint1 = unProject(screenPoint1, modelWorldMatrix, projectionMatrix, viewPort);
  vec3 modelPoint2 = unProject(screenPoint2, modelWorldMatrix, projectionMatrix, viewPort);

  //This is your ray with the two points modelPoint1, modelPoint2

示例代码3

  //normalOfPlane is the normal of the plane. If it's a xy plane then the normal is vec3(0,0,1)
  //P0 is a point on the plane

  //L is the direction of your ray
  //L0 is a point on the ray
  vec3 L = modelPoint1 - modelPoint2;
  vec3 L0 = modelPoint1;

  //Solve for d where dot((d * L + L0 - P0), n) = 0
  float d = dot(P0 - L0,normalOfPlane) / dot(L, normalOfPlane);

  //Use d to get back to point on plane
  vec3 pointOnPlane = d * L + L0; 

  //Sound of trumpets in the background
  cout << pointOnPlane.x << " " << pointOnPlane.y << endl;

Answer 2

这与我正在研究的iPad项目的等距界面非常相似。这与您的项目屏幕分辨率相同。几天前，当我第一次阅读你的问题时，我正在构建其他项目代码。但是需要构建用于拾取对象的接口代码。因此尝试开发代码是有意义的。我使用空白彩色棋盘设置测试以适合您的问题。

这是一个快速视频演示。请原谅它的丑陋外观。另请注意视频，光标后面的值是整数，但代码会产生浮点值。我创建了另一个函数来完成我自己使用的额外工作。如果那是你想要的东西，我会在答案中加入。

http://youtu.be/JyddfSf57ic

这段代码故意冗长，因为我在数学上生气了。所以我在过去几天花了很多时间重新学习点和交叉产品，还阅读重心代码但决定反对它。

代码前的最后一件事：问题中的投影矩阵存在问题。您将相机变换包含在投影矩阵中。技术上是允许的，但网上有很多资源表明这是不好的做法。

希望这有帮助！

void calculateTouchPointOnGrid(CGPoint screenTouch) {
    float backingWidth = 1024;
    float backingHeight = 768;
    float aspect = backingWidth / backingHeight;
    float zNear = 0;
    float zFar = 20;
    glm::detail::tvec3<float> unprojectedNearZ;
    glm::detail::tvec3<float> unprojectedFarZ;

    /*
     Window coordinates, including zNear 
     This code uses zNear and zFar as two arbitrary values of 
             magnitude along the direction (vector) of the camera's 
             view (affected by projection and model-view matrices) that enable
     determining the line/ray, originating from screenTouch (or 
             mouse click, etc) that later intersects the plane.
     */

    glm::vec3 win = glm::vec3( screenTouch.x, backingHeight - screenTouch.y, zNear);


    // Model & View matrix

    glm::detail::tmat4x4<float> modelTransformMatrix = glm::detail::tmat4x4<float>(1);
    modelTransformMatrix = glm::translate(modelTransformMatrix, glm::detail::tvec3<float>(0,0,-1));
    modelTransformMatrix = glm::rotate(modelTransformMatrix, 0.f, glm::detail::tvec3<float>(0,1,0));
    modelTransformMatrix = glm::scale(modelTransformMatrix, glm::detail::tvec3<float>(1,1,1));

    glm::detail::tmat4x4<float> modelViewMatrix = lookAtMat * modelTransformMatrix;


    /*
     Projection
    */
    const glm::detail::tmat4x4<float> projectionMatrix =
        glm::ortho<float>(-aspect*5, aspect*5, -5, 5, 0, 20);


    /* 
     Viewport
    */      
    const glm::vec4 viewport = glm::vec4(0, 0, backingWidth, backingHeight);

    /*
     Calculate two points on a line/ray based on the window coordinate (including arbitrary Z value), plus modelViewMatrix, projection matrix and viewport
    */
    unprojectedNearZ = glm::unProject(win,
                                 modelViewMatrix,
                                 projectionMatrix,
                                 viewport);
    win[2] = zFar;
    unprojectedFarZ = glm::unProject(win,
                                 modelViewMatrix,
                                 projectionMatrix,
                                 viewport);


            /*
     Define the start of the ray
     */

    glm::vec3 rayStart( unprojectedNearZ[0], unprojectedNearZ[1], unprojectedNearZ[2] );

    /*
     Determine the vector traveling parallel to the camera from the two
      unprojected points
    */

    float lookatVectX = unprojectedFarZ[0] - unprojectedNearZ[0];
    float lookatVectY = unprojectedFarZ[1] - unprojectedNearZ[1];
    float lookatVectZ = unprojectedFarZ[2] - unprojectedNearZ[2];

    glm::vec3 dir( lookatVectX, lookatVectY, lookatVectZ );



    /*
     Define three points on the plane that will define a triangle.
     Winding order does not matter.
     */

    glm::vec3 p0(0,0,0);
    glm::vec3 p1(1,0,0);
    glm::vec3 p2(0,0,1);

    /*
     And finally the destination of the calculations
     */

    glm::vec3 linePlaneIntersect;


    if (cartesianLineIntersectPlane(rayStart, dir, p0, p1, p2, linePlaneIntersect)) {
        // do work here using the linePlaneIntersect values
    }
}


bool cartesianLineIntersectPlane(glm::vec3 rayStart, glm::vec3 dir,
                                 glm::vec3 p0, glm::vec3 p1, glm::vec3 p2, glm::vec3 &linePlaneIntersect) {

    /*
     Create edge vectors to form the plane normal
     */
    glm::vec3 edge1 = p1 - p0;
    glm::vec3 edge2 = p2 - p0;

    /*
     Check if the ray direction is parallel to plane, before continuing
     */

    glm::vec3 perpendicularvector = glm::cross(dir, edge2);


    /*
     dot product of edge1 on perpendicular vector
     if orthogonal (approximately 0) then ray is parallel to plane, has 0 or infinite contact points
     */

    float det = glm::dot(edge1, perpendicularvector);
    float Epsilon = std::numeric_limits<float>::epsilon();
    if (det > -Epsilon && det < Epsilon)
        // Parallel, return false
        return false;


    /*
     Calculate the normalized/unit normal vector
     */

    glm::vec3 planeNormal = glm::normalize( glm::cross(edge1, edge2) );


    /*
     Calculate d, the dot product of the normal and any point on the plane.
     D is the magnitude of the plane's normal, to the origin.
     */

    float d = planeNormal[0] * p0[0] + planeNormal[1] * p0[1] + planeNormal[2] * p0[2];


    /*
     Take the x,y,z equations, ie:
     finalP.xyz = raystart.xyz + dir.xyz * t

     substitute them into the scalar equation for plane, ie:
     ax + by + cz = d

     and solve for t.  (This gets a bit wordy) t is the 
    multiplier on the direction vector, originating at raystart, 
    where it intersects the plane. 


     eg:

     ax + by + cz = d
     a(raystart.x + dir.x * t) + b(raystart.y + dir.y * t) + c(raystart.z + dir.z * t) = d
     a(raystart.x) + a(dir.x*t) + b(raystart.y) + b(dir.y*t) + c(raystart.z) + c(dir.z*t) = d
     a(raystart.x) + a(dir.x*t) + b(raystart.y) + b(dir.y*t) + c(raystart.z) + c(dir.z*t) - d = 0
     a(raystart.x) + b(raystart.y) + c(raystart.z) - d = - a(dir.x*t) - b(dir.y*t) - c(dir.z*t)
     (a(raystart.x) + b(raystart.y) + c(raystart.z) - d) / ( a(-dir.x) + b(-dir.y) + c(-dir.z) = t

     */

    float leftsideScalars = (planeNormal[0] * rayStart[0] +
                             planeNormal[1] * rayStart[1] +
                             planeNormal[2] * rayStart[2] - d);
    float directionDotProduct = (planeNormal[0] * (-dir[0]) +
                                 planeNormal[1] * (-dir[1]) +
                                 planeNormal[2] * (-dir[2]) );


    /*
     Final calculation of t, hurrah!
     */

    float t = leftsideScalars / directionDotProduct;


    /*
     This is the particular value of t for that line that lies
     in the plane. Then you can solve for x, y, and z by going
     back up to the line equations and substituting t back in.
     */

    linePlaneIntersect = glm::vec3(rayStart[0] + dir[0] * t,
                                   rayStart[1] + dir[1] * t,
                                   rayStart[2] + dir[2] * t
                                   );
    return true;
}