Expanding on @Spektre's excellent answer, this was how I implemented a working solution. I'm working with Unity, so I used Ivan Kutskir's awesome lightweight C# matrix class to handle the matrix maths. There's probably faster/cleaner ways to do this but this was very easy and works correctly.
Obviously you have to ensure that when you discard the Z axis, you don't end up with a degenerate triangle.
// tri is a 3D triangle with points p0, p1 and p2
// point is a 2D point within that triangle, assuming the Z axis is discarded
/*
Equivalent to this part of @Spektre's answer:
| u | | (q1.x - q0.x) , (q2.x - q0.x) , q0.x | | q.x |
| v | = inverse | (q1.y - q0.y) , (q2.y - q0.y) , q0.y | * | q.y |
| 1 | | 0 , 0 , 1 | | 1 |
*/
Matrix m1 = new Matrix(3, 3);
Matrix m2 = new Matrix(3, 1);
m1[0, 0] = tri.p1.x - tri.p0.x;
m1[0, 1] = tri.p2.x - tri.p0.x;
m1[0, 2] = tri.p0.x;
m1[1, 0] = tri.p1.y - tri.p0.y;
m1[1, 1] = tri.p2.y - tri.p0.y;
m1[1, 2] = tri.p0.y;
m1[2, 0] = 0;
m1[2, 1] = 0;
m1[2, 2] = 1;
m2[0, 0] = point.x;
m2[1, 0] = point.y;
m2[2, 0] = 1;
Matrix mResult = m1.Invert() * m2;
float u = (float)mResult[0, 0];
float v = (float)mResult[1, 0];
/*
Equivalent to this part of @Spektre's answer:
p.x = p0.x + (p1.x - p0.x) * u + (p2.x - p0.x) * v
p.y = p0.y + (p1.y - p0.y) * u + (p2.y - p0.y) * v
p.z = p0.z + (p1.z - p0.z) * u + (p2.z - p0.z) * v
*/
float newX = tri.p0.x + (tri.p1.x - tri.p0.x) * u + (tri.p2.x - tri.p0.x) * v;
float newY = tri.p0.y + (tri.p1.y - tri.p0.y) * u + (tri.p2.y - tri.p0.y) * v;
float newZ = tri.p0.z + (tri.p1.z - tri.p0.z) * u + (tri.p2.z - tri.p0.z) * v;
Vector3 newPoint = new Vector3(newX, newY, newZ);
Alternatively, you can achieve the same result without the matrix (though this may be a less robust method, I'm not sure). To calculate the barycentric coordinates I used this implementation, but the accepted answer also works.
// tri is a 3D triangle with points p0, p1 and p2
// point is a 2D point within that triangle, assuming the Z axis is discarded
// Find the barycentric coords for the chosen 2D point...
float u, v, w = 0;
Barycentric2D(point, new Vector2(tri.p0.x, tri.p0.y), new Vector2(tri.p1.x, tri.p1.y), new Vector2(tri.p2.x, tri.p2.y), out u, out v, out w);
// ...and then find what the Z value would be for those barycentric coords in 3D
float newZ = tri.p0.z * u + tri.p1.z * v + tri.p2.z * w;
Vector3 newPoint = new Vector3(point.x, point.y, newZ);
// https://gamedev.stackexchange.com/a/63203/48697
void Barycentric2D(Vector2 p, Vector2 a, Vector2 b, Vector2 c, out float u, out float v, out float w)
{
Vector2 v0 = b - a;
Vector2 v1 = c - a;
Vector2 v2 = p - a;
float den = v0.x * v1.y - v1.x * v0.y;
v = (v2.x * v1.y - v1.x * v2.y) / den;
w = (v0.x * v2.y - v2.x * v0.y) / den;
u = 1.0f - v - w;
}
