Call your points a,b,c,d. They form a tetrahedron.
If the tetrahedron is not degenerated, the points are the vertex points (extreme points) of their convex hull.
To check this, check if a,b,c form a non-degenerated triangle. To do this compute the normal vector (call it n) of the triangle (b-a) cross (c-a). If it is zero then the triangle is degenerated, otherwise it is not..
The check if d is not in the plane of this triangle. It is if (d-a) dot product with n is zero. If so, the tetrahedron is degenerated.
Compute the normal vector for each of the 4 triangles of the tetrahedron.
Each normal vector together with (any) point of the triangle describes a half-space;
that is, all the points on one side of a plane.
Let n be the normal vector and p a point of the triangle.
A point q is said to be inside of the half-space, if n dot (q - p) is negative.
Check for all 4 triangles, if the forth point is inside. If not, adjust the sign of the normal vector.
A point is inside the tetrahedron if it is inside for all 4 half-spaces.