This general term for this question is called Convex Hull
It's widely used in GIS
https://gis.stackexchange.com/questions/1200/concave-hull-definition-algorithms-and-practical-solutions
And the most famous algorithm is done by Graham Scan from ACM
GRAHAM_SCAN(Q)
Find p0 in Q with minimum y-coordinate (and minimum x-coordinate if there are ties).
Sorted the remaining points of Q (that is, Q ? {p0}) by polar angle in counterclockwise order with respect to p0.
TOP [S] = 0 ? Lines 3-6 initialize the stack to contain, from bottom to top, first three points.
PUSH (p0, S)
PUSH (p1, S)
PUSH (p2, S)
for i = 3 to m ? Perform test for each point p3, ..., pm.
do while the angle between NEXT_TO_TOP[S], TOP[S], and pi makes a nonleft turn ? remove if not a vertex
do POP(S)
PUSH (S, pi)
return S
http://en.wikipedia.org/wiki/Graham_scan
http://www.personal.kent.edu/~rmuhamma/Compgeometry/MyCG/ConvexHull/GrahamScan/grahamScan.htm
Fun fact: convex OR concave hull has a patent:
https://stackoverflow.com/a/2241263/41948