Aside from the obvious dumbest algorithm which would be to take every Nth vertex from the polygon (reducing the number of vertices by N), here is an idea inspired by some 3D algorithms.
Usually, in 3D, what is required is to remove the faces that contribute less to the overall volume. To do this, we try to simplify the "flattest" areas of the model.
Now in 2D, you could translate this to "simplify the segments that have the smallest angle between them. A first naïve implementation could be:
- Compute all angles between the segments Si and Si+1 of the polygon
- Take all the angles below a given threshold (or take the M smallest angles)
- Simplify the segments we identified in 2. (replace [Pi, Pi+1] and [Pi+1, Pi+2] by [Pi, Pi+2])
- Repeat from 1. until we have sufficiently reduced our polygon
Of course, this is not optimal but it should be a good trade of between quality and speed. Instead of the angle, you could take the minimal distance between the middle point of two segments (Pi+1) and the potentially simplified segment ([Pi, Pi+2])
Edit:
Another algorithm I would try if I did not need too much performance:
- Consider the original polygon vertices as the control points of a Catmull-Rom spline
- Tesselate this spline at the desired number of points
Finally, I found some source code for you on that link: http://motiondraw.com/md/as_samples/t/LineGeneralization/demo.html, as well as the associated algorithms: http://www.geom.unimelb.edu.au/gisweb/LGmodule/LGSimplification.htm