I was always using KITTI raw LiDaR scans data set as a ground truth for my depth studies and thought that they are representing distance between some point and camera plane, i.e. z coordinate.
For example, pixel depth of 364th row of the image below
is shown here
I got more or less the same distance for all the points on the road (left spikes in the plot) and conclude that the distance is from camera plane. If not, there should be some curvature. Is my understanding correct?