You have three zones to consider. The "perpendicular" approach is for the zone in the middle:

For the other two zones the distance is the distance to the nearest segment endpoint.
The equation for the segment is:
y[x] = m x + b
Where
m -> -((Ay - By)/(-Ax + By)),
b -> -((-Ax By + Ay By)/(Ax - By))
And the perpendiculars have slope -1/m
The equations for the perpendicular passing thru A is:
y[x] = (-Ax + By)/(Ay - By) x + (Ax^2 + Ay^2 - Ax By - Ay By)/(Ay - By)
And the perpendicular passing thru B is the same exchanging the A's and B's in the equation above.
So you can know in which region lies your point introducing its x coordinate in the above equations and then comparing the y coordinate of the point with the result of y[x]
Edit
How to find in which region lies your point?
Let's suppose Ax ≤ Bx (if it's the other way, just change the point labels in the following formulae)
We will call your point {x0,y0}
1) Calculate
f[x0] = (-Ax + By)/(Ay - By) x0 + (Ax^2 + Ay^2 - Ax By - Ay By)/(Ay - By)
and compare with y0.
If y0 > f[x0], then your point lies in the green field in the figure above and the nearest point is A.
2) Else, Calculate
g[x0] = (-Bx + Ay)/(By - Ay) x0 + (Bx^2 + By^2 - Bx Ay - By Ay)/(By - Ay)
and compare with y0.
If y0 < g[x0], then your point lies in the yellow field in the figure above and the nearest point is B.
3) Else, you are in the "perpendicular light blue zone", and any of the other answer tell you how to calculate the nearest point and distance (I am not going to plagiarize :))
HTH!