Given your reference to raster::pointDistance
I am assuming you are referring to geospatial data. The below shows how you can check if distance is to the line or, as you fear, to the nearest vertex defining a (poly-)line (not strictly a line in the mathematical sense).
library(terra)
pnts <- cbind(seq(1000,10000,1000), seq(1000,10000,1000))
crs <- "+proj=utm +zone=1 +datum=WGS84"
# 8 points
p <- vect(pnts[2:9, ], crs=crs)
# line defined by 2 vertices
line <- pnts[c(1,10), ]
line[,1] = line[,1]-500
line[,2] = line[,2]+500
x <- vect(line, type="lines", crs=crs)
plot(x)
points(x)
points(p, col="red")
distance(p, x)
# [,1]
#[1,] 707.1068
#[2,] 707.1068
#[3,] 707.1068
#[4,] 707.1068
#[5,] 707.1068
#[6,] 707.1068
#[7,] 707.1068
#[8,] 707.1068
sqrt(2*500^2)
#[1] 707.1068
The distances are all the same, so in this case you clearly get the distance to the line, not to its vertices. You could use the same test for methods in other packages.
And you can draw the orthogonal lines like this (with terra > 1.1-14):
n <- nearest(p, x)
lines(p, n)
For lon/lat data, you can use geosphere::distLine