The use of shaders and the method used to send in geometry are orthogonal, i.e. the one thing has nothing to do with the other.
The big problem with Immediate Mode is, that it hogs your CPU, as all these little calls go up and down through plenty of code. The secondary problem is, that both Immediate Mode and Client Side Vertex Arrays are limited by system bus bandwidth. To put this into numbers: PCI-Express Gen 3 with 16 Lanes can transfer about 15GiB/s (in practical applications it's more like 10GiB/s). The GPU memory is connected with bandwidth of at least 10 times that. By using a VBO^1 you can make the GPU draw huge amounts of geometry by sending only a few bytes (whatever the glDrawElements command translates into).
Shaders OTOH are the little programs executed on the GPU that turn geometry data into on-screen-locations and fill them with pixel values. The shader doesn't care which way the data was sent into the GPU. Also "not using" a shader doesn't mean things don't happen on the GPU. For one there are default shaders written in-situ to match the configured fixed function state. And even if that wasn't the case, what makes you think not using a shader would put things off the GPU?
[1] Vertex Array Objects are only abstract state holders, collecting a set of VBOs and which vertex attributes they're bound to.