In general, you don't. OpenGL is ultimately a hardware abstraction. And OpenGL simply doesn't provide a way to get that sort of information.
There are vendor-specific extensions that will give you ways to ask, though what you get back depends on the architecture. AMD hardware provides the ATI_meminfo extension. It breaks memory down into types of objects: buffer objects, textures, and renderbuffers.
NVIDIA provides the experimental extension NVX_gpu_memory_info. There's no information in the registry about how to use it, so I can't link you to anything.
In any case, the most effective way to know what the GPU is using is to just keep track of it yourself. Always use internal image formats with sizes; this means you can compute a pretty good estimate of how much memory a texture takes up. The same goes for buffer objects and so forth.
You won't get exact numbers, as padding, alignment, and the like can confound you. But you'll get something pretty decent.