I assume that your CADisplayLink is running on the main thread, because you don't specify otherwise. If that's the case, then you could be overloading the main thread with your rendering actions, preventing your touch events from being processed (all touch events are handled on the main thread).
My solution for a similar problem was to use a serial GCD queue for OpenGL ES rendering actions, combined with a dispatch semaphore to make sure that only one frame was being rendered at a time in response to a CADisplayLink firing. This answer of mine describes the process in detail, and I link to source code for an application which uses this. As a side benefit, this can improve your rendering speed by up to 40% on the newer multicore devices.