OpenAL might not be great for this purpose. I would recommend using SDL (perhaps with SDL_Mixer) or PortAudio. Both are cross-platform and both are fairly easy to use.
To use OpenAL for this purpose you have to poll to see how many samples have accumulated in a hidden circular buffer. Then you grab them out when you decide you're ready. Then you apply your filter. Then you attach the filtered data to a buffer. Then you queue the buffer onto a source and tell the source to play (unless it's already playing). The strength of OpenAL, in my opinion, is for applying 3D effects. It's not as great at real-time filtering, although I expect you can make it work if you're willing to accept a little more lag on the output. I prefer the callback model over the buffer-object model for filtering.
Edit: I posted some code that would do exactly this over at: OpenAL: How to create simple "Microphone Echo" programm? A comment points out where one would put the processing.