How pick geometries in OpenGL with multisample framebuffer?

Question

(Edit) I made working geometry picking with framebuffer. My goal is draw huge scene in one draw call, but I need to draw to multisample color texture attachment (GL_COLOR_ATTACHMENT0) and draw to (eddited) non-multisample picking texture attachment (GL_COLOR_ATTACHMENT1). The problem is if I use multisample texture to pick, picking is corrupted because of multi-sampling.

I write geometry ID to fragment shader like this:

//...

// Given geometry id
uniform int in_object_id;

// Drawed to screen (GL_COLOR_ATTACHMENT0)
out vec4 out_frag_color0;

// Drawed to pick texture (GL_COLOR_ATTACHMENT1)
out vec4 out_frag_color1;

// ...

void main() {

    out_frag_color0 = ...; // Calculating lighting and other stuff

    //...

    const int max_byte1 = 256;
    const int max_byte2 = 65536;
    const float fmax_byte = 255.0;

    int a1 = in_object_id % max_byte1;
    int a2 = (in_object_id / max_byte1) % max_byte1;
    int a3 = (in_object_id / max_byte2) % max_byte1;

    //out_frag_color0 = vec4(a3 / fmax_byte, a2 / fmax_byte, a1 / fmax_byte, 1);
    out_frag_color1 = vec4(a3 / fmax_byte, a2 / fmax_byte, a1 / fmax_byte, 1);
}

(Point of that code is use RGB space for store geometry ID which is then read back a using for changing color of cube)

This happens when I move cursor by one pixel to left:

enter image description here

Because of alpha value of cube pixel:

enter image description here

Without multisample is works well. But multisampling multiplies my output color and geometry id is then corrupted, so it selects random cube with multiplied value.

(Edit) I can't attach one multisample texture target to color0 and non-multisample texture target to color1, it's not supported. How can I do this in one draw call?

enter image description here

Multisampling is not my friend I am not sure If I understand it well (whole framebuffering). Anyway, this way to pick geometries looks horrible for me (I meant calculating ID to color). Am I doing it well? How can I solve multisample problem? Is there better way?

PS: Sorry for low english. :)

Thanks.

Couldn't you simply use a different (non-multisampled) framebuffer for picking? You could even have a different shader that you only use for picking, instead of generating the rendering and picking values as two outputs of the same shader. — Reto Koradi, May 24 '15 at 01:29
I read again my question and I didn't write it correctly. I editted it. I use one framebuffer and multiple color attachments. Problem is I can't mix multisample and non-multisample attachments. Is it possible to use multiple framebuffers in one draw call? One for draw and one for pick with different fragment shaders? What is the solution? Thanks. — eSeverus, May 24 '15 at 11:09
@eSeverus: Picking by rendering IDs to color might not be the best strategy. You are ususally far better of just using some raycasting completely on CPU. However, I also don't know why you try to avoid an additional render pass for the picking - you especially will need only a 1x1 viewport/framebuffer for that, not the whole scene. Another thing is that you seem to do the the color picking after the multisample data is resolved. You could simply directly read one spefic sample of the multisampled texture and get one object or the other, but never a wild mix of IDs. — derhass, May 24 '15 at 16:44
@derhass Is there a way to read back a sample? That would be interesting, but I haven't found a way. It looks like `glReadPixels()` is not supported on multisample render targets. You could probably do something where you downsample with a custom shader that does not interpolate between samples, but just picks one of them. — Reto Koradi, May 25 '15 at 18:37
@RetoKoradi: yeah, you have to go thorugh a `texelFetch` for this to work. — derhass, May 25 '15 at 19:10

score 3 · Accepted Answer · answered May 25 '15 at 18:33

You can't do multisampled and non-multisampled rendering in a single draw call.

As you already found, using two color targets in an FBO, with only one of them being multisampled, is not supported. From the "Framebuffer Completeness" section in the spec:

The value of RENDERBUFFER_SAMPLES is the same for all attached renderbuffers; the value of TEXTURE_SAMPLES is the same for all attached textures; and, if the attached images are a mix of renderbuffers and textures, the value of RENDERBUFFER_SAMPLES matches the value of TEXTURE_SAMPLES.

You also can't render to multiple framebuffers at the same time. There is always one single current framebuffer.

The only reasonable option I can think of is to do picking in a separate pass. Then you can easily switch the framebuffer/attachment to a non-multisampled renderbuffer, and avoid all these issues.

Using a separate pass for picking seems cleaner to me anyway. This also allows you to use a specialized shader for each case, instead of always producing two outputs even if one of them is mostly unused.

I expected answer similar to yours. I seems I have to draw scene second time in special 'pick' mode, where are drawed only necessary polygons without effects. I assume I need some raycasting bounding boxes on CPU side to eliminate unnecessary objects as @derhass suggested, since scene is drawed second time to save a lot of time. I consider both answers are correct. Thanks. — eSeverus, May 25 '15 at 19:11

Thomas · Answer 2 · 2016-05-25T15:47:47.373

I think it is posible... You have to set the picking texture to multisampled and after rendering the scene, you can render 2 triangles over the screen and inside another fragmentshader you can readout each sample... to do that you have to use the GLSL command:

texelFetch(sampler, pixelposition/*[0-texturesize]*/, /*important*/layernumber);

Then you can render it into a single-sampled texture and read the color via glReadPixel.

I haven't tested it now, but I think it works

How pick geometries in OpenGL with multisample framebuffer?

2 Answers2