OpenGl blurring

Question

I'm using libgdx and want to make a common Gaussian blur post-processing effect. Following this guide I faced some issues with texture filtering. Here the images:

actual image:

blurred with radius = 1:

blurred with radius = 5:

In the simplest case, I just have a transparent frame buffer object with some objects rendered to it. Then I need to apply some shader effects to this, basically only blur and then render the result to the screen. I also want to adjust the blur radius, but if I set the radius to more than 1 it becomes looking very rough. Guess it should be used with some linear filtering, but it isn't here. So I just need the same effect applied with a soft blur and configurable radius, maybe some other shader-sided options. And I've also tried to explicitly assign Linear filtering to FBO textures, which doesn't change anything.

Fragment shader:

//"in" attributes from our vertex shader
varying vec4 vColor;
varying vec2 vTexCoord;

//declare uniforms
uniform sampler2D u_texture;
uniform float resolution;
uniform float radius;
uniform vec2 dir;

void main() {
    //this will be our RGBA sum
    vec4 sum = vec4(0.0);
    
    //our original texcoord for this fragment
    vec2 tc = vTexCoord;
    
    //the amount to blur, i.e. how far off center to sample from 
    //1.0 -> blur by one pixel
    //2.0 -> blur by two pixels, etc.
    float blur = radius/resolution; 
    
    //the direction of our blur
    //(1.0, 0.0) -> x-axis blur
    //(0.0, 1.0) -> y-axis blur
    float hstep = dir.x;
    float vstep = dir.y;
    
    //apply blurring, using a 9-tap filter with predefined gaussian weights
    
    sum += texture2D(u_texture, vec2(tc.x - 4.0*blur*hstep, tc.y - 4.0*blur*vstep)) * 0.0162162162;
    sum += texture2D(u_texture, vec2(tc.x - 3.0*blur*hstep, tc.y - 3.0*blur*vstep)) * 0.0540540541;
    sum += texture2D(u_texture, vec2(tc.x - 2.0*blur*hstep, tc.y - 2.0*blur*vstep)) * 0.1216216216;
    sum += texture2D(u_texture, vec2(tc.x - 1.0*blur*hstep, tc.y - 1.0*blur*vstep)) * 0.1945945946;
    
    sum += texture2D(u_texture, vec2(tc.x, tc.y)) * 0.2270270270;
    
    sum += texture2D(u_texture, vec2(tc.x + 1.0*blur*hstep, tc.y + 1.0*blur*vstep)) * 0.1945945946;
    sum += texture2D(u_texture, vec2(tc.x + 2.0*blur*hstep, tc.y + 2.0*blur*vstep)) * 0.1216216216;
    sum += texture2D(u_texture, vec2(tc.x + 3.0*blur*hstep, tc.y + 3.0*blur*vstep)) * 0.0540540541;
    sum += texture2D(u_texture, vec2(tc.x + 4.0*blur*hstep, tc.y + 4.0*blur*vstep)) * 0.0162162162;

    gl_FragColor = vColor * sum;
}

The full class

Answer 1

From what I see you are not doing Gaussian blur at all...

Gaussian blur on image is convolution between image and Gaussian weighted matrix of resolution 1+2*r where r is radius of your blur. So outputted color should be weighted sum of all pixels up to distance of r from the targeted pixel.

What you are doing is just weighted sum of 9 pixels regardless of the radius that is wrong (in my optinion) as you should sum up ~6.28*r*r pixels instead. So I would expect 2 nested for loops instead ...

Here a small GLSL example I just busted together:

//---------------------------------------------------------------------------
// Fragment
//---------------------------------------------------------------------------
#version 420 core
//---------------------------------------------------------------------------
in vec2 pos;                    // screen position <-1,+1>
out vec4 gl_FragColor;          // fragment output color
uniform sampler2D txr;          // texture to blur
uniform float xs,ys;            // texture resolution
uniform float r;                // blur radius
//---------------------------------------------------------------------------
void main()
    {
    float x,y,xx,yy,rr=r*r,dx,dy,w,w0;
    w0=0.3780/pow(r,1.975);
    vec2 p;
    vec4 col=vec4(0.0,0.0,0.0,0.0);
    for (dx=1.0/xs,x=-r,p.x=0.5+(pos.x*0.5)+(x*dx);x<=r;x++,p.x+=dx){ xx=x*x;
     for (dy=1.0/ys,y=-r,p.y=0.5+(pos.y*0.5)+(y*dy);y<=r;y++,p.y+=dy){ yy=y*y;
      if (xx+yy<=rr)
        {
        w=w0*exp((-xx-yy)/(2.0*rr));
        col+=texture2D(txr,p)*w;
        }}}
    gl_FragColor=col;
    }
//---------------------------------------------------------------------------

output for r=5:

I normalized the weight so the brightness does not change too much from the original texture regardless of selected r... However for r=5 is the error biggest, all other radiuses seem much better... That might be because of aliasing of the circumference with the circle inside condition...

You should also add some edge cases handling (when convolution is done near edges of texture) as there the number of summed pixels is different so the coefficients should be scaled accordingly.

To describe the code above: the 2 loops just loop through all pixels up to radius r from pos in texture and convert from pixels x,y and NDC pos to texture coordinate p. Then for each position the Gaussian weight is computed and then its used for the weighted sum. After all this the resulting color is outputted.

[Edit1] 2 pass approach

I ported this to 2 pass rendering (first integrate horizontal lines, then vertical) I got this output (r=5):

Here Vertex shader:

//---------------------------------------------------------------------------
// Vertex
//---------------------------------------------------------------------------
#version 420 core
//---------------------------------------------------------------------------
layout(location=0) in vec4 vertex;
out vec2 pos;   // screen position <-1,+1>
void main()
    {
    pos=vertex.xy;
    gl_Position=vertex;
    }
//---------------------------------------------------------------------------

Here Fragment shader:

//---------------------------------------------------------------------------
// Fragment
//---------------------------------------------------------------------------
#version 420 core
//---------------------------------------------------------------------------
in vec2 pos;                    // screen position <-1,+1>
out vec4 gl_FragColor;          // fragment output color
uniform sampler2D txr;          // texture to blur
uniform float xs,ys;            // texture resolution
uniform float r;                // blur radius
uniform int axis;
//---------------------------------------------------------------------------
void main()
    {
    float x,y,rr=r*r,d,w,w0;
    vec2 p=0.5*(vec2(1.0,1.0)+pos);
    vec4 col=vec4(0.0,0.0,0.0,0.0);
    w0=0.5135/pow(r,0.96);
    if (axis==0) for (d=1.0/xs,x=-r,p.x+=x*d;x<=r;x++,p.x+=d){ w=w0*exp((-x*x)/(2.0*rr)); col+=texture2D(txr,p)*w; }
    if (axis==1) for (d=1.0/ys,y=-r,p.y+=y*d;y<=r;y++,p.y+=d){ w=w0*exp((-y*y)/(2.0*rr)); col+=texture2D(txr,p)*w; }
    gl_FragColor=col;
    }
//---------------------------------------------------------------------------

And just to be sure also CPU side C++/VCL/OpenGL code (old api to keep it simple):

//---------------------------------------------------------------------------
#include <vcl.h>
#pragma hdrstop
#include "Unit1.h"
#include "gl_simple.h"
//---------------------------------------------------------------------------
#pragma package(smart_init)
#pragma resource "*.dfm"
TForm1 *Form1;
GLuint  txr_img,txr_scr;
//---------------------------------------------------------------------------
void gl_draw()
    {
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    glUseProgram(prog_id);

    glUniform1i(glGetUniformLocation(prog_id,"txr" ),0);
    glUniform1f(glGetUniformLocation(prog_id,"xs"  ),xs);
    glUniform1f(glGetUniformLocation(prog_id,"ys"  ),ys);
    glUniform1f(glGetUniformLocation(prog_id,"r"   ),15.0);

    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glMatrixMode(GL_TEXTURE);
    glLoadIdentity();
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();

    glDisable(GL_DEPTH_TEST);
    glEnable(GL_TEXTURE_2D);

    glBindTexture(GL_TEXTURE_2D,txr_img);
    glUniform1i(glGetUniformLocation(prog_id,"axis"),0);
    glBegin(GL_QUADS);
    glColor3f(1,1,1);
    glVertex2f(-1.0,-1.0);
    glVertex2f(-1.0,+1.0);
    glVertex2f(+1.0,+1.0);
    glVertex2f(+1.0,-1.0);
    glEnd();

    glBindTexture(GL_TEXTURE_2D,txr_scr);
    glUniform1i(glGetUniformLocation(prog_id,"axis"),1);
    BYTE *dat=new BYTE[xs*ys*4];
    if (dat!=NULL)
        {
        glReadPixels(0,0,xs,ys,GL_BGRA,GL_UNSIGNED_BYTE,dat);
        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, xs, ys, 0, GL_BGRA, GL_UNSIGNED_BYTE, dat);
        delete[] dat;
        }
    glBegin(GL_QUADS);
    glColor3f(1,1,1);
    glVertex2f(-1.0,-1.0);
    glVertex2f(-1.0,+1.0);
    glVertex2f(+1.0,+1.0);
    glVertex2f(+1.0,-1.0);
    glEnd();

    glUseProgram(0);
    glBindTexture(GL_TEXTURE_2D,0);
    glFlush();
    SwapBuffers(hdc);
    }
//---------------------------------------------------------------------------
__fastcall TForm1::TForm1(TComponent* Owner):TForm(Owner)
    {
    gl_init(Handle);

    // textures
    Byte q;
    unsigned int *pp;
    int xs,ys,x,y,adr,*txr;
    union { unsigned int c32; Byte db[4]; } c;
    Graphics::TBitmap *bmp=new Graphics::TBitmap;   // new bmp

    // image texture
    bmp->LoadFromFile("texture.bmp");   // load from file
    bmp->HandleType=bmDIB;      // allow direct access to pixels
    bmp->PixelFormat=pf32bit;   // set pixel to 32bit so int is the same size as pixel
    xs=bmp->Width;              // resolution should be power of 2
    ys=bmp->Height;
    txr=new int[xs*ys];         // create linear framebuffer
    for(adr=0,y=0;y<ys;y++)
        {
        pp=(unsigned int*)bmp->ScanLine[y];
        for(x=0;x<xs;x++,adr++)
            {
            // rgb2bgr and copy bmp -> txr[]
            c.c32=pp[x];
            q      =c.db[2];
            c.db[2]=c.db[0];
            c.db[0]=q;
            txr[adr]=c.c32;
            }
        }
    glGenTextures(1,&txr_img);
    glEnable(GL_TEXTURE_2D);    // copy it to gfx card
    glBindTexture(GL_TEXTURE_2D,txr_img);
    glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S,GL_CLAMP);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T,GL_CLAMP);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_NEAREST);
    glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_MODULATE);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, xs, ys, 0, GL_RGBA, GL_UNSIGNED_BYTE, txr);
    glDisable(GL_TEXTURE_2D);
    delete[] txr;
    delete bmp;

    // screen texture
    glGenTextures(1,&txr_scr);
    glEnable(GL_TEXTURE_2D);
    glBindTexture(GL_TEXTURE_2D,txr_scr);
    glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S,GL_CLAMP);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T,GL_CLAMP);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_NEAREST);
    glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_MODULATE);
    glDisable(GL_TEXTURE_2D);


    int hnd,siz; char vertex[4096],fragment[4096];
    hnd=FileOpen("blur.glsl_vert",fmOpenRead); siz=FileSeek(hnd,0,2); FileSeek(hnd,0,0); FileRead(hnd,vertex  ,siz); vertex  [siz]=0; FileClose(hnd);
    hnd=FileOpen("blur.glsl_frag",fmOpenRead); siz=FileSeek(hnd,0,2); FileSeek(hnd,0,0); FileRead(hnd,fragment,siz); fragment[siz]=0; FileClose(hnd);
    glsl_init(vertex,fragment);
    hnd=FileCreate("GLSL.txt"); FileWrite(hnd,glsl_log,glsl_logs); FileClose(hnd);

    ClientWidth=xs;
    ClientHeight=ys;
    }
//---------------------------------------------------------------------------
void __fastcall TForm1::FormDestroy(TObject *Sender)
    {
    glDeleteTextures(1,&txr_img);
    glDeleteTextures(1,&txr_scr);
    gl_exit();
    glsl_exit();
    }
//---------------------------------------------------------------------------
void __fastcall TForm1::FormPaint(TObject *Sender)
    {
    gl_draw();
    }
//---------------------------------------------------------------------------
void __fastcall TForm1::Timer1Timer(TObject *Sender)
    {
    gl_draw();
    }
//---------------------------------------------------------------------------
void __fastcall TForm1::FormResize(TObject *Sender)
    {
    gl_resize(ClientWidth,ClientHeight);
    gl_draw();
    }
//---------------------------------------------------------------------------

The gl_simple.h can be found here:

• complete GL+GLSL+VAO/VBO C++ example

Answer 2

This is not caused by a lack of linear interpolation. This shader is doing only 9 texture fetches along each axis. You can't expect to sample only 9 times and get a smooth blur with a larger kernel, because you are skipping over a lot of pixels which might contain important information. Only radius = 1 is valid.

For a larger blur, you either need a larger kernel, or apply the smaller kernel several times.

To optimize if this gets too slow, you can leverage the linear interpolation technique from this article. Because linear interpolation lets you compute an arbitrary weighted average between two adjacent texels for the price of one, you can get an equivalent filter that performs just 5 texture fetches instead of 9, or use 9 texture fetches to get a kernel of size 17. Cleverly sampling from a pyramid of downsampled images is also a possibility.

By the way, instead of this verbose thing:

vec2(tc.x - 4.0*blur*hstep, tc.y - 4.0*blur*vstep)

You can simply write:

tc - 4.0*blur*dir

And similar for the other 7 lines.