Drawing with openTK, save to file, show on screen

Question

I am drawing multiple images over earh map, I am using perspective correct texturing link on each image.

I want to store rendered image in to file (sorce file is 1280x760, the rendered images is around 160x90 in most cases rotated). Currently I am doing this with GL.ReadPixels

int width = 1920;
int height = 1080;
using (Bitmap bitmap = new Bitmap(width, height))
{
    System.Drawing.Imaging.BitmapData bits = bitmap.LockBits(new Rectangle(0, 0, width, height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
    GL.ReadPixels(0, (0, width, height, OpenTK.Graphics.OpenGL.PixelFormat.Bgra, PixelType.UnsignedByte, bits.Scan0);
    bitmap.UnlockBits(bits);
    bitmap.RotateFlip(RotateFlipType.Rotate180FlipX);
    bitmap.Save("output.png", System.Drawing.Imaging.ImageFormat.Png);
}

The problem occurs when I move the map, rendered images are not visible anymore on the screen and it looks like that in this case GL.ReadPixels returns empty pixels.

How to get rendered image even if this is not currently rendered on screen?

Extra question, if I use framebuffer the result is the same but using the frame buffer I never see image on the screen, it looks like framebuffer is not drawn on the screen, but GL.ReadPixels can get image out. Which lines of code are needed to draw framebuffer also on the screen?

Any idea?

I am adding some code with drawing in to framebuffer but the result is empty image.

int FBOHandle = 0;
int ColorTexture = 0;
int DepthTexture = 0;

public bool canRender = false;
public void onRender()
{
    int zoom = (int)MainForm.mainMap.Zoom;
    VideoMapOverlayBitmap pob = null;
    lock (videoMapOverlayBitmapsSync)
        videoMapOverlayBitmaps.TryGetValue(zoom, out pob);
    if (pob == null)
        return;

    if (canRender)
    {
        canRender = false;
        int fboWidth = 1920;
        int fboHeight = 1080;

        // Create Color Tex for framebuffer
        GL.GenTextures(1, out ColorTexture);
        GL.BindTexture(TextureTarget.Texture2D, ColorTexture);
        GL.TexImage2D(TextureTarget.Texture2D, 0, PixelInternalFormat.Rgba8, fboWidth, fboHeight, 0, OpenTK.Graphics.OpenGL.PixelFormat.Rgba, PixelType.UnsignedByte, IntPtr.Zero);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMinFilter, (int)TextureMinFilter.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMagFilter, (int)TextureMagFilter.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapS, (int)TextureWrapMode.ClampToBorder);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapT, (int)TextureWrapMode.ClampToBorder);
        // GL.Ext.GenerateMipmap( GenerateMipmapTarget.Texture2D );

        // Create Depth Tex for framebuffer
        GL.GenTextures(1, out DepthTexture);
        GL.BindTexture(TextureTarget.Texture2D, DepthTexture);
        GL.TexImage2D(TextureTarget.Texture2D, 0, (PixelInternalFormat)All.DepthComponent32, fboWidth, fboHeight, 0, OpenTK.Graphics.OpenGL.PixelFormat.DepthComponent, PixelType.UnsignedInt, IntPtr.Zero);
        // things go horribly wrong if DepthComponent's Bitcount does not match the main Framebuffer's Depth
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMinFilter, (int)TextureMinFilter.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMagFilter, (int)TextureMagFilter.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapS, (int)TextureWrapMode.ClampToBorder);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapT, (int)TextureWrapMode.ClampToBorder);
        // GL.Ext.GenerateMipmap( GenerateMipmapTarget.Texture2D );

        // Create a FBO and attach the textures
        GL.Ext.GenFramebuffers(1, out FBOHandle);
        GL.Ext.BindFramebuffer(FramebufferTarget.FramebufferExt, FBOHandle);
        GL.Ext.FramebufferTexture2D(FramebufferTarget.FramebufferExt, FramebufferAttachment.ColorAttachment0Ext, TextureTarget.Texture2D, ColorTexture, 0);
        GL.Ext.FramebufferTexture2D(FramebufferTarget.FramebufferExt, FramebufferAttachment.DepthAttachmentExt, TextureTarget.Texture2D, DepthTexture, 0);

        //check for errors on framebuffer
        FramebufferErrorCode errorCode = GL.Ext.CheckFramebufferStatus(FramebufferTarget.Framebuffer);
        if (errorCode != FramebufferErrorCode.FramebufferComplete)
        {
            if (errorCode == FramebufferErrorCode.FramebufferUnsupported)
                Console.WriteLine("FramebufferUnsupported");
            OnUnload();
            return;
        }

        GL.ClearColor(0, 0, 0, 0);
        GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.DepthBufferBit);
        /* 
        //this corrupts my main screen
        GL.ClearColor(Color.White);
        GL.MatrixMode(MatrixMode.Projection);
        GL.LoadIdentity();
        GL.Ortho(0, fboWidth, fboHeight, 0, -1, 1);  // Up-left corner pixel has coordinate (0, 0)
        GL.Viewport(0, 0, fboWidth, fboHeight);      // Use all of the glControl painting area   
        */

        // Render all images
        PureProjection proj = MainForm.mainMap.MapProvider.Projection;
        List<VideoLogEntry> log = Log;
        //go over all images in the loop
        foreach (var videoEntry in log)
        {
            if (videoEntry == null)
                continue;
            if (videoEntry.projectedRectangleEmpty())
                continue;

            PointLatLng[] rect = videoEntry.getProjectedRectangle();

            if (videoEntry.bmp != null)
                videoEntry.createTexture();

            GL.Enable(EnableCap.Texture2D);
            GL.BindTexture(TextureTarget.Texture2D, videoEntry.texture);
            //GL.Ext.FramebufferTexture2D(FramebufferTarget.FramebufferExt, FramebufferAttachment.ColorAttachment0Ext, TextureTarget.Texture2D, videoEntry.texture, 0);
            //GL.DrawBuffers(1, new int[] { videoEntry.texture }); //compiler error

            //Do the magick for "Perspective correct texturing"
            // center point
            GPoint localTargetPosition = MainForm.instance.gMapControl.FromLatLngToLocalWithOffset(videoEntry.target);
            videoEntry.UpdatePolygonLocalPosition(videoEntry.projectedRectangle);
            // determines distances to center for all vertexes
            double dUL = Common.distance(new double[] { videoEntry.LocalPoints[0].X, videoEntry.LocalPoints[0].Y }, new double[] { localTargetPosition.X, localTargetPosition.Y });
            double dUR = Common.distance(new double[] { videoEntry.LocalPoints[1].X, videoEntry.LocalPoints[1].Y }, new double[] { localTargetPosition.X, localTargetPosition.Y });
            double dLR = Common.distance(new double[] { videoEntry.LocalPoints[2].X, videoEntry.LocalPoints[2].Y }, new double[] { localTargetPosition.X, localTargetPosition.Y });
            double dLL = Common.distance(new double[] { videoEntry.LocalPoints[3].X, videoEntry.LocalPoints[3].Y }, new double[] { localTargetPosition.X, localTargetPosition.Y });

            var texCoords = new[]
            {
                new Vector4(0, 0, 1, 1),
                new Vector4(1, 0, 1, 1),
                new Vector4(1, 1, 1, 1),
                new Vector4(0, 1, 1, 1)
            };

            texCoords[0] *= (float)((dUL + dLR) / dLR);
            texCoords[1] *= (float)((dUR + dLL) / dLL);
            texCoords[2] *= (float)((dLR + dUL) / dUL);
            texCoords[3] *= (float)((dLL + dUR) / dUR);

            GL.Begin(PrimitiveType.Quads);
            {
                GL.TexCoord4(texCoords[0]); GL.Vertex4(videoEntry.LocalPoints[0].X, videoEntry.LocalPoints[0].Y, 1, 1); //UL  LocalPoints[0] gimbalUL
                GL.TexCoord4(texCoords[1]); GL.Vertex4(videoEntry.LocalPoints[1].X, videoEntry.LocalPoints[1].Y, 1, 1); //UR  LocalPoints[1] gimbalUR
                GL.TexCoord4(texCoords[2]); GL.Vertex4(videoEntry.LocalPoints[2].X, videoEntry.LocalPoints[2].Y, 1, 1); //LR  LocalPoints[2] gimbalLR
                GL.TexCoord4(texCoords[3]); GL.Vertex4(videoEntry.LocalPoints[3].X, videoEntry.LocalPoints[3].Y, 1, 1); //LL  LocalPoints[3] gimbalLL
            }

            GL.End();
            GL.Disable(EnableCap.Texture2D);
        }

        // Grab your screenshot
        // draw FBO in to file
        lock (pob.masterBitmapSync)
        {
            using (Bitmap mybitmap = new Bitmap(fboWidth, fboHeight))
            {
                //fill bitmal so we will see what ReadPixels draw
                using (Graphics gfx = Graphics.FromImage(mybitmap))
                using (SolidBrush brush = new SolidBrush(Color.FromArgb(0, 0, 255)))
                {
                    gfx.FillRectangle(brush, 0, 0, mybitmap.Width, mybitmap.Height);
                }

                GPoint p = new GPoint(0, 0);
                int outputWidth = mybitmap.Width;
                int outputHeight = mybitmap.Height;

                System.Drawing.Imaging.BitmapData bits = mybitmap.LockBits(new Rectangle(0, 0, mybitmap.Width, mybitmap.Height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
                GL.ReadPixels((int)p.X, (int)p.Y, outputWidth, outputHeight, OpenTK.Graphics.OpenGL.PixelFormat.Bgra, PixelType.UnsignedByte, bits.Scan0);
                mybitmap.UnlockBits(bits);
                //mybitmap.RotateFlip(RotateFlipType.Rotate180FlipX);
                mybitmap.Save(@"c:\Downloads\aaa\ReadPixels_" + DateTime.Now.ToString("HHmmss_fff") + ".png", System.Drawing.Imaging.ImageFormat.Png);
                pob.masterBitmap.Dispose();
                pob.masterBitmap = null;
                pob.masterBitmap = (Bitmap)mybitmap.Clone();
            }
        }

        // Unload and dispose the frame buffer
        GL.Ext.BindFramebuffer(FramebufferTarget.FramebufferExt, 0);
        
        // Clean up what we allocated before exiting
        if (ColorTexture != 0)
            GL.DeleteTextures(1, ref ColorTexture);
        ColorTexture = 0;

        if (DepthTexture != 0)
            GL.DeleteTextures(1, ref DepthTexture);
        DepthTexture = 0;

        if (FBOHandle != 0)
            GL.Ext.DeleteFramebuffers(1, ref FBOHandle);
        FBOHandle = 0;
    }
}

score 0 · Answer 1 · answered Jan 11 '21 at 07:38

How to get rendered image even if this is not currently rendered on screen?

Create a new frame buffer, bind that frame buffer, set up viewport, set up ortho projection matrix, render your desired piece of the map into it, call GL.ReadPixels, save screenshot, unload and dispose the framebuffer.

Here's some sample code:

// Create framebuffer
int fboId = GL.GenFramebuffer();
GL.BindFramebuffer(FramebufferTarget.Framebuffer, fboId);

// Set up a framebuffer attachment here
int width = ...
int height = ...
int textureId = GL.GenTexture();
GL.BindTexture(TextureTarget.Texture2D, textureId);
GL.TexImage2D(TextureTarget.Texture2D, 0, PixelInternalFormat.Rgba8, width, height, 0, PixelFormat.Rgba, PixelType.Float, IntPtr.Zero);

GL.FramebufferTexture2D(FramebufferTarget.Framebuffer, (FramebufferAttachment)fbAtt.AttachmentType, TextureTarget.Texture2D, textureId, 0);

GL.DrawBuffers(1, new int[] { textureId });

GL.Viewport(0, 0, width, height);

// Set up ortho modo. probably also want to disable depth testing and any active blend modes
....

// Render your map now
....

// Grab your screenshot
.... 

// Unload and dispose the frame buffer
...

// Reset everything (viewport, ortho mode, etc.) if you don't your normal map to flicker for a frame
....

Your other question I don't quite understand

@Tryron I edit my question with code as you suggested. But I get empty image. Do I need to call GL.Ext.BindFramebuffer or GL.BindFramebuffer? — Martin86, Jan 11 '21 at 13:31
You might still need to set the ortho mode and viewport right. Reset them their original values after you are done to fix your mainscreen "corruption". That being said many things can cause your image to be empty. Might also be related to blending, dept testing or other reasons. — Tyron, Jan 11 '21 at 15:44

Drawing with openTK, save to file, show on screen

1 Answers1

Linked