How to take & save picture / screenshot using Unity ARCore SDK?

Question

I found the answer to how to do it natively here but Im running Unity.

How to take picture with camera using ARCore

I'm not sure how to access the Unity surface renderer thread to be able to drop in those functions.

This looks like a good start. Ideas?

Edit:

Using Texture2d ReadPixels or ScreenCapture.CaptureScrenshot are not viable as they are blocking the render thread. The code below is enough to block the thread.

StartCoroutine(TakeScreenshot());

private IEnumerator TakeScreenshot()
    {
        yield return new WaitForEndOfFrame();

        Texture2D ss = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, false);
        ss.ReadPixels(new Rect(0, 0, Screen.width, Screen.height), 0, 0);
        ss.Apply();

Edit 2: Am considering using this technique. Will report back if successful.

What's wrong with using the second screenshot method from [this](https://stackoverflow.com/questions/46482323/capture-and-save-screenshot-with-screencapture-capturescreenshot) answer? — Programmer, Feb 27 '18 at 20:03
@Programmer while it says that its asynchronous, as noted below, it blocks the render thread and freezes the app for about 3 seconds. Its not a good user experience. — codeisforeva, Feb 27 '18 at 20:07
The saving part is what's making it take 3 seconds. Remove the `File.WriteAllBytes` at the end and you should be fine — Programmer, Feb 27 '18 at 20:09
@Programmer I guess I should edit this and say that I want to save it ;) — codeisforeva, Feb 27 '18 at 20:13
Ok. Do that. Before you do that, check and make sure that without the `File.WriteAllBytes`, it is not blocking for 3 seconds anymore. It shouldn't but just do the test to verify that. Also post your current code that is causing the 3 seconds wait. — Programmer, Feb 27 '18 at 20:17
@Programmer its not the file WriteAllBytes. Its still blocked with the code above. — codeisforeva, Feb 27 '18 at 20:36
This is not true. `ReadPixels` or the lines of code in that `TakeScreenshot` function cannot block Unity or 3 seconds. How did you conclude that it is freezing for 3 seconds and what makes you think it is that line of code? I did a quick test again and your conclusion is wrong.... — Programmer, Feb 27 '18 at 21:00
@Programmer lol k, glad you're running my code base with the Unity AR Core SDK. I push the button, with all code commented in the function, the app doesn't lock. I add those lines of code to the function, app locks up. I can provide a video if it makes you feel better about telling someone on the internet they're wrong. — codeisforeva, Feb 27 '18 at 21:09
@Programmer, `glReadPixels()` blocks the pipeline until all rendering commands are complete, and waits until all pixel data are transferred and ready for use before it returns control to the calling application. — Hamid Yusifli, Feb 27 '18 at 21:20
@Programmer here is a very clear outline of why it blocks the thread- https://medium.com/google-developers/real-time-image-capture-in-unity-458de1364a4c — codeisforeva, Feb 27 '18 at 23:21
To @Shaman and OP.... I know this and I am not saying that it doesn't block. I am just saying that 3 seconds of blocking is just too much. The blocking is usually in ms not seconds.I *wasn't* expecting a video but instead a screenshot from the Profiler and I think that *"adding videos will make me feel better"*.Anyways, good luck with this issue. I hope you find an solution to your problem. — Programmer, Feb 27 '18 at 23:47
@Programmer next time you want to help someone and you tell them that they're wrong and they can absolutely verify is not true, you might want to do it with a little more tact. I am looking at using a newer API that has been shown to me. Thanks for wasting my time. https://docs.unity3d.com/2018.1/Documentation/ScriptReference/Experimental.Rendering.AsyncGPUReadback.Request.html — codeisforeva, Feb 28 '18 at 00:34
You are welcome. Next time you post a question about performance issue and get a comment from another user who actually spent their time to do a test and some profiler but ended up with a different result, and asked for more information, don't be offended. Instead of getting mad, counter them with a proof of your issue such as a screenshot from the Profiler which will show what is causing the issue. — Programmer, Feb 28 '18 at 01:04
Also, the medium link you provided is doing extra steps that you don't even need especially since you just need to save the image.You don't need the code from that post since you are not doing image processing... Finally, this post a duplicate to another question with an actual solution but I won't close it since you'll think I retaliating against you for some reason. Happy coding! — Programmer, Feb 28 '18 at 01:10
@Programmer again with the passive aggressive response, its not necessary at all. Just carry on and don't get involved if you don't like the way other people debug. There are multiple ways to solve every programming problem and the "duplicate" that you state is not suitable in my case. I wouldn't have opened this question if so but go ahead and close if you want to threaten me with your power to. — codeisforeva, Feb 28 '18 at 04:55
@codeisforeva have you read this? https://answers.unity.com/questions/465409/reading-from-a-rendertexture-is-slow-how-to-improv.html Also this could possibly make it faster: https://docs.unity3d.com/ScriptReference/Graphics.CopyTexture.html — Hristo, Feb 28 '18 at 08:37
@Hristo thanks for those. Im hesistant to try the native OpenGL import. Seems dangerous. The yielding extra frames didn't help unfortunately. — codeisforeva, Mar 01 '18 at 18:16

score 1 · Answer 1 · answered Feb 27 '18 at 20:05

1

I found this repo on github which makes it super simple to use both native iOS and Android code to save a screenshot to the device. Technically you could just use the Texture2D capture code to get the screenshot, but the follow NativeGallery call allows you to write it to the device. The GitHub readme covers what settings you need to enable in order to allow writing to the device.

private void ScreenshotButtonClicked()
{
    StartCoroutine(TakeScreenshot());
}

private IEnumerator TakeScreenshot()
{
    SetUIActive(false);

    yield return new WaitForEndOfFrame();

    Texture2D ss = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, false);
    ss.ReadPixels(new Rect(0, 0, Screen.width, Screen.height), 0, 0);
    ss.Apply();

    NativeGallery.SaveToGallery(ss, "Native Screenshots", DateTime.Now.ToString().Replace("/", "-"));

    SetUIActive(true);
}

private void SetUIActive(bool active)
{
    Button.gameObject.SetActive(active);
    ScaleSlider.gameObject.SetActive(active);
}

answered Feb 27 '18 at 20:05

hanoldaa

616
5
10

Unfortunately this has the same symptoms as using File.WriteAllBytes in that the thread is blocked on the Unity side even without using the NativeGallery plugin. – codeisforeva Feb 27 '18 at 20:37
I know it's not the most elegant or optimized solution, but if you're using File.WriteAllBytes, could you just split the byte array into chunks and append each chunk over several frames in a coroutine? – hanoldaa Feb 28 '18 at 00:47
Ya definitely could work that way. Unfortunately the issue is further up the chain in ReadPixels. This causes a GPU flush of commands that appears to be even more problematic when running something like ARCore. – codeisforeva Feb 28 '18 at 05:04
A screenshot is not a photo. This is a hack at best. – Andrea Feb 01 '22 at 14:07

score 1 · Accepted Answer · answered Mar 01 '18 at 18:56

1

ARCore provides its own TextureReader class that is extremely fast and doesn't reproduce the side effects noted. TextureReader_acquireFrame

https://github.com/google-ar/arcore-unity-sdk/blob/master/Assets/GoogleARCore/Examples/ComputerVision/Scripts/TextureReaderApi.cs

answered Mar 01 '18 at 18:56

codeisforeva

471
5
20

The ComputerVisionController class has a great example. – codeisforeva Mar 01 '18 at 19:29

score 0 · Answer 3 · answered Feb 27 '18 at 19:20

0

Are you looking to learn how to do it? If you are just looking to capture a screenshot - it is built into Unity. See https://docs.unity3d.com/ScriptReference/Application.CaptureScreenshot.html

answered Feb 27 '18 at 19:20

Clayton Wilkinson

4,524
1
16
25

Yea this function is garbage though. It always blocks the render thread. I can already do it in native iOS. The point being I would like to capture a photo and have my app not pause for 4 seconds. I have tried this many time. https://stackoverflow.com/questions/46482323/capture-and-save-screenshot-with-screencapture-capturescreenshot – codeisforeva Feb 27 '18 at 19:29

How to take & save picture / screenshot using Unity ARCore SDK?

3 Answers3

Linked