how can i synchronize between the screenshot and the extraction of gameObject location in Unity?

Question

I’m trying to create data for detection.

not on android.

I’m moving an object on screen and every X seconds I’m saving a screenshot and the coordinate of the object on a json file.

The problem is that the screenshot doesn’t happening in the exact time I’m asking it to but the coordinate finding and json file creation is.

I managed to solve this problem by waiting a fixed frame number between each of the operation.

public static class WaitFor
{
    public static IEnumerator Frames(int frameCount)
    {
        while (frameCount > 0)
        {
            frameCount--;
            yield return null;
        }
    }
}

public IEnumerator CoroutineAction()
{
    yield return StartCoroutine(WaitFor.Frames(3)); 
}

public void Save()
{
    ScreenCapture.CaptureScreenshot(@".\" + counter + ".png");

    CoroutineAction();        

    Vector3 location = Camera.main.WorldToScreenPoint(gameObject.transform.position);

    System.IO.File.WriteAllText(@".\" + counter + ".json", toJson(location));

    counter++;
}

This solution somewhat works but it still not the exact location of the nodes and I would appreciate it if someone would know a better and cleaner solution.

Another problem I have is that if I try to use the super-resolution option

Example :

ScreenCapture.CaptureScreenshot(@".\" + counter + ".png" , 4);

And multiply the location vector by 4.

The locations will not be in the location of the object in the 2D image, not even close. Which doesn’t make any sense and It would really help me if I could create a bigger image without the use of external libraries to extend my picture by 4 (which work perfectly fine).

What is this `CoroutineAction` for? 1). It waits 3 frames for doing nothing? 2). You have to start it using `StartCoroutine(CoroutineAction)`. 3.) starting it in your `Save` method doesn't make the rest of that method get delayed if this i what you are trying to do. Did you try to save the location before taking the screenshot so place `location = ...` before `ScreenCapture.CaptureScreenshot ...`? — derHugo, Jan 10 '19 at 08:16
Possible duplicate of [Capture and save screenshot with ScreenCapture.CaptureScreenshot](https://stackoverflow.com/questions/46482323/capture-and-save-screenshot-with-screencapture-capturescreenshot) — derHugo, Jan 10 '19 at 08:18
1. yes , as far as i understood the image capturing is happening outside of the Update function as separate event , i dont like that solution which is why i posted a question here. 2. its do work and waiting the relevant frame. 3. hmm i will check it again but it seems to work for me. i tried to save it before, that was my first try. Thanks . — Ori Yampolsky, Jan 10 '19 at 08:31

derHugo · Answer 1 · 2019-01-10T11:18:27.647

Might not be answering your question so far but what I meant with the comment is

currently you 1. use the wait coroutine wrong and 2. it doesn't do anything

To 1.:

public void Save()
{
    ScreenCapture.CaptureScreenshot(@".\" + counter + ".png");

    CoroutineAction();        

    Vector3 location = Camera.main.WorldToScreenPoint(gameObject.transform.position);

    System.IO.File.WriteAllText(@".\" + counter + ".json", toJson(location));

    counter++;
}

What happens actually is:

ScreenCapture.CaptureScreenshot(@".\" + counter + ".png");
Vector3 location = Camera.main.WorldToScreenPoint(gameObject.transform.position);
System.IO.File.WriteAllText(@".\" + counter + ".json", toJson(location));
counter++;

is executed immediately. Than "parallel" (not really ofcourse) the coroutine would run if you used StartCoroutine(CoroutineAction());

But now to 2.:

the routine

public IEnumerator CoroutineAction()
{
    yield return StartCoroutine(WaitFor.Frames(3)); 
}

does absolutetly nothing than waiting 3 frames ...

I guess what you wanted to do instead is something like

// actually start a coroutine
public void Save()
{
    StartCoroutine(SaveRoutine());
}

// optional parameter superSize => if not provided uses 1
private IEnumerator SaveRoutine(int superSize = 1)
{
    // makes sure the factor is always >= 1
    var sizeFactor = Mathf.Max(1,superSize);

    ScreenCapture.CaptureScreenshot(@".\" + counter + ".png", sizeFactor);

    //Wait for 3 frames
    // I think this makes more sense than your dedicated class with the static method
    for (int i = 0; i < 4; i++)
    {
        yield return null;
    }

    Vector3 location = Camera.main.WorldToScreenPoint(gameObject.transform.position) * sizeFactor;

    System.IO.File.WriteAllText(@".\" + counter + ".json", toJson(location));

    counter++;
}

Than as also said already I'm not sure because I can't tell if you should get the location in the frame where you start the capture or in the frame where you end. But I guess you rather should get the location beforehand and than start the capture:

// optional parameter superSize => if not provided uses 1
private IEnumerator SaveRoutine(int superSize = 1)
{
    // make sure the factor is always at least 1
    var sizeFactor = Mathf.Max(1,superSize);
    var location = Camera.main.WorldToScreenPoint(gameObject.transform.position) * sizeFactor;

    ScreenCapture.CaptureScreenshot(@".\" + counter + ".png", sizeFactor);

     //Wait for 3 frames
    // I think this makes more sense than your dedicated class with the static method
    for (int i = 0; i < 4; i++)
    {
        yield return null;
    }    

    System.IO.File.WriteAllText(@".\" + counter + ".json", toJson(location));

    counter++;
}

Also note that as mentioned in the linked duplicate

The CaptureScreenshot returns immediately on Android. The screen capture continues in the background. The resulting screen shot is saved in the file system after a few seconds.

You are waiting 3 Frames .. that's usully not even a 1/10 of a second (depending on the frame rate obviously).

You should rather wait for actual seconds using WaitForSeconds e.g.

yield return new WaitForSeconds(3);

to wait e.g. 3 seconds. BUT also note that this will not work in combination with suggested Time.timeScale since WaitForSecodns depends on the timeScale as well.

I think you also should not / don't need to use

@".\" + counter + ".png"

in the filename but simply

counter + ".png"

since

On mobile platforms the filename is appended to the persistent data path. See Application.persistentDataPath for more information.

score 0 · Answer 2 · answered Jan 10 '19 at 08:06

0

Try to pause your movements like this:

Time.timeScale = 0;

(Game loop continues)

You can wait some frames/time, or see if you get some feedback when the screenshot is done.

This works best if you use Time.deltaTime in your Update() method which is recommended anyway.

answered Jan 10 '19 at 08:06

KYL3R

3,877
1
12
26

i tried it but for some reason it doesn't seems to work. i think there should be a way to connect events or put a function that will be performed exactly after the Screenshot. – Ori Yampolsky Jan 10 '19 at 08:09
are you on pc or android? [This post says the function returns immediately on android](https://stackoverflow.com/questions/46482323/capture-and-save-screenshot-with-screencapture-capturescreenshot) and it also gives an alternative. Hint: In general using "WaitForEndOfFrame" or `Update()` instead of `FixedUpdate` is recommended, because the (rendered) position seen on screen will not match with the position in `FixedUpdate`. – KYL3R Jan 10 '19 at 08:15
i'm running on a very strong computer , Update function. i cant say for sure but i think that the screen shot function is event caller. even if you will call it 3 times in row with different names it still will save the image only with the last name. i monitored it and it seems like the image is created outside of the update loop. – Ori Yampolsky Jan 10 '19 at 08:21
I think you should save your position first, then call `captureScreenshot` and wait 10 frames (to make sure). I guess captureScreenshot is async and takes some time to complete. Sadly there is no callback. But still the screenCapture should capture the exact moment of when you call the function. Just the saving takes time. – KYL3R Jan 10 '19 at 08:30
Thanks you, my first try was to do exactly as you said. but it didn't seems to work. anyways , i will try it again. as far as i managed to understand through monitoring, the image capturing doesn't happening immediately and its not exactly the same. and i think that if capturing the location before the image saving was working there is no reason to wait any number of frame at all. – Ori Yampolsky Jan 10 '19 at 08:35
The reason to wait is that you can't save the image with the position you just saved. And if you call `Capture` in too short interval, it will discard the unfinished screenshot and only save the last one. For accuracy: Use LateUpdate so that rendered position matches the position in code. – KYL3R Jan 10 '19 at 08:43

how can i synchronize between the screenshot and the extraction of gameObject location in Unity?

2 Answers2