0

I am using renderscript for live image processing (camera preview). I am having an issue on Nexus 6 Marshmallow 6.1 where some of my scripts won't run for more than a couple of frames (rs kernel runs). Those same scripts work flawlessly on Nexus 4 Lollipop 5.1.

Symptoms: Script works for several runs (builds). On an nth run some script ceases to work as expected and all subsequent runs exhibit the above issue. I haven't been able to establish some particular operation in code which causes the issue. It appears completely random, at least from what I can gather.

What iv'e tried:

  • Rebooting phone, uninstalling app, cleaning the projects, invalidating caches in Android Studio produce no results.
  • Adding rsDebug() per this post:
    RenderScript code not working without rsDebug seemed to have fixed the problem, but after a few builds problem reappeared.
  • Adding rsDebug() in code and actually logging appears to run the script as intended, but needless to say this is no solution since it slows the scripts down to a halt.
  • Removing #pragma rs_fp_relaxed seemed to have fixed the problem, but after a few builds problem reappeared.
  • adb shell setprop debug.rs.default-CPU-driver 1 fixes the problem, but the whole point of using renderscript was to utilize heterogeneous computing

I am using this kernel signature uchar4 __attribute__((kernel)) filter(uchar4 v_in, uint32_t x, uint32_t y), although RS_KERNEL causes the same issue.

Thanks for any help and ideas.

Sample of affected code: (Google demo code from here: https://android.googlesource.com/platform/frameworks/rs/+/master/java/tests/ImageProcessing2/src/com/android/rs/image/)

static float sr = 0.f;
static float sg = 0.f;
static float sb = 0.f;

void prepareBwFilter(uint32_t rw, uint32_t gw, uint32_t bw) {

    sr = rw;
    sg = gw;
    sb = bw;

    float imageMin = min(sg,sb);
    imageMin = fmin(sr,imageMin);
    float imageMax = max(sg,sb);
    imageMax = fmax(sr,imageMax);
    float avg = (imageMin + imageMax)/2;
    sb /= avg;
    sg /= avg;
    sr /= avg;

}

void bwFilterKernel(const uchar4 *in, uchar4 *out) {
    float r = in->r * sr;
    float g = in->g * sg;
    float b = in->b * sb;
    float localMin, localMax, avg;
    localMin = fmin(g,b);
    localMin = fmin(r,localMin);
    localMax = fmax(g,b);
    localMax = fmax(r,localMax);
    avg = (localMin+localMax) * 0.5f;
    out->r = out->g = out->b = rsClamp(avg, 0, 255);
}
Community
  • 1
  • 1
Sinisa
  • 240
  • 3
  • 14
  • Can you post more about your code? This looks like a GPGPU issue (since the setprop fixes things). I agree that setprop isn't a solution, but it is peculiar that this fixes it, while other GPGPU-disabling features don't. If you can isolate things further, we can look at what is really going wrong. – Stephen Hines Mar 10 '16 at 15:52
  • I gave my best to pinpoint the issue, but with little success. I agree it's most probably a GPU issue.I red you comment in this post [How can one verify whether a Android Mobile Device supports GPU renderscript or not?](http://stackoverflow.com/questions/21150292/how-can-one-verify-whether-a-android-mobile-device-supports-gpu-renderscript-or?rq=1) but what's interesting here is that both phones use Adreno GPUs (320 & 420). Still, i'm not sure if Nexus 4 provides GPU support for renderscript at all, although I would assume so by the speed of rendering. – Sinisa Mar 10 '16 at 16:21
  • You have "localMin = r > localMin ? r : localMin;" twice in your code, so perhaps the algorithm is just misbehaving? – Stephen Hines Mar 10 '16 at 16:25
  • It works on Nexus 4, or on Nexus 6 when CPU is forced. Also, this is one of many scripts that do not work and i intentionally posted Googles demo code because i assume they wrote optimal code for renderscript since renderscript it self is their product. – Sinisa Mar 10 '16 at 16:30
  • That is sample code, not really optimal code. You also did mis-translate it when you removed the fmax/fmin operations, which is what I was pointing out about your code above (you set localMin a third time instead of setting localMax). It also would help to understand what you mean by the failure in this case. Does the app crash? Does it just stop running RenderScript code? Does it stop displaying things (or making things black&white)? – Stephen Hines Mar 10 '16 at 16:41
  • It renders one frame, then produces black screen, one or two more frames and then it's all black. App is responsive the whole time and logcat produces no errors or warnings. I agree the code is not optimized, but it's proven to work on both nexus 4 and 6. This exact code worked on nexus 6 for a few days of app development and then abruptly stopped working. And this is what i don't get i have scripts working, then not working, then at somepoint they start working again and so on. With no changes to code that is. – Sinisa Mar 10 '16 at 21:24
  • Did you set renderscriptTargetApi 18? – Settembrini Mar 11 '16 at 08:18
  • I am using regular renderscript currently since i was affected by issues discussed here [Bug: cannot instantiate RenderScript on](https://code.google.com/p/android/issues/detail?id=182356). – Sinisa Mar 11 '16 at 08:57
  • Just as an idea. Could this erratic behaviour also be caused by certain features of the camera from which you use preview pics as input? Is your code using new camera2? Do you have other apps on the device that possibly don't release the camera? – Settembrini Mar 11 '16 at 12:40
  • I'm using the old camera API. Checked the preview and it's working as expected. I know that camera works as well when the issue appears since i have more than one filter in app and i switch between them for sanity checks. – Sinisa Mar 11 '16 at 13:02

2 Answers2

0

Can you double check the image you are writing to is not looking at the alpha channel or try to set it explicitly. (This is more of a comment, but I don't have enough points)

sakridge
  • 578
  • 3
  • 9
  • Checked, also, the issue i have involves multiple scripts (some from the google demo mentioned above) which do different things in terms of image processing. Also, as i mentioned all the scripts that do not work on Nexus 6 actually work on Nexus 4. – Sinisa Mar 11 '16 at 08:50
  • OK; please add a code sample which reproduces the problem. – sakridge Mar 15 '16 at 02:52
0

After quite a bit of research I decided that this is most probably a GPU driver issue. Having said that, I mentioned above that I tried removing #pragma rs_fp_relaxed which appeared to have solved the problem temporarily. I now believe that it was a gamble with the choice of fp precision RS was using and that that was the reason it sometimes worked and sometimes not. I came to this conclusion when i explicitly set #pragma rs_fp_full which seems to have fixed the issue permanently since it, along with native functions, should be a hardware backed calculation (so far works for all scripts that caused problems on Nexus 6).

I found a few cases on the web with people solving RS issues by flashing new drivers, but this is unacceptable for me.

To recap: I explicitly set #pragma rs_fp_full.

Sinisa
  • 240
  • 3
  • 14