I am attempting to build an application on IOS using gluon mobile that uses the devices camera to take a picture then send the file data of that picture to a backend server. However, this cannot be done directly since the PictureService
provided by gluon only returns a javafx Image object when a picture is taken.
To get around this I am attempting to get the raw pixel data from the Image object IOSImageService.java
, then use UIKit and Objective-C to take advantage of IOS native libraries to convert the raw pixel data into a JPEG file format JPEGConversion.m
(code mainly taken from here).
From my DEBUG OUTPUT
I can see that when I attempt to get the NSData object from the UIImage object it always comes out NULL and with a length of 0. I am not sure where I am going wrong with this code.
Am I using the UIKit libraries wrong? Am I not using Objective-C correctly? Or, am I not even sending to data to the Objective-C code in the proper format?
Everything here seems like I am going in the proper direction, I just have to successfully get the NSData for the JPEG file and then convert that into a jbyteArray
and return it.
Java: IOSImageService.java
package com.gluonhq.charm.down.plugins.ios;
import com.gluonhq.charm.down.plugins.ImageService;
import javafx.scene.image.Image;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelReader;
public class IOSImageService implements ImageService{
static {
System.loadLibrary("JPEGConversion");
}
private native byte[] nativeGetImageBytes(byte[] array, int width, int height);
@Override
public byte[] getImageBytes(Image image) {
int width = (int)image.getWidth();
int height = (int)image.getHeight();
System.out.println("Width: " + Integer.toString(width) + "\nHeight: " + Integer.toString(height));
//Get raw pixel data from image
byte[] BGRAFormattedArray = new byte[width * height * 4];
PixelReader p = image.getPixelReader();
p.getPixels(0, 0, width, height, PixelFormat.getByteBgraPreInstance(), BGRAFormattedArray, 0, width * 4);
System.out.println("Size: " + Integer.toString(BGRAFormattedArray.length));
//Change format from BGRA to RGBA
byte[] RGBAFormattedArray = new byte[width * height * 4];
for(int i=0;i<BGRAFormattedArray.length;i+=4) {
RGBAFormattedArray[i + 0] = BGRAFormattedArray[i + 2];
RGBAFormattedArray[i + 1] = BGRAFormattedArray[i + 1];
RGBAFormattedArray[i + 2] = BGRAFormattedArray[i + 0];
RGBAFormattedArray[i + 3] = BGRAFormattedArray[i + 3];
}
return nativeGetImageBytes(RGBAFormattedArray, width, height);
}
}
Objective-C: JPEGConversion.m
JNIEXPORT jbyteArray JNICALL Java_com_gluonhq_charm_down_plugins_ios_IOSImageService_nativeGetImageBytes
(JNIEnv *env, jobject obj, jbyteArray array, jint width, jint height)
{
jbyte* elements = (*env)->GetByteArrayElements(env, array, NULL);
if(elements == NULL) {
NSLog(@"RETURNING BECAUSE DATA WAS NULL");
return NULL;
}
int length = (*env)->GetArrayLength(env, array);
int w = (int)width;
int h = (int)height;
NSLog(@"ARRAY LENGTH: %i", length);
NSLog(@"Width: %i", w);
NSLog(@"Height: %i", h);
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, elements, (64 * 64 * 4), NULL);
int bitsPerComponent = 8;
int bitsPerPixel = 32;
int bytesPerRow = 4 * w;
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;
CGImageRef imageRef = CGImageCreate(w, h, bitsPerComponent, bitsPerPixel, bytesPerRow, colorSpaceRef, kCGBitmapByteOrderDefault | kCGImageAlphaLast, provider, NULL, false, renderingIntent);
size_t b_width = CGImageGetWidth(imageRef);
NSLog(@"IMAGE REFFERENCE WIDTH: %i", (int) b_width);
if(imageRef == NULL) {
NSLog(@"RETURNING BECAUSE IMAGE REFERENCE IS NULL");
return NULL;
}
UIImage *image = [UIImage imageWithCGImage:imageRef];
if(image == NULL) {
NSLog(@"UIIMAGE WAS NULL");
}else {
NSLog(@"UIIMAGE WAS NOT NULL");
}
NSData* pictureData = UIImageJPEGRepresentation(image, 1.0);
if(pictureData == NULL) {
NSLog(@"PICTURE DATA NULL");
}
NSLog(@"bytes in hex: %@", [pictureData description]);
//int picLength = (int)[pictureData length];
NSLog(@"Picture Length: %ld", [pictureData length]);
CGImageRelease(imageRef);
(*env)->ReleaseByteArrayElements(env, array, elements, JNI_ABORT);
//dummy return value
jbyte a[] = {1,255,1,31,1,15};
jbyteArray result = (*env)->NewByteArray(env, 6);
(*env)->SetByteArrayRegion(env, result, 0, 6, a);
return result;
}
DEBUG OUTPUT
Width: 128
Height: 128
Size: 65536
2018-03-14 15:51:22.117174-0230 MultiplatformGluonApplicationApp[1166:1138266] ARRAY LENGTH: 65536
2018-03-14 15:51:22.117270-0230 MultiplatformGluonApplicationApp[1166:1138266] Width: 128
2018-03-14 15:51:22.117349-0230 MultiplatformGluonApplicationApp[1166:1138266] Heigth: 128
2018-03-14 15:51:22.117413-0230 MultiplatformGluonApplicationApp[1166:1138266] IMAGE REFFERENCE WIDTH: 128
2018-03-14 15:51:22.117518-0230 MultiplatformGluonApplicationApp[1166:1138266] UIIMAGE WAS NOT NULL
2018-03-14 15:51:22.119407-0230 MultiplatformGluonApplicationApp[1166:1138266] PICTURE DATA NULL
2018-03-14 15:51:22.119504-0230 MultiplatformGluonApplicationApp[1166:1138266] bytes in hex: (null)
2018-03-14 15:51:22.119537-0230 MultiplatformGluonApplicationApp[1166:1138266] Picture Length: 0
JPG size: 6
[ 0x01 0xFF 0x01 0x1F 0x01 0x0F ]