Binary Data and GWT

Question

There are a couple of questions about binary data and GWT already. After reading them I am still not sure if the following is possible or not (I am a complete GWT beginner though!):

I have some very complicated data-files with only exist in binary form and I cannot convert them to something like XML or JSON. I have a closed source library though that accepts a byte[] and returns a Java object I can use. To get my GWT-app running I 'printed out' one of those binary data files and hard-coded the resulting byte[] in a .java file I access from my GWT-app code. Everything works fine. Obviously this is only a test and in the deployed app I cannot hard-code those data-files. I want to place them in the directory my GWT-app resides and the 'load' them with my GWT app.

I take it I can 'load' text files from my server with GWT, right? Why can't I read binary data with GWT? Or can I read the binary-data-files as text and the String into a byte[]? I read a lot about base64 encoding and that GWT can read it, although I don't really understand what they are talking about. Can I configure my server to serve those binary-data-files as base64 encoded and then read them with GWT?

Or is there some other solution? I wouldn't like to touch any JS code if I can help it. That's why I started using GWT ;)

Thanks for your help :)

Answer 1

Let's presume we are on HTML 4.

GWT client cannot "read" files. GWT client is javascript running on a browser. Browser security does not allow you to read local files. You have to get the servlet to proxy read the file for you on the server.

You set the mime type for a file because you want the browser to download a file and invoke the local PC to invoke the appropriate software - for example, pdf to invoke pdf reader or xls to invoke ms excel. Nothing to do with GWT Java or Javascript (except to enable the download).

Why do you need GWT client to read the binary file? If you do, your architecture is probably wrong. "Wrong" is an unkind word. Perhaps, misaligned is a better word. Your concept of AJAX thin client-server is misaligned. Drop your desktop processing concepts and habits at the door when you enter the door of GWT.

GWT is Java but not Java

I keep having to remind people that GWT Java is merely a more coherent representation of Javascript. When you code in GWT Java, always remember you are actually coding in Javascript not Java. All Java source is translated to Javascript

Therefore, the GWT compiler needs all Java classes to be supplied in source code. The GWT compiler has no ability to translate Java bytecode jar/class files into Javascript. If your library is in bytecode or your source library calls a bytecode library anywhere down the calling chain, the compilation will fail.

Confusion between server side and client side GWT

GWT RPC is sometimes source of confusion for GWT newbies. They don't seem to realise that the remote servlet is the only part that is compiled into bytecode because it is running on the server. Especially so, if you are using Vaadin - because they have so intentionally blurred the line between server and browser. And so the GWT newbie goes off wondering, "why do my bytecode libraries work at certain parts of the app only?"

The ajax client server architecture

GWT is merely a web enabled UI. Why can't you do whatever you want to do on the server and let the server reflect what it is doing or has done to the UI? Why must it be done on the browser?

Just imagine your GWT interface as a souped up JSP. Imagine you are writing a JSP. Do you get your JSP to suck your binary data into the browser and get the JSP to generate Javascript to analyse the binary data there?

I have written complex statistical analyses and I merely used the browser as a reflection of what is being done on the server. The engineer thinks he/she is running the analysis on his/her PC. Charts/reports are generated. But it's all done on the server by calling SAS.

The service oriented pattern/architecture

Your server will present services. Your browser GWT client will request for those services. Open a file, read the file, analyse the file, generate a visual/mime representation of analysis and pass it to the browser. Simply think of the GWT browser client as the display monitor for your server based manipulation. GWT is a magician's trick to help me conjure the illusion to let the engineers feel they are performing analysis on the local PC. Being engineers, of course, most of them know the browser is not actually doing the work.

When your user is satisfied with the analysis, get your service to generate a mime-representation of the results so that it could be downloaded by the browser to invoke the appropriate local PC software as mapped by the mime.

Do it on the server and reflect it on the browser.

Further Edits: Concerning binary data ...

The motivation behind base64 encoding being used in web apps: transmission of auth tokens, picture, audio files - so that their binary representation and sequencing would not be messed up by architectural nuances like endianness.

e.g., do not attempt writing a browser app to read a raw binary spreadsheet - always have the server translate it into XML or JSON (preferably JSON) where any binary element should be base64 encoded, before sending it to the browser app. Or if the purpose of your life is to climb Mt Everest, invent an architecture-agnostic encoding in place of base64 to transmit binary data.

Use only binary info if it was for the browser's OS processing (like audio, pictures, pdfs). No point in sending binary data to be processed solely by a javascript routine. The javascript routine would have to use extraneous processing time to translate it (unless again, if the purpose in your life is to climb ... ).

Answer 2

Yes it is possible.

Two solutions depending on the type of data.

1. Dynamic (if the binary data is dynamic and might change):
   Just Base64 encode your binary data on your backend and serve them (i.e GET request). Then you can use any of GWT's communication protocols (see here for more details) to retrieve the data from the backend.
   You then have to base64 decode the data and work with it (as you already have solved it).

2. Static (if the binary data won't change and is known during compile time):
   You can use ClientBundle (i.e.: DataResource) to generate those binary files during compile time and they can then be automatically retrieved on the client side without manually setting up transferring them.

Answer 3

Client Side:

@Override
public void onModuleLoad()
{
    RequestBuilder rb = new RequestBuilder(RequestBuilder.GET, "/test");
    try
    {
        rb.sendRequest(null, new RequestCallback()
        {
            @Override
            public void onResponseReceived( Request request, Response response )
            {
                String encoded = response.getText();
                byte[] data = decode(encoded);
                System.out.println(Arrays.toString(data));
            }

            @Override
            public void onError( Request request, Throwable exception )
            {
            }
        });
    }
    catch( RequestException e )
    {
        e.printStackTrace();
    }
}

private final static String base64chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

public static byte[] decode( String s )
{

    // remove/ignore any characters not in the base64 characters list
    // or the pad character -- particularly newlines
    s = s.replaceAll("[^" + base64chars + "=]", "");

    // replace any incoming padding with a zero pad (the 'A' character is
    // zero)
    String p = (s.charAt(s.length() - 1) == '=' ? (s.charAt(s.length() - 2) == '=' ? "AA" : "A") : "");
    s = s.substring(0, s.length() - p.length()) + p;
    int resLength = (int) Math.ceil(((s.length()) / 4f) * 3f);
    byte[] bufIn = new byte[resLength];
    int bufIn_i = 0;

    // increment over the length of this encrypted string, four characters
    // at a time
    for( int c = 0; c < s.length(); c += 4 )
    {

        // each of these four characters represents a 6-bit index in the
        // base64 characters list which, when concatenated, will give the
        // 24-bit number for the original 3 characters
        int n = (base64chars.indexOf(s.charAt(c)) << 18) + (base64chars.indexOf(s.charAt(c + 1)) << 12)
                + (base64chars.indexOf(s.charAt(c + 2)) << 6) + base64chars.indexOf(s.charAt(c + 3));

        // split the 24-bit number into the original three 8-bit (ASCII)
        // characters
        char c1 = (char) ((n >>> 16) & 0xFF);
        char c2 = (char) ((n >>> 8) & 0xFF);
        char c3 = (char) (n & 0xFF);

        bufIn[bufIn_i++] = (byte) c1;
        bufIn[bufIn_i++] = (byte) c2;
        bufIn[bufIn_i++] = (byte) c3;
    }

    byte[] out = new byte[bufIn.length - p.length()];
    System.arraycopy(bufIn, 0, out, 0, out.length);
    return out;
}

Server Side(Java):

@Override
public void doGet( HttpServletRequest request, HttpServletResponse response ) throws ServletException, IOException
{
    byte[] binaryData = new byte[1000];
    for( int i = 0; i < 1000; i++ )
        binaryData[i] = (byte) (Byte.MIN_VALUE + (i % (Math.pow(2, Byte.SIZE))));
    System.out.println("Sending: " + Arrays.toString(binaryData));

    byte[] base64Encoded = org.apache.commons.codec.binary.Base64.encodeBase64(binaryData);

    response.setContentType("application/octet-stream");
    PrintWriter out = response.getWriter();
    out.write(new String(base64Encoded));
}

Answer 4

Here's a solution that lets you easily read bytes from any URL:

        XMLHttpRequest request = XMLHttpRequest.create();
        request.open("GET", "http://127.0.0.1:8888/sample/index.bin");
        request.setResponseType(ResponseType.ArrayBuffer);
        request.setOnReadyStateChange(new ReadyStateChangeHandler() {

            @Override
            public void onReadyStateChange(XMLHttpRequest xhr) {
                if (xhr.getReadyState() == XMLHttpRequest.DONE) {
                    if (xhr.getStatus() == 200) {
                        ArrayBuffer buffer = xhr.getResponseArrayBuffer();
                        Uint8Array array = TypedArrays.createUint8Array(buffer);
                        System.out.println("got " + array.length() + " bytes: ");
                        for (int i = 0; i < array.length(); i++) {
                            System.out.println(array.get(i));
                        }
                    } else {
                        System.out.println("response status: " + xhr.getStatus() + " " + xhr.getStatusText());
                    }
                }
            }
        });
        request.send();