Working with Computer Vision Thumbnails API repsonse data in JavaScript

Question

I'm using the Microsoft Cognitive Computer Vision API (the thumbnails function).

I'm trying to use JavaScript and I cannot make sense of the response.

My entire HTML document with embedded JS code is as follows:

<html>
<head>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>
    <button id="btn">Click here</button>
    <p id="response">

    <script type="text/javascript">
        $('#btn').click(function () {
            $.ajax({
                url: "https://api.projectoxford.ai/vision/v1.0/generateThumbnail?width=100&height=100&smartCropping=true",        
                beforeSend: function (xhrObj) {
                    xhrObj.setRequestHeader("Content-Type", "application/json");
                    xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key", "382f5abd65f74494935027f65a41a4bc");
                },
                type: "POST",
                data: '{"url": "https://oxfordportal.blob.core.windows.net/emotion/recognition1.jpg"}'
            })
            .done(function (response) {
                $("#response").text(response);
            })
            .fail(function (error) {
                $("#response").text(error);
            });
        });        
    </script>
</body>
</html>

The response I'm getting does not appear to be JSON, it look like this:

How can I work with the response from this API such that I get the image as a base 64 string that I can set to be the src on an image element.

It will end up being something like this but I do not know how to get the <base64string> bit.

<img src="data:image/png;base64,<base64string>">

I've tried everything in the api test console at https://dev.projectoxford.ai/docs/services/56f91f2d778daf23d8ec6739/operations/56f91f2e778daf14a499e1fb/console and it seems to work fine.

score 1 · Answer 1 · edited May 23 '17 at 12:07

1

The response from the service is a binary JPEG image, indicated by the response header "Content-Type: image/jpeg".

For advice on how to encode this as base64, and display it, you could look to these related answers:

edited May 23 '17 at 12:07

Community

1
1

answered Apr 30 '16 at 17:05

oliverw

31
2

Thanks, but this does not seem to answer the question (or I cannot conclude an answer from these links at least). I cannot find the right syntax for setting the image returned from the web service to be the src of my image. I have updated my code as follows: ` .done(function (response) { $('#thumbnail').attr("src",("data:image/jpg;base64," +response)); })` where '#thumbnail' is a standard img element in my html. I'm not using PHP, asp.net or any other kind of server-side code, just JavaScript and JQuery – Martin Kearn May 01 '16 at 13:51
From the links, I was hoping to suggest something like this: `$('#thumbnail').attr("src",("data:image/jpeg;base64,"+btoa(response)));`, which uses the browser's `btoa` function to do the base64 encoding. However, that doesn't seem to work, and on further reading it seems that jQuery's `.ajax` function does not play nice with binary responses, as discussed [here](http://www.henryalgus.com/reading-binary-files-using-jquery-ajax/). That page has a couple of solutions, in the form of a jQuery plugin, or just plain XHR calls. – oliverw May 02 '16 at 18:04
Thanks, I'll investigate further and report back here when i have an update – Martin Kearn May 03 '16 at 12:21
No luck with any of these links. I feel I'm getting close with `FileReader`. This is my latest `done` function: `var response = document.querySelector('#response'); var reader = new FileReader(); reader.addEventListener("load", function () { var image = new Image(); image.src = this.result; response.appendChild(image); }, false); reader.readAsDataURL(new Blob([file]));` but it is still not displaying the image – Martin Kearn May 06 '16 at 12:25

score 1 · Accepted Answer · answered May 06 '16 at 18:47

I think the problem is that jQuery converts the argument passed to .done into a string – not sure how to stop it doing that. You could try converting that string back to a binary object but that doesn't feel right or you could work out how to get the raw response from jQuery.

I tried this using XMLHttpRequest (which works):

            var xhr = new XMLHttpRequest();
            xhr.onreadystatechange = function() {
              if (this.readyState == 4 && this.status == 200) {

                console.log(this.response, typeof this.response);

                var response = document.querySelector('#response');
                var img = new Image();
                var url = window.URL || window.webkitURL;
                img.src = url.createObjectURL(this.response);
                response.appendChild(img);
              }
            }
            xhr.open('POST', 'https://api.projectoxford.ai/vision/v1.0/generateThumbnail?width=5&height=5&smartCropping=true');
            xhr.setRequestHeader("Content-Type", "application/json");
            xhr.setRequestHeader("Ocp-Apim-Subscription-Key", "382f5abd65f74494935027f65a41a4bc");
            xhr.responseType = 'blob';
            xhr.send('{"url": "https://oxfordportal.blob.core.windows.net/emotion/recognition1.jpg"}');

Working with Computer Vision Thumbnails API repsonse data in JavaScript

2 Answers2