How to convert an image to pixels, edit it, and draw the edited image in Javascript

Question

I'm completely new to JS, I want to learn how to do the following:

Get an image data (convert it into an array of pixels so I could edit it) and then return the edited array back from the pixels editing function so that I could use these edited values to draw an edited image.

I'm not even sure if that's the way to go about this, but here's what I got so far:

  var img = new Image();
      img.src = 'img.png';
      var canvas = document.getElementById('canvas');
      var canvasEdited =  document.getElementById('canvasEdited');
      var ctx = canvas.getContext('2d');
      var arr = [];
      img.onload = function() {
          ctx.drawImage(img, 0, 0);
          function editPixels(ctx) {
             for (i of ctx) {
                 // edit pixels values 
                 arr.push(i - 10);    
             }
          }
          console.log(arr);
          function drawEditedImage() {
             var ctxEdited = canvasEdited.getContext('2d');
             ctxEdited.drawImage(img, 0, 0);
          }
     };

Console output shows an array with length:0.

Why doesn't it pushes edited pixels to arr?

heads up that you may run into issues with CORS trying to do something like this. — Max von Hippel, Jul 11 '17 at 23:34
@MaxvonHippel thanks for the heads up, but how do I edit an image then? Is that a completely wrong way of doing it? — Un1, Jul 11 '17 at 23:35
Try this tutorial https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Pixel_manipulation_with_canvas — Khauri, Jul 11 '17 at 23:36
How are you calling editPixels ? Right now you are not calling it at all so, your array's length is the same when initialized i.e, 0 — karthick, Jul 11 '17 at 23:39
@Un1 I don't know what the canonical answer should be to this, but I definitely think you're on the right track using `Canvas`. Do you specifically want to do this in straight-up JS, without using any tools or frameworks? (e.g, ImageMagick) — Max von Hippel, Jul 11 '17 at 23:42
@karthick I put it after the onload function but then it gave me an error and I removed it, and forgot to call it. How should I call it? — Un1, Jul 11 '17 at 23:51
@MaxvonHippel Well, I don't really mind using other libraries. I just want to learn how to edit images efficiently by changing pixels values and saving and drawing the edit image instead of the original. Thanks for the idea, I'll look into it as well — Un1, Jul 11 '17 at 23:55
@Un1 you have to call it within onload. But the argument you need to pass within that should be imagedata not the canvas context. — karthick, Jul 11 '17 at 23:59

score 6 · Answer 1 · answered Jul 12 '17 at 00:27

I was not as fast as the others, but I decided to post my answer anyway, because it still has some added value (break-down, base64, tainted canvas mention...).

I assume you want to change pixel data of an image via the canvas.

First, render an image to a canvas. You already got that right:

var canvasEdited =  document.getElementById('canvasEdited');
var ctxEdited = canvasEdited.getContext('2d');
ctxEdited.drawImage(img, 0, 0, canvasEdited.width, canvasEdited.height);

Then, get the pixel data out:

var imageData = ctxEdited.getImageData(0, 0, canvasEdited.width, canvasEdited.height);
// you will get an array of pixel data
// this array consists of rgba values
// each 'set' of four stands for red, green, blue and alpha

Then, do something with it, in this example making it gray:

for(var i = 0; i < imageData.length; i += 4) {
    var brightness = 0.34 * imageData[i] + 0.5 * imageData[i + 1] + 0.16 * imageData[i + 2];
    imageData[i] = brightness;
    imageData[i + 1] = brightness;
    imageData[i + 2] = brightness;
}

Then, draw the changed pixel data back on the canvas:

ctxEdited.putImageData(imageData, 0, 0);

If you would like to re-use that image outside the canvas, you can convert it to base64 like thisM

var base64URI = canvasEdited.toDataURL();

You can then set that data as a source of an element, like this:

img.src = base64URI;

Things to note:

You must open your page via localhost or a web server, otherwise the browser will block it. (unless you set the right flags)
You cannot get pixel data out a tainted canvas. This happens when you draw an image which does not specify crossOrigin. Read more about this at https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_enabled_image

score 4 · Accepted Answer · answered Jul 12 '17 at 00:18

There are lot of issues that needs to be addressed. If you want to getImageData from canvas then the image needs to be in the same domain. Easier fix is base64 format. Not an url. So convert your image to base64 string.

You are not calling the edit pixels anywhere, even if you are calling it in someplace and have not mentioned in the question, form your argument naming its clear that you are passing context rather than imagedata.

Use getImageData and putImageData to get and set data in canvas.

For loop needs to be changed, so that you don't set negative value on the alpha data. Image data is of format rgba.

Since you didn't mention how to manipulate the pixel. I am just reducing the rgb values by 100 to show the difference between the original image and the new image.

var img = new Image();
      img.src = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASwAAAEsBAMAAACLU5NGAAAAG1BMVEXMzMyWlpa3t7eqqqqcnJyjo6PFxcWxsbG+vr6NAD6nAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACjklEQVR4nO3XO2/bMBDA8fNTGn2OlGS00S8QAWnnaKi7xnBQdJSBFl3joY/RRpHv3SNFykYtdKOm/w8BHOkOIM3HkRYBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/uddvWvsI38sj+7xT/2lJ2n8WH64iMbkdCaqurTPteqda98eX6+zKnu9OkdDckIH3W21kbmWlR5F9rrT+6ukTMv3ettFY3JC9UZyfZBJ0czrZxuWjXwtr5KmRSOnmy4ak9PJ3bc+PMveJvJ0a+OwslfNv1knG6Ks7KIhOaGxulYXsl7YmNy1j/XV4nLRXLtoSE4oczN2epLK5mRSuj/x/7tuPImfOfHDKXnRRUNyaq6l17bhu7aX/q1N1fY8Va7TIRqSE/tuS2a78i3Nludu7QsbsW4+54dlFw3JadVqM1Uf3b6XqSsO+4V/n+nrtGt7qrZdYzQkp6WuFLUtNW3DYebqzXoZk0ZqWzZGQ3LqblmLvhFtRq5Ho9Ct9c25OI1c52M0JKftlrzpc+9ozfSilOfbctjRsv226FtbtuBvLpJmOuza8hW7Zyfa4lpeJFl5H24nZr/Fz4srReNQt9ahW5nGpn8d/azFaEhOyJ8hNi+HtnBn5yrvClcsW+44slmL0UPyKj8Jh0/fmVjd12HxV+3hM+CZ6DZUFW4Q9+GOcPShuT6sw5o/3PovEKMhOaG5frZBWPXdt2a2xkMV2JeNuNI62H1LqmJXF9J3O7XbQ65t4xP9+OguFMPdTr9Zkd9I313eLaxt28X5VtUuqAPe5eVH/ckalPyl/THzFn/5jF0H9qEMZC/Fz4toTAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEjpL514clJrNSt2AAAAAElFTkSuQmCC';
      var canvas = document.getElementById('canvas');
      var canvasEdited =  document.getElementById('canvasEdited');
      var ctx = canvas.getContext('2d');
      var arr = [];
      img.onload = function() {
          ctx.drawImage(img, 0, 0);

          var imageData = ctx.getImageData(0,0, canvas.width, canvas.height)

           editPixels(imageData.data);

           drawEditedImage( imageData);
     };

     function editPixels(imgData) {
              for (var i=0; i < imgData.length; i += 4) {
                  imgData[i] = imgData[i] - 100;
                imgData[i+1] = imgData[i+1] - 100;
                imgData[i+2] = imgData[i+2] - 100;
             }
     }

     function drawEditedImage(newData) {
         var ctxEdited = canvasEdited.getContext('2d');
         ctxEdited.putImageData(newData, 0, 0);
     }

jsfiddle for the same https://jsfiddle.net/karthick6891/3yhyyLyf/

Yeah, thanks a lot, that helped me to understand it! It works now. — Un1, Jul 12 '17 at 00:59
@Un1 sorry for the late reply. But to answer your earlier question. You need some kind of converted or algorithm to convert the img byte array to rgba pixels do the manipulation. convert it to byte array again and then create a blob and set it it img tag. I can't think of any libraries or implementation on top of the head now. But thats the way to do it, without canvas. But that involves cross browser issues, performance etc., so I suggest canvas. You can create a in memory canvas and manipulate using that. If you don't want canvas in markup. — karthick, Jul 12 '17 at 01:10

score 1 · Answer 3 · answered Jul 12 '17 at 00:08

1

I use the following to check if a given pixel is not transparent

canvas.getImageData[((mousePos.y - canvas.y) * canvaswidth + (mousePos.x - canvas.x)) * 4 + 3] !== 0

Canvas data is returned in rgba format so you need the color you want to specify in rgba values.

//red
canvas.getImageData[((mousePos.y - canvas.y) * canvaswidth + (mousePos.x - canvas.x)) * 4] = 0;

//green
canvas.getImageData[((mousePos.y - canvas.y) * canvaswidth + (mousePos.x - canvas.x)) * 1] = 0;

//blue
canvas.getImageData[((mousePos.y - canvas.y) * canvaswidth + (mousePos.x - canvas.x)) * 2] = 0;

//alpha
canvas.getImageData[((mousePos.y - canvas.y) * canvaswidth + (mousePos.x - canvas.x)) * 2] = 0;

Generally editing images in games will make use of an offscreen canvas which contains a spritesheet of all the game elements.

answered Jul 12 '17 at 00:08

David

609
1
4
11

Thanks for the info, unfortunately, I can't make my code work, it gives me either `cross-origin data` exception or an array of `length:0` – Un1 Jul 12 '17 at 00:15
That's normal you need to run the code on a server. Mostly you would use what you are talking about for rendering the character. You take a screenshot behind of what is behind the character and draw your character onto it otherwise your game gets too slow. – David Jul 12 '17 at 00:19
@Un1 this is the CORS issue you were given a heads up about - you can get a Chrome plugin to allow cross-origin-data if you are developing locally. – ESR Jul 12 '17 at 00:38
@EdmundReed oh, so that's just a problem with local development, thanks for the clarification! I didn't know that – Un1 Jul 12 '17 at 00:41
@Un1 yep, you shouldn't experience the cross-origin data issue once published to a server. – ESR Jul 12 '17 at 00:47
Oh and for buttons I find the easiest way to use them is to specify a specific color that no other pixel has like giving an alpha value of 0.8 or something and then just check if the pixel has that alpha to know if the button was clicked. – David Jul 12 '17 at 00:50

How to convert an image to pixels, edit it, and draw the edited image in Javascript

3 Answers3