I'm writing a Chrome extension that blocks possibly offensive content. One method that I am implementing is to scan all the images and see how much skin is showing. I create a new image object, set the crossOrigin flag to " ", then make a onload function that will draw the image onto the canvas, read the data from the canvas, and then perform the analysis, setting a boolean flag for the calling function. After defining the onload function, I assign a src to my image node from my list of sources from the webpage.
The image_scanner function is called inside of a for loop that is looping through each image node on the webpage and performing various operations to block on. This is the last operation that I perform. Here is the code that calls image_scanner:
if (image_scanner(options.scanner_sensitivity, images[i]))
{
// Replace the image with a blank white image
images[i].src = chrome.extension.getURL("replacement.png");
}
Here is the image_scanner function
function image_scanner(sensitivity, image)
{
// Sensitivity is a number and image is an image node.
// Declare a variable to count the number of skin pixels
var skin_count = 0;
if (image.width == 0 && image.height ==0)
{
// This means the image has no size and we cannot block it.
return false;
} // end if
var return_value = null; // set bool flag
// Create an HTML5 canvas object.
var canvas = document.createElement('canvas');
//window.alert("Created Canvas."); // used for testing.
// Get the context for the canvas.
var context = canvas.getContext("2d"); // This is what we actually use to draw images and pull the data from them.
context.canvas.width = image.width; // Set the canvas width to the width of the image
context.canvas.height = image.height; // Set the canvas height to the height of the image
img = new Image(); // Create a new image node to circumvent cross-domain restrictions.
img.crossOrigin = " "; // Set crossOrigin flag to ' ' so we can extract data from it.
img.onload = function(){
window.alert(img.src); // This always gives the same src until Chrome ends the function
context.drawImage(this, 0,0); // Draw the image onto the canvas.
var pixels = context.getImageData(0, 0, image.width, image.height).data;
// Now pixels is an array where every four entries in the array is the RGBa for a single pixel.
// So pixels[0] is the R value for the first pixel, pixels[1] is the G value for the first pixel,
// pixels[2] is the B value for the first pixel, and pixels[3] is the a (alpha or transparency) value for the first pixel.
// This means that pixels.length/4 is the number of pixels in the image.
// Now we calculate the number of skin pixels we can have before blocking the image.
var limit = ((pixels.length)/4) * (sensitivity/100);
// Now we go through the array of pixel data, checking if each pixel is a skin colored pixel based on its RGB value (the first 3 entries for that pixel in the pixels array)
// Each time we find a skin colored pixel, we increment skin_count and check if skin_count >= limit. If so, we return true.
for (var i = 0; i < pixels.length; i += 4) // We go up by four since every four entries describes 1 pixel
{
// pixel is skin if 0 <= (R-G)/(R+G) <= .5 and B/(R+G) <= .5 pixels[i] is the R value, pixels[i+1] is the G value, and pixels[i+2] is the B value.
if ((0 <= ((pixels[i] - pixels[i+1])/(pixels[i] + pixels[i+1]))) && (((pixels[i] - pixels[i+1])/(pixels[i] + pixels[i+1])) <= 0.5) && ((pixels[i+2]/(pixels[i] + pixels[i+1])) <= 0.5))
{
skin_count++;
//window.alert("Found skin pixel."); // used for testing.
if (skin_count >= limit)
{
//window.alert("Blocking image with src: " + image.src); // used for testing.
img.onload = null; // try to clear the onload function
return_value = true;
return false;
} // end inner if
} // end outer if
} // end for loop
//var temp;
img.onload = null;
return_value = false;
return false;
}; // end onload function
img.src = image.src; // Set the new image to the same url as the old one.
return return_value;
} // end image_scanner
I'm not sure what the problem is, but the onload function will run, go through the pixels, set the flag, return, and then run again. I've tried debugging in Chrome's debugger, and that's all I could find. I've tried setting the onload to null inside of the onload function, but it doesn't work. I've tried returning false from the onload function. I've tried waiting in the image_scanner function until return_value != null, but that just seemed to enter an infinite loop and I never even got the alert from the onload function. If anyone has any idea why the onload function will repeatedly execute, I would be very grateful.
If you're going to set .src to the blank image and don't want .onload to get called again when that loads, then you should clear .onload before you set .src.
if (image_scanner(options.scanner_sensitivity, images[i])) {
// clear our onload handler
images[i].onload = function() {};
// Replace the image with a blank white image
images[i].src = chrome.extension.getURL("replacement.png");
}
It also looks like you're returning a value from the onload handler and expecting that value to get returned from the image_scanner function. It doesn't work that way. The onload handler gets called some significant time later, long after image_scanner has already returned. You will need to rewrite your code to work with the asynchronous handling of onload.
Related
I want to compare two svg paths (user and model) at some point. The idea is to transform each of them onto ImageData to be able to make pixel comparisons. The problem I have is using the drawImage which leads me to an empty canvas half of the time.
let modelCanvas = document.createElement("canvas");
let modelContext = modelCanvas.getContext("2d");
modelCanvas.width = 898;
modelCanvas.height = 509;
document.body.appendChild(modelCanvas);
let modelImg = new Image(898, 509);
modelImg.src = 'data:image/svg+xml;base64,PHN2ZyBjbGFzcz0ic3ZnLW[....]';
modelContext.drawImage(modelImg, 0, 0, 898, 509);
The code is pretty straightforward and always run without producing error. Still drawImage seems to fail silently times to times.
Here is the JSFiddle (with the full data string) :
https://jsfiddle.net/Ldgpuo03/
Thank you very much for your help.
Image loading by web browser is an asynchronous operation.
You are trying to call modelContext.drawImage when the image is not guaranteed to be loaded.
You must place your drawing code inside the image.onload callback function
This function will be called once when the image loading is fully finished.
let modelCanvas = document.createElement("canvas");
let modelContext = modelCanvas.getContext("2d");
modelCanvas.width = 40;
modelCanvas.height = 40;
document.body.appendChild(modelCanvas);
let modelImg = new Image();
modelImg.src = 'https://i.stack.imgur.com/EK1my.png?s=48';
modelImg.onload = function(){
modelContext.drawImage(modelImg, 0, 0, 40, 40);
}
Why can't the images be defined in an array as shown here.
Why is it necessary push a new Image object in the array every time?
var canvas = null;
var ctx = null;
var assets = [
'/media/img/gamedev/robowalk/robowalk00.png',
'/media/img/gamedev/robowalk/robowalk01.png',
'/media/img/gamedev/robowalk/robowalk02.png',
'/media/img/gamedev/robowalk/robowalk03.png',
'/media/img/gamedev/robowalk/robowalk04.png',
'/media/img/gamedev/robowalk/robowalk05.png',
'/media/img/gamedev/robowalk/robowalk06.png',
'/media/img/gamedev/robowalk/robowalk07.png',
'/media/img/gamedev/robowalk/robowalk08.png',
'/media/img/gamedev/robowalk/robowalk09.png',
'/media/img/gamedev/robowalk/robowalk10.png',
'/media/img/gamedev/robowalk/robowalk11.png',
'/media/img/gamedev/robowalk/robowalk12.png',
'/media/img/gamedev/robowalk/robowalk13.png',
'/media/img/gamedev/robowalk/robowalk14.png',
'/media/img/gamedev/robowalk/robowalk15.png',
'/media/img/gamedev/robowalk/robowalk16.png',
'/media/img/gamedev/robowalk/robowalk17.png',
'/media/img/gamedev/robowalk/robowalk18.png'
];
var frames = [];
var onImageLoad = function() {
console.log("IMAGE!!!");
};
var setup = function() {
j=0;
body = document.getElementById('body');
canvas = document.createElement('canvas');
ctx = canvas.getContext('2d');
canvas.width = 100;
canvas.height = 100;
body.appendChild(canvas);
for (i = 0; i <= assets.length - 1; ++i) {
frames[i].src = assets[i];
}
setInterval(animate,30);
}
var animate = function() {
ctx.clearRect(0,0,canvas.width,canvas.height);
if (j >= assets.length) {
j=0;
}
var image = new Image();
image.src = frames[j];
ctx.drawImage(image,0,0);
++j;
}
The first reason is to reduce latency. Putting only the URLs into an array means that images have not been pre-fetched before the animation starts. The first round of animation is going to be slow and jerky as each image is retrieved from the net. If the animation is repeated, the next round will be faster. This consideration mostly applies to animations which replaced image elements on the page (in the DOM) rather than by writing to a canvas.
The second reason is to remove overhead and improve efficiency in the animation loop. Using new Image() inside the loop means that drawing time for each frame includes the time taken to create a new Image object as well as draw it on the canvas. In addition the image content can only be written to the canvas after it has been fetched, making it necessary to write to the canvas from an onload handler attached to the image object. The posted code does not do this and could throw an error in some browsers trying to synchronously write an image with no data to the canvas. Even if otherwise successful, repeated animations would be creating a new Image object each time a frame is displayed and churning memory usage.
Note the original version probably used onImageLoad to check when the image has been fully loaded from the web before pushing the object into an array of preloaded image objects. This is the preferred method of prefetching animation images.
And don't forget to define j before use :-)
I'm trying to create a dynamic canvas animation using javascript. The intent is to increment the x property of many image objects and then draw them in the x position of the updated x property each time a draw() function runs. I'm not able to successfully increment this x property though - for some reason it always resets to 0.
I defined this global array to contain all the objects I will draw:
window.character = [];
I have this object constructor to create new image objects:
function Character(name, x, y){
//define the image object within the Character
this.imageObject = new Image();
this.imageObject.src = name+'iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAW0lEQVR42mL8//8/AzpgZGTcC6KBcs5wMRwK/0MVMsLEmLAoEmXAApiwiKUhaRJCltgLsQVsWwIQ/wTx0fBeRigD7B6Y24i1mj4Kn4KI7Uie2Y7FI8+B2AMgwABjRynfWgpcxQAAAABJRU5ErkJggg==';
window.character.push(this);
window.characterPosition = window.character.indexOf(this);
//set natural width and natural height once the image is loaded
if (this.imageObject.addEventListener){
this.imageObject.addEventListener('load', function(){
window.imgWidth = this.naturalWidth/2;
window.imgHeight = this.naturalHeight/2;
//set natural width and natural height to object
window.character[characterPosition]['imageObject']['w'] = window.character[characterPosition]['imageObject']['w0'] = window.imgWidth;
window.character[characterPosition]['imageObject']['h'] = window.character[characterPosition]['imageObject']['h0'] = window.imgHeight;
//set initial x and y position
console.log(x);
window.character[characterPosition]['imageObject']['x'] = x;
window.character[characterPosition]['imageObject']['y'] = y;
console.log(window.character[characterPosition]['imageObject']['x']);
//set loaded property for the object once loading is done
window.character[characterPosition]['imageObject']['loaded'] = true;
function imageLoaded(element, index, array){
return element['imageObject']['loaded'] == true;
}
//test whether every object in array has the image loaded
if(character.every(imageLoaded)){
$('button#play').show();
};
});
} //end object constructor
Inside that constructor function, there is some weird behavior. I created a new object using:
var sun0 = new Character('data:text/javascript;base64,', 12, 12);
Then here's what I see from the console.log messages inside the object constructor:
console.log(x); //returns 12, as expected
window.character[characterPosition]['imageObject']['x'] = x;
window.character[characterPosition]['imageObject']['y'] = y;
console.log(window.character[characterPosition]['imageObject']['x']); //returns 0. Thought it would be 12
Ultimately, here's how I intend to animate the object across the screen. This draw() function runs on an interval every 10ms.
function draw(){
clear();
//draw characters
drawCharacter(window.character[0]['imageObject'],window.character[0]['imageObject']['x'],window.character[0]['imageObject']['y'],window.character[0]['imageObject']['w'],window.character[0]['imageObject']['h']);
window.character[0]['imageObject']['x'] += 10;
console.log(window.character[0]['imageObject']['x']); //returning 0 every time, not incrementing the way I expected it to.
}
How can I get this 'x' property to increment?
here's the JS Fiddle
Pointy answered your question in the comments, but to expand on this... you're trying to set x on an Image which can't be set. For instance, try this in your browser's console:
var img = new Image();
img.x = 1;
console.log(img.x); // this will be zero
That's effectively what you're doing. Additionally, in your code try changing ['x'] to ['myX'] and then it will work as expected.
<img src="circle.png" onclick="alert('clicked')"/>
Let's imagine that circle.png is a 400x400 px transparent background image with a circle in the middle.
What I've got now is that the entire image area (400x400px) is clickable. What I would like the have is that only the circle (non transparent pixels) are clickable.
Of course I know that in this example I could use the <map> tag and a circular area, but I'm looking for a general solution which will take into consideration actual image transparency and work for any kind of images (i.e. non regular shapes).
The most complex way I could see is to trace the contour of the image basing on each pixel alpha, convert to a path (maybe simplify) and apply as a map.
Is there a more efficient / straightforward way to do so?
Using the canvas tag, you can determine the color value of a pixel under a given spot. You can use the event data to determine the coordinates, then check for transparency. All that remains is loading the image up into a canvas.
First, we'll take care of that:
var ctx = document.getElementById('canvas').getContext('2d');
var img = new Image();
img.onload = function(){
ctx.drawImage(img,0,0);
};
img.src = [YOUR_URL_HERE];
This first bit grabs the canvas element, then creates an Image object. When the image loads, it is drawn on the canvas. Pretty straightforward! Except... if the image is not on the same domain as the code, you're up against the same-domain policy security. In order to get the data of our image, we'll need the image to be locally hosted. You can also base64 encode your image, which is beyond the scope of this answer. (see this url for a tool to do so).
Next, attach your click event to the canvas. When that click comes in, we'll check for transparency and act only for non-transparent click regions:
if (isTransparentUnderMouse(this, e))
return;
// do whatever you need to do
alert('will do something!');
The magic happens in the function isTransparentUnderMouse, which needs two arguments: the target canvas element (this in the click handler's scope) and the event data (e, in this example). Now we come to the meat:
var isTransparentUnderMouse = function (target, evnt) {
var l = 0, t = 0;
if (target.offsetParent) {
var ele = target;
do {
l += ele.offsetLeft;
t += ele.offsetTop;
} while (ele = ele.offsetParent);
}
var x = evnt.page.x - l;
var y = evnt.page.y - t;
var imgdata = target.getContext('2d').getImageData(x, y, 1, 1).data;
if (
imgdata[0] == 0 &&
imgdata[1] == 0 &&
imgdata[2] == 0 &&
imgdata[3] == 0
){
return true;
}
return false;
};
First, we do some dancing around to get the precise position of the element in question. We're going to use that information to pass to the canvas element. The getImageData will give us, among other things, a data object that contains the RGBA of the location we specified.
If all those values are 0, then we're looking at transparency. If not, there's some color present. -edit- as noted in the comments, the only value we really need to look at is the last, imgdata[3] in the above example. The values are r(0)g(1)b(2)a(3), and transparency is determined by the a, alpha. You could use this same approach to find any color at any opacity that you know the rgba data for.
Try it out here: http://jsfiddle.net/pJ3MD/1/
(note: in my example, I used a base64 encoded image because of the domain security I mentioned. You can ignore that portion of the code, unless you also intend on using base64 encoding)
Same example, with changes to the mouse cursor thrown in for fun: http://jsfiddle.net/pJ3MD/2/
Documentation
Image object on MDN - https://developer.mozilla.org/en/DOM/Image
Tutorial for using images with canvas on MDN - https://developer.mozilla.org/en/Canvas_tutorial/Using_images
Canvas portal on MDN - https://developer.mozilla.org/en/HTML/Canvas
HTML canvas element on MDN (getContext) - https://developer.mozilla.org/en/DOM/HTMLCanvasElement/
CanvasRenderingContext2D on MDN (getImageData) - https://developer.mozilla.org/en/DOM/CanvasRenderingContext2D
Pixel manipulation on MDN - https://developer.mozilla.org/En/HTML/Canvas/Pixel_manipulation_with_canvas/
You can do this using HTML5 canvas. Draw the image in to the canvas, attach a click handler to the canvas, and in the handler, check if the pixel that was clicked on is transparent.
thank you chris for this great answer
I added a few lines to the code to handle with scaling the canvas
so what I do now is creating a canvas of the exact pixel size the image has that is drawn on it. like (for an Image 220px*120px):
<canvas width="220" height="120" id="mainMenu_item"></canvas>
and scale the canvas using css:
#mainMenu_item{
width:110px;
}
and the adjusted isTransparentUnderMouse function looks like:
var isTransparentUnderMouse = function (target, evnt) {
var l = 0, t = 0;
if (target.offsetParent) {
var ele = target;
do {
l += ele.offsetLeft;
t += ele.offsetTop;
} while (ele = ele.offsetParent);
}
var x = evnt.pageX - l;
var y = evnt.pageY - t;
var initialWidth = $(evnt.target).attr('width');
var clientWidth = evnt.target.clientWidth;
x = x * (initialWidth/clientWidth);
var initialHeight = $(evnt.target).attr('height');;
var clientHeight = evnt.target.clientHeight;
y = y * (initialHeight/clientHeight);
var imgdata = target.getContext('2d').getImageData(x, y, 1, 1).data;
if (
imgdata[0] == 0 &&
imgdata[1] == 0 &&
imgdata[2] == 0 &&
imgdata[3] == 0
){
return true;
}
return false;
};
Once again, completely out of my depth but I need to preload some images and then add them to the page when 'all elements (including xml files etc.)' are loaded. The images and references are stored in an array for later access. Trying to draw and image from that array throws an error yet I know it is available as I can just appendTo the page:
preloadImages: function (loadList, callback) {
var img;
var loadedFiles = [];
var remaining = loadList.length;
$(loadList).each(function(index, address ) {
img = new Image();
img.onload = function() {
--remaining;
if (remaining <= 0) {
callback(loadedFiles);
}
};
img.src = loadList[index];
loadedFiles.push({file: 'name of image to be loaded', image: img }); //Store the image name for later refernce and the image
});
}
//WHEN CERTAIN OTHER CONDITIONS EXIST I CALL THE FUNCTION BELOW
buildScreen: function ( imageLocs, image){
//THIS FUNCTION LOOPS THROUGH imageLocs (XML) AND CREATES CANVAS ELEMENTS, ADDING CLASSES ETC AND DRAWS PART OF A SPRITE (image)
//INTO THE CANVASES CREATED
var ctx = $('ID of CANVAS').get(0).getContext("2d");
var x = 'position x in imageLocs'
var y = 'position y in imageLocs'
var w = 'width in imageLocs'
var h = 'position x in imageLocs'
ctx.drawImage(image, x,y, w, h, 0, 0, w, h); //THIS THROWS AN ERROR 'TypeError: Value could not be converted to any of: HTMLImageElement, HTMLCanvasElement, HTMLVideoElement'
//$(image).appendTo("#innerWrapper") //YET I KNOW THAT IT IS AVAILABE AS THIS LINE ADDS THE IMAGE TO THE PAGE
}
Problem
The issue is caused because you are passing a jQuery object to a native function, in this case ctx.drawImage, drawImage will only support native objects.
startSequence : function(){
$('#innerWrapper').empty();
var screenImageRef = $.grep(ST.imageFilesLoaded, function(e){
return e.file == 'AtlasSheet'
});
var screenImage = $(screenImageRef[0].image);
var imageLocsRef = $.grep(ST.xmlFilesLoaded, function(e){
return e.file == 'IMAGELOCS'
});
var imageLocs = $(imageLocsRef[0].xml);
//$(screenImage).appendTo("#innerWrapper") //appends screenImage
Utilis.buildScreen('1', imageLocs, screenImage, ST.didYouSeeIt, 'ST')
}
Your screenImage var is created by $(screenImageRef[0].image), this will return a jQuery object that wrappers the native image object. To get back to the original native image object use the following:
screenImage.get(0)
or
screenImage[0]
The former is the jQuery supported way.
Solution
So the fix to your code should be either, changing the following line:
Utilis.buildScreen('1', imageLocs, screenImage.get(0), ST.didYouSeeIt, 'ST');
Or changing the line in the buildScreen method:
ctx.drawImage(image.get(0), x,y, w, h, 0, 0, w, h);
... Whichever you prefer.
Confusion when debugging
The reason why everything appears to work when you append the image, is because you are using jQuery to append the image, and jQuery supports being passed jQuery wrapped elements. If you had tried to append your screenImage using native functions i.e. Element.appendChild() you would have got similar errors.
Just to help in future, it's always best to use console.log to find out what type/structure a variable actually has. Using console.log on your previous image var would have given a strange object dump of the jQuery wrapper (which might have rang alarm bells), rather than the expected [object HTMLImageElement] or some other image/console related output (depending on the browser).
I think your image preloader isn't quite correct as it uses the same img variable for all images.
Here is one that I know works well: https://gist.github.com/eikes/3925183