I have a 3D matrix of size (X, Y, Z) which is stored in a data structure as Z matrices, each X x Y in size. I would like to re-slice these matrices to obtain X slices, each Y x Z in size. In other words, I want to reslice a 3D matrix stored as XY slices in the YZ plane. The use case is to reslice axial CT images into sagittal images. I am working inside a browser environment.
Here's an example of what I am trying to achieve:
I have implemented the naive (iterative) solution in Python, which takes O(Y * Z) per slice. I haven't even bothered writing out the corresponding JavaScript implementation, because this approach is too slow by several orders of magnitude.
import glob
import numpy as np
import matplotlib.pyplot as plt
from scipy.misc import imread
height, width, depth = 512, 512, 100
volume = np.zeros((height, width, depth))
s = 0
for filename in glob.iglob('./*.jpg'):
volume[:,:,s] = imread(filename)[...,0]/255.0
s += 1
reslice = np.zeros((depth, height, width))
for s in xrange(0, width):
current = np.zeros((depth, height))
for i in xrange(0, height):
for j in xrange(0, depth):
current[j,i] = volume[i,s,j]
reslice[:,:,s] = current
This algorithm seems to be amenable to parallelization. For example, in CUDA, one could load the 3D data into global memory, create one thread per pixel, then iterate for every slice in the new direction, and on each iteration ask the right pixels to fire in order to fill out the current slice. This would be a trivial kernel to write, and would be approximately O(1) per slice. However, I don't have access to CUDA in the browser.
Mapping from CUDA to WebCL is relatively straightforward, but WebCL is out of question given inexistent vendor support ATM. Therefore, I'm thinking WebGL is the ideal solution.
I'm not too sure how this would be done in the "WebGL" paradigm, but I'm sure it can be done, and I suspect it is fairly trivial as well. I can't seem to find where to start, however, and resources on doing general-purpose computations with OpenGL are extremely scarce. How would I go about using OpenGL to speed up reslicing of a 3D matrix inside the browser?
You don't have necessarily to use webGL to be fast enough.
If you use a 3D array, JavaScript might be too slow but by using a flat array the time to reslice is in fact similar to the time it takes to create the array!
Another trick is to use a typed array to reduce memory usage and improve performances (Uint8Array).
I created a small class to handle such a flat array and to slice it.
I think the most relevant thing you want is in fact to get a view, either on (x, y) axes or (y, z) axes.
Since Array creation is very slow, you want to build the view on place within a fixed buffer. And since you want also a sliced view, you have to create a buffer and method also for the sliced view.
It's fast: creating a view for your 512X512x100 example take less than 5 ms!
(So in fact, the putImageData you'll have to do afterward will quite take more time! )
Fiddle is here: http://jsfiddle.net/n38mwh95/1/
Here's the class handling the data, you'll have to change the constructor so it accepts the real raw data:
function Array3D(xSize, ySize, zSize) {
this.xSize = xSize;
this.ySize = ySize;
this.zSize = zSize;
var xyMultiplier = xSize * ySize;
this.array = new Uint8Array(xSize * ySize * zSize);
this.view = new Uint8Array(xSize * ySize);
this.slicedView = new Uint8Array(ySize * zSize);
this.valueAt = function (x, y, z) {
return this.array[x + xSize * (y + z * ySize)];
};
this.setValueAt = function (x, y, z, val) {
return this.array[x + xSize * (y + z * ySize)] = val;
};
this.buildView = function (z) {
var src = this.array;
var view = this.view;
for (var x = 0; x < xSize; x++) {
for (var y = 0; y < ySize; y++) {
view[x + xSize * y] = src[x + xSize * (y + z * ySize)];
}
}
return view;
};
this.buildSlicedView = function (x) {
var src = this.array;
var sView = this.slicedView;
for (var y = 0; y < ySize; y++) {
for (var z = 0; z < zSize; z++) {
sView[y + ySize * z] = src[x + xSize * (y + z * ySize)];
}
}
return sView;
};
}
In use:
var xSize = 512;
var ySize = 512;
var zSize = 100;
var t1, t2;
t1 = performance.now();
var testArray = new Array3D(xSize, ySize, zSize);
t2 = performance.now();
console.log('created in :' + (t2 - t1));
t1 = performance.now();
var resliced = testArray.buildView(10);
t2 = performance.now();
console.log('building view in :' + (t2 - t1));
var x = 80;
t1 = performance.now();
var resliced = testArray.buildSlicedView(x);
t2 = performance.now();
console.log('building sliced view in :' + (t2 - t1));
Results:
created in :33.92199998779688 (index):73
building view in :2.7559999871300533 (index):79
building sliced view in :5.726000003051013
At the end of the code I also added some code to render the view.
Don't forget to cache the canvas imageData: create it only once then re-use it for best performance.
You could easily have a real-time rendering in fact.
Related
Im creating an object that randomly moves in a natural way using noise like this (works as intended):
The objects encounter a collision and their trajectory is manipulated, the movement path now changes to straight line (words as intended)
thisRabbit.x = _world.width * (noise(thisRabbit.t));
thisRabbit.y = _world.height * (noise(thisRabbit.t+5));
thisRabbit.t += 0.001;
The problem is after this movement , i want the object to start moving in a random direction again as it was initially. If i use the same function, the object jumps to the last location before the trajectory was modified.
let vx = this.acquiredFood[0] - this.x;
let vy = this.acquiredFood[1] - this.y;
let f = (this.genes.speed + 10) / Math.sqrt(vx*vx+vy*vy);
vx = vx * f;
vy = vy * f;
let newX = this.x + vx;
let newY = this.y + vy;
So how do i get the object to move as before, given a starting position
edit: snippet here: https://editor.p5js.org/vince.chinner/sketches/HPFKR8eIw
Your problem is that you used a factor from 0 to 1 generated with noise and an incremented seed to generate the position by multiplying directly the world dimentions. When reaching food, you cannot increment the seed as to be in the exact position where the movement to get your food led you (I found no inverse function for noise to get the seed from the return value).
What you need to do instead is use the noise to increment or decrement the coordinates, so that no matter where the seed is, you don't loose your current position.
Here are the different corrections I applied to the code, as there were also syntax errors, I can't really paste the whole stuff here for copyright reasons (you didn't share the whole code here and the sketch belongs to you)
MAIN CORRECTION:
used a var found because returning from the forEach callback doesn't make you leave the findFood function, but the callback one. And the forEach loop doesn't stop. Using this var prevents the further forEach tests to be made and allows you to return from findFood so that no further move is made after seeing food.
noise is now applied to a value of 4 and I subtract 2, so that x and y now change with a range of -2 to 2 each. Of course, with this method, you need to check against world dimentions or else the rabbit could leave the world. The seed increment has been changed too or else it would vary too slowly (adapt values as you wish)
findFood(){
var thisRabbit = this, found = false;
_world.food.forEach(f => {
if(!found){
let d = int(dist(f[0], f[1], thisRabbit.x, thisRabbit.y));
if(d < (thisRabbit.genes.vision / 2)+3){
thisRabbit.state = "foundFood";
this.acquiredFood = f;
found = true;
}
}
});
if(found){ return; }
thisRabbit.x += (noise(thisRabbit.t) * 4) - 2;
if(thisRabbit.x < 0){ thisRabbit.x = 0; }
if(thisRabbit.x > _world.width){ thisRabbit.x = _world.width; }
thisRabbit.y += (noise(thisRabbit.t + 5) * 4) - 2;
if(thisRabbit.y < 0){ thisRabbit.y = 0; }
if(thisRabbit.y > _world.height){ thisRabbit.y = _world.height; }
thisRabbit.t += 0.01;
}
SYNTAX ERRORS:
lines 23 / 24: assignment should be with a value (null or false)
this.genes = null;
this.acquiredFood = null;
lines 129 to 133: end you instructions with a ; instead of a ,
this.width = w;
this.height = h;
this.foodDensity = foodDensity;
this.food = [];
this.rabits = [];
line 156 to 160: there should be no space between rabbit and .t. Additionnally, because the coordinates are not directly linked to t, I would prefer to use random for starting position:
let x = this.width * random();
let y = this.height * random();
let _rabbit = new rabbit(x, y);
_rabbit.genes = genes;
_rabbit.t = t;
I'm working on a simple web app which simplifies the colours of an uploaded image to a colour palette selected by the user. The script works, but it takes a really long time to loop through the whole image (for large images it's over a few minutes), changing the pixels.
Initially, I was writing to the canvas itself, but I changed the code so that changes are made to an ImageData object and the canvas is only updated at the end of the script. However, this didn't really make much difference.
// User selects colours:
colours = [[255,45,0], [37,36,32], [110,110,105], [18,96,4]];
function colourDiff(colour1, colour2) {
difference = 0
difference += Math.abs(colour1[0] - colour2[0]);
difference += Math.abs(colour1[1] - colour2[1]);
difference += Math.abs(colour1[2] - colour2[2]);
return(difference);
}
function getPixel(imgData, index) {
return(imgData.data.slice(index*4, index*4+4));
}
function setPixel(imgData, index, pixelData) {
imgData.data.set(pixelData, index*4);
}
data = ctx.getImageData(0,0,canvas.width,canvas.height);
for(i=0; i<(canvas.width*canvas.height); i++) {
pixel = getPixel(data, i);
lowestDiff = 1024;
lowestColour = [0,0,0];
for(colour in colours) {
colour = colours[colour];
difference = colourDiff(colour, pixel);
if(lowestDiff < difference) {
continue;
}
lowestDiff = difference;
lowestColour = colour;
}
console.log(i);
setPixel(data, i, lowestColour);
}
ctx.putImageData(data, 0, 0);
During the entire process, the website is completely frozen, so I can't even display a progress bar. Is there any way to optimise this so that it takes less time?
There is no need to slice the array each iteration. (As niklas has already stated).
I would loop over the data array instead of looping over the canvas dimensions and directly edit the array.
for(let i = 0; i < data.length; i+=4) { // i+=4 to step over each r,g,b,a pixel
let pixel = getPixel(data, i);
...
setPixel(data, i, lowestColour);
}
function setPixel(data, i, colour) {
data[i] = colour[0];
data[i+1] = colour[1];
data[i+2] = colour[2];
}
function getPixel(data, i) {
return [data[i], data[i+1], data[i+2]];
}
Also, console.log can bring a browser to it's knees if you've got the console open. If your image is 1920 x 1080 then you will be logging to the console 2,073,600 times.
You can also pass all of the processing off to a Web Worker for ultimate threaded performance. Eg. https://jsfiddle.net/pnmz75xa/
One problem or option for improvement is clearly your slice function, which will create a new array every time it is called, you do not need this. I would change the for loop like so:
for y in canvas.height {
for x in canvas.width {
//directly alter the canvas' pixels
}
}
Finding difference in color
I am adding an answer because you have use a very poor color match algorithm.
Finding how closely a color matches another is best done if you imagine each unique possible colour as a point in 3D space. The red, green, and blue values represent the x,y,z coordinate.
You can then use some basic geometry to locate the distance from one colour to the another.
// the two colours as bytes 0-255
const colorDist = (r1, g1, b1, r2, g2, b2) => Math.hypot(r1 - r2, g1 - g2, b1 - b2);
It is also important to note that the channel value 0-255 is a compressed value, the actual intensity is close to that value squared (channelValue ** 2.2). That means that red = 255 is 65025 times more intense than red = 1
The following function is a close approximation of the colour difference between two colors. Avoiding the Math.hypot function as it is very slow.
const pallet = [[1,2,3],[2,10,30]]; // Array of arrays representing rgb byte
// of the colors you are matching
function findClosest(r,g,b) {
var closest;
var dist = Infinity;
r *= r;
g *= g;
b *= b;
for (const col of pallet) {
const d = ((r - col[0] * col[0]) + (g - col[1] * col[1]) + (b - col[2] * col[2])) ** 0.5;
if (d < dist) {
if (d === 0) { // if same then return result
return col;
}
closest = col;
dist = d;
}
}
return closest;
}
As for performance, your best bet is either via a web worker, or use webGL to do the conversion in realtime.
If you want to keep it simple to prevent the code from blocking the page cut the job into smaller slices using a timer to allow the page breathing room.
The example uses setTimeout and performance.now() to do 10ms slices letting other page events and rendering to do there thing. It returns a promise that resolves when all pixels are processed
function convertBitmap(canvas, maxTime) { // maxTime in ms (1/1000 second)
return new Promise(allDone => {
const ctx = canvas.getContext("2d");
const pixels = ctx.getImageData(0, 0, canvas.width, canvas.height);
const data = pixels.data;
var idx = data.length / 4;
processPixels(); // start processing
function processPixels() {
const time = performance.now();
while (idx-- > 0) {
if (idx % 1024) { // check time every 1024 pixels
if (performance.now() - time > maxTime) {
setTimeout(processPixels, 0);
idx++;
return;
}
}
let i = idx * 4;
const col = findClosest(data[i], data[i + 1], data[i + 2]);
data[i++] = col[0];
data[i++] = col[1];
data[i] = col[2];
}
ctx.putImageData(pixels, 0, 0);
allDone("Pixels processed");
}
});
}
// process pixels in 10ms slices.
convertBitmap(myCanvas, 10).then(mess => console.log(mess));
Question: What in the world is this piece of code doing?
Also: Is the way 'w' is being used some sort of existing algorithm? I'm trying to figure out the intent of the function, or at least describe what sorts of numbers it produces.
Context: I'm looking at Martin O'Leary's "Fantasy Map Generation" code - full source here, which in short summary generates fantasy maps on the canvas. There is some insightful explanations of how the higher level process works in a blog post, but this is too low level to get any coverage there. There is a particular function called 'rnorm' that gets used in a couple of places, and I'm lost at how it works. I've included it below, followed by a couple of instances where it comes up for some context. Any help on what this thing is doing would be great!
var rnorm = (function() {
var z2 = null;
function rnorm() {
if (z2 != null) {
var tmp = z2;
z2 = null;
return tmp;
}
var x1 = 0;
var x2 = 0;
var w = 2.0;
while (w >= 1) {
x1 = runif(-1, 1);
x2 = runif(-1, 1);
w = x1 * x1 + x2 * x2;
}
w = Math.sqrt(-2 * Math.log(w) / w);
z2 = x2 * w;
return x1 * w;
}
return rnorm;
})();
runif(), which is called in the code above, is a short function that generates a random number between two given values
function runif(lo, hi) {
return lo + Math.random() * (hi - lo);
}
This code is used to produce random vectors (actually the only place it's used during the generation process) -
function randomVector(scale) {
return [scale * rnorm(), scale * rnorm()];
}
But I think it's doing more than that because the following, when provided a direction of 'randomVector(4),' produces a gradual slope over the entire mesh heightmap: EDIT: no, it actually is having no effect on the gradual slope. That comes from some sneakyness using the fact that one side of the map is 0,0, and the other side of the map is width,height, which creates numbers that gradually increase.
function slope(mesh, direction) {
return mesh.map(function (x) {
return x[0] * direction[0] + x[1] * direction[1];
});
}
Let me know if there's anything else I should be providing. This is my first question here, so I may be a little soft on conventions.
I think it's horrible code. It appears to create a pair of values, z1 and z2, but instead of putting them in a tuple and returning that it returns z1 and on every second call the corresponding z2 value. I have no idea why they'd do such a thing, my only guess would be to avoid allocation of objects and make usage syntactically more convenient.
It should be simplified to
function rnorm() {
var x1 = 0;
var x2 = 0;
var w = 2.0;
while (w >= 1) {
x1 = runif(-1, 1);
x2 = runif(-1, 1);
w = x1 * x1 + x2 * x2;
}
w = Math.sqrt(-2 * Math.log(w) / w);
return [x1 * w, x2 * w];
}
function randomVector(scale) {
var [z1, z2] = rnorm();
return [scale * z1, scale * z2];
}
A modern compiler should be able to avoid array allocation for the returned literal and subsequent destructuring. If it's not, you can do it manually by inlining rnorm in randomVector, especially if this is the only place where it's called anyway.
The problem is currently solved. In case some one wants to see the colored fractal, the code is here.
Here is the previous problem:
Nonetheless the algorithm is straight forward, I seems to have a small error (some fractals are drawing correctly and some are not). You can quickly check it in jsFiddle that c = -1, 1/4 the fractal is drawing correctly but if I will take c = i; the image is totally wrong.
Here is implementation.
HTML
<canvas id="a" width="400" height="400"></canvas>
JS
function point(pos, canvas){
canvas.fillRect(pos[0], pos[1], 1, 1); // there is no drawpoint in JS, so I simulate it
}
function conversion(x, y, width, R){ // transformation from canvas coordinates to XY plane
var m = R / width;
var x1 = m * (2 * x - width);
var y2 = m * (width - 2 * y);
return [x1, y2];
}
function f(z, c){ // calculate the value of the function with complex arguments.
return [z[0]*z[0] - z[1] * z[1] + c[0], 2 * z[0] * z[1] + c[1]];
}
function abs(z){ // absolute value of a complex number
return Math.sqrt(z[0]*z[0] + z[1]*z[1]);
}
function init(){
var length = 400,
width = 400,
c = [-1, 0], // all complex number are in the form of [x, y] which means x + i*y
maxIterate = 100,
R = (1 + Math.sqrt(1+4*abs(c))) / 2,
z;
var canvas = document.getElementById('a').getContext("2d");
var flag;
for (var x = 0; x < width; x++){
for (var y = 0; y < length; y++){ // for every point in the canvas plane
flag = true;
z = conversion(x, y, width, R); // convert it to XY plane
for (var i = 0; i < maxIterate; i++){ // I know I can change it to while and remove this flag.
z = f(z, c);
if (abs(z) > R){ // if during every one of the iterations we have value bigger then R, do not draw this point.
flag = false;
break;
}
}
// if the
if (flag) point([x, y], canvas);
}
}
}
Also it took me few minutes to write it, I spent much more time trying to find why does not it work for all the cases. Any idea where I screwed up?
Good news! (or bad news)
You're implementation is completely. correct. Unfortunately, with c = [0, 1], the Julia set has very few points. I believe it is measure zero (unlike say, the Mandelbrot set). So the probability of a random point being in that Julia set is 0.
If you reduce your iterations to 15 (JSFiddle), you can see the fractal. One hundred iterations is more "accurate", but as the number of iterations increase, the chance that a point on your 400 x 400 grid will be included in your fractal approximation decreases to zero.
Often, you will see the Julia fractal will multiple colors, where the color indicates how quickly it diverges (or does not diverge at all), like in this Flash demonstration. This allows the Julia fractal to be somewhat visible even in cases like c = i.
Your choices are
(1) Reduce your # of iterations, possibly depending on c.
(2) Increase the size of your sampling (and your canvas), possibly depending on c.
(3) Color the points of your canvas according to the iteration # at which R was exceeded.
The last option will give you the most robust result.
I have a "bubble generator" that is mostly working, but is not properly clearing the bubbles and I can't figure out why. Been staring at this for a while now. Specifically, some of the bubbles are getting "cleared" as they float up, others aren't, and I can't see why. ARGH!
http://jsfiddle.net/Dud2q/7/ (slowed waaay down so that you can easily watch a single bubble)
Logic flow (this just describes the code in the fiddle):
Create an imageData array (long list of pixels)
imgData = ctx.getImageData(0, 0, w, h);
push new random bubbles onto the beginning of the "bubbles array" which draws bottom-up:
for(var i=0, l=generators.length; i<l; i++){
for(var j=0, m=0|Math.random()*6; j<m; j++){
newBubbles.push( 0|generators[i] + j );
}
generators[i] = Math.max(0, Math.min(w, generators[i] + Math.random()*10 - 5));
}
bubbles.unshift(newBubbles);
loop all bubbles to be drawn and:
clear the bubbles that were drawn in the last loop by setting alpha channel to 0 (transparent):
if(i<(l-1)){
x = 0|bubbles[i+1][j];
offset = y * w * 4 + x * 4;
pixels[offset+3] = 0;
}
draw new bubbles (offset+1 = g, offset+2 = b, offset+3 = alpha):
x = 0|(bubbles[i][j] += Math.random() * 6 - 3);
offset = y * w * 4 + x * 4;
pixels[offset+1] = 0x66;
pixels[offset+2] = 0x99;
pixels[offset+3] = 0xFF;
Increasing the number of generators to a higher number seems to make it work.
Eg. 50..times(createBubbleGenerator); almost works.
Cheers!!!