We Keep Coding

Make two instances of d3.forceCollide() play nice together

I want two instances of d3.forceCollide(). In one, every node is pushed away from one another to prevent overlap. In the second, only a subset of nodes are pushed away from one another, with a much bigger radius.
To accomplish the second force, I tweak the initialize method to filter the incoming nodes, like so:
function selective(force,filter){
var init = force.initialize;
force.initialize = function(_){return init(_.filter(filter));};
return force;
}
var dpi = 90; // approximate pixels per inch in SVG
var size = dpi * (1/4); // quarter-inch unit size
var universally_applied =
d3.forceCollide()
.radius(size)
.strength(1);
var selectively_applied =
selective(
d3.forceCollide(),
function(d){return d.id === color;}
)
.radius(size*5)
.strength(1);
}
Now, this ALMOST works. I created a fiddle to see it in action: https://jsfiddle.net/jarrowwx/0dax43ue/38/ - every colored circle is supposed to repel every other circle of the same color, from a distance. Every other color, it just bumps into and pushes it out of the way.
If I do not change the order in which things are defined, then the selectively applied force is ONLY applied to the first color (red). If I shuffle the data array before applying forces, it is difficult to define exactly what happens, but the force is applied to some circles and not most of the others, even among the same color.
Any ideas what is going on here, or how to fix it?

The D3 team decided that this behavior was a bug (Isolating forces to a subset of nodes? #72), and fixed it. The fix was included in version 1.0.4 of d3-force, which is available as part of the full D3 build as of version 4.4.0.
The problem is resolved using the solution suggested by "Partial forces on nodes in D3.js", and the code works as intended now.

What's the idiomatic way to sync transitions so that adjacent shapes move together?

Say I have an arbitrary path like this:
[##########]
I also have a circle like this: o
I want to keep o at the tip of the arbitrary path, so it looks like this:
[##########]o
(Assume o is centered vertically between the top and bottom of the path object) And when the path grows or shrinks, the o should always stay at the tip.
[###############]o
Most importantly, when a transform is applied to the path, the transform should also be applied accordingly to the circle -- they should be in sync when in motion.
I've tried making the circle a path marker, but run into trouble
(a) getting it to only move through the center of the path
(b) getting it to "stick" in the final position
(all examples have it infinitely rotating around the path, like this and this and this)

Calling two transition functions (one for each set of shapes) one after another is usually sufficient, because the time it takes the browser to run through the code is much less than the delay between animation frames.
However, if your animation is sufficiently complex that there is a noticeable lag between the two, or if you are doing a lot of complex calculations that will affect both elements, you could use a custom tween function on one selection, and within it select the other shape and update it (you'll want to select it in your "outer" function, so that your inner function which gets called at every update can just quickly reposition it to match the new value).
Regarding transformations, the easiest way to keep things coordinated is to put both elements in a <g> and transform the group instead of the individual elements.
Putting the ideas together, you could get a transition process something like this:
d3.selectAll("g.groups").transition().delay(time)
.attr("transform", function(d,i){ /* Calculate new transform */ })
.tween("stretch", function(d,i){
/* Select the sub-elements, do all the calculations
then create interpolators for both objects */
var g = d3.select(this);
var path = g.select("path");
var dot = g.select("circle");
var newEndPoint = /*** Calculate final position ***/;
var offset = /*** distance from end point to center of circle ***/;
var pathInterpolator = d3.interpolateString(
path.attr(d),
/*** new path including new end point ***/
);
var dotInterpolator = d3.interpolateObject(
{cx=dot.attr("cx"), cy=dot.attr("cy")},
{cx=newEndPoint.x + offset, cy=newEndPoint.y}
);
return function(t){
/* the function that updates both objects at each tick */
path.attr("d", pathInterpolator(t) );
dot.attr( dotInterpolator(t) );
};
});
How complex your real calculations are will depend on how arbitrary your "arbitrary path" is, of course. Maybe you'll need to calculate both x and y offsets to keep the circle positioned correctly. But that becomes an issue of geometry, not of synchronization. Regardless of what else your path shape does as it transitions, if the end point is an actual point in the path data, it will transition in a direct line, the same as the transition of the circle's coordinates.

How to update transforms on an SVG element in Javascript?

I'm working on a simple visual editor using SVG and Snap.SVG library.
It has a bunch of complex shapes as templates, each shape is a <g> element that contains paths.
The editor lets the user transform shapes by moving, scaling and rotating them.
This is done by applying the transforms as a matrix on the main <g> element of each shape.
The math is mainly done by the Snap.SVG library.
The transforms must by applied in a specific order for things to look right. The user might do any of the transforms in any order they like. So each user interaction with the shape is an update to the transform matrix of that shape.
When the user moves, rotates, scales, rotates again and scales again, what is the best way to update the transform matrix on the shape and keep the transforms in correct order?
Currently, all transforms are done in the order of: translate => rotate => scale. In this case, updating the rotation angle after the scale has already been applied to the matrix would break the shape. Because the matrix is already calculated with the scale taken into account. It would become translate => rotate => scale => rotate and the last rotation would get stretched by the scale transform.
I'm trying to avoid storing any extra data about transforms (e.g in data- attributes) and also avoid using transform strings, such as transform="translate(x, y) rotate(a, x, y) scale(x, y)" as it would be too complicated to parse it.

I can only really advise on what I have done before that has worked.
This is the bits of code from an example I did a while back...(originally from a plugin, but the updateTransform function is the key bit when it comes to orders)
Element.prototype.ftStoreInitialTransformMatrix = function() {
this.data('initialTransformMatrix', this.transform().localMatrix );
return this;
};
Element.prototype.ftGetInitialTransformMatrix = function() {
return this.data('initialTransformMatrix');
};
Element.prototype.ftUpdateTransform = function() {
var tstring = "t" + this.data("tx") + "," + this.data("ty") + this.ftGetInitialTransformMatrix().toTransformString() + "r" + this.data("angle") + 'S' + this.data("scale" );
this.attr({ transform: tstring });
return this;
};
These are the key bits I did from this free transform thing I was playing with here (you can view the source if you want to see it all).
Not sure if that helps at all, fiddling purely with the matrix was hard to keep track. I stored the original transform as well, partly because any imported object may have a transform already applied, but also keeping it as an update as you edit it.

Trace path with DOM object

I'm new to javascript and d3js. I would like a DOM object to trace out a path specified by a parametrized curve (x(t),y(t)). Here is an example of such a parametrization:
var theta = [];
for(var i = 0; i <= N; i++){
theta.push(2*Math.PI*i/N);
}
var points = [];
for(var i = 0; i <= N; i++){
points.push([Math.cos(theta[i]),Math.sin(theta[i])]);
}
The above is the parametrization of a curve -- in this case, also a circle -- and I would like my DOM object to follow the trajectory of this curve. [Aside: is there any better way to define points? It seems ridiculous to run a for loop.]
A crude way to achieve the sort of effect I'm looking for is to run a for loop in the update() part of d3. First, I simply append a circle to the svg variable, so that it need not be linked to any data. It is then selected and updated without required enter/exit.
for (var i = 0; i <= N; i++){
svg.selectAll("circle")
.transition()
.attr("cx",points[i][0]+w/2) // w: width
.attr("cy",points[i][1]+h/2) // h: height
.duration(dt) //
.delay(dt*i);
}
[Aside: I've heard queue() would be better, as opposed to calculating the total delay. Comments?] However, the easing property of the transition makes it run in a choppy fashion. I imagine I could specify no easing, but I'm sure there must be a better way to achieve what I want, which is simply for the initial DOM object (the circle) to move smoothly along a specific trajectory.
In the end, I would want to do this for multiple DOM objects which will eventually be linked to data, each with a specific curve to follow. Any tips on how I would do this?
Thanks in advance for any help, and I will gladly take any advice, including references.

Interesting but not terribly practical approach
The SVG spec actually has a number of animation options, including the ability to move an object along a path. The path is defined in the same form as for a <path> element, so you could use the d3.svg.arc functions to create the path.
Once you have a path defined, it is easy to use d3 to add in the animation:
http://fiddle.jshell.net/RnNsE/1/
although you'll want to read up on SVG animation elements and attributes.
However, there is a limitation to this wonderful animation: poor browser support. So if this is for a website, you're going to need to do the animation with d3 and Javascript.
Production-ready approach
The key to getting d3 to create smooth animations for you is to use a custom "tween" function on a transition.
When you do a transition, d3 initializes a tween function for each change on each element, and starts up timer functions to trigger the updates. At each "tick" of the timer, d3 calls the appropriate "tween" function with the information about how far along the transition it is. So if the tick occurs 500ms into a 2000ms transition, the tween function will given the value 0.25 (assuming a linear easing function, other easing functions complicate the relationship between time elapsed and the expected "distance" along the transition).
Now, for most changes the tween function is fairly straightforward, and d3 will figure one out for you automatically. If you change a "cx" value from 100 to 200, then the tween function is going to return 125 when the transition value 25%, 150 when the transition is 50%, and so on. If you change a "fill" value from red to yellow, it will calculate the numerical values of those colours and convert between them.
The value returned by the tween function at each tick is then used to update the attribute or style of the element. Since the updates happen many times a second, it usually results in a smooth animation. For the simple example of changing the "cx" value of a circle, the circle moves in a straight line from the starting point to the end point.
But you don't want it to move in a straight line. You want it to move in a circle (or along any path you choose). So you're going to need to create a custom function that tells the circle where it should be 25% of the way through the transition, and where it should be 50% through the transition, and so on.
And if you're worried you have to figure that out on your own, never fear. Like so, so much with D3, Mike Bostock has done the hard work for you. But even he didn't have to do the hard hard work. His approach uses two built-in Javascript functions for SVG paths, getTotalLength() and getPointAtLength(). The first tells you the total length of the path, the second gives you the coordinates of the point a certain distance from the start of the path.
With those two values, it is straightforward to figure out the coordinates you should be at if you want to be a certain percent of the way along the path: at 25%, you want to be at path.getPointAtLength(0.25*path.getTotalLength() ).
Here's Mike's function that makes that happen:
// Returns an attrTween for translating along the specified path element.
function translateAlong(path) {
var l = path.getTotalLength();
return function(d, i, a) {
return function(t) {
var p = path.getPointAtLength(t * l);
return "translate(" + p.x + "," + p.y + ")";
};
};
}
A little confusing, no? A function that returns a function that returns a function.
That's because when you specify a "tween" for a transition, what you actually have to specify is a "tween factory" -- the function that will return an appropriate tween function for each element in your selection.
Now, in his example he only has one path and one object moving along it, so those extra layers don't get used. But in the general case, your tween factory function would take the arguments d (the data object for that element in the selection), i (the index of that element) and a (the initial value of the attribute or style that you're changing). With those values, you have to return the custom tween function which take a transition state value t (a number between 0 or 1, or possibly a bit beyond 1 for certain easing functions) and computes the attribute value at that state in the transition.
You'll note that this function returns a translation instruction. That's generally going to be an easier way to move an object around, compared to using cx and cy, since you can specify both horizontal and vertical movement in one transform attribute call, so you only need the one tween function to do both.
Here's my example from above, updated to use a d3 tween to move the circles along the path:
http://fiddle.jshell.net/RnNsE/2/
Key code:
circles.transition().ease("linear")
.duration(5000)
.delay(function(d,i){return i*5000;})
.attrTween("transform", createPathTween);
//creates a tween function to translate an element
//along the path that is a sibling to the element
function createPathTween(d, i, a) {
var path = this.parentNode.getElementsByTagName("path")[0];
//i.e., go from this <circle> -> parent <g> -> array of child <path> elements
-> first (and only) element in that array
var l = path.getTotalLength();
return function(t) {
var p = path.getPointAtLength(t * l);
return "translate(" + p.x + "," + p.y + ")";
};
}
My version strips out the outermost layer of nested functions from Mike's version, but it adds in a bit of Javascript to find the correct <path> element for each circle element.
Note that you need an SVG path element in order to use the getTotalLength() and getPointAtLength() functions; however, this path can be invisible (fill:none; stroke:none; in CSS) if you don't want it to show on the screen. And again, while my path definitions are hard-coded, you could use one of d3's arc or line generators to construct it for you.
And just for fun, here's my example with a different easing function:
http://fiddle.jshell.net/RnNsE/3/
Note that I didn't change anything about the tweening function -- all that's changed is the t values that d3 passes in to that function as the transition progresses.
P.S. Here's another good resource on d3 custom tween functions:
http://blog.safaribooksonline.com/2013/07/11/reusable-d3-js-using-attrtween-transitions-and-mv/

How to draw on an HTML5 Canvas, pixel-by-pixel

Suppose that I have a 900x900 HTML5 Canvas element.
I have a function called computeRow that accepts, as a parameter, the number of a row on the grid and returns an array of 900 numbers. Each number represents a number between 0 and 200. There is an array called colors that contains an array of strings like rgb(0,20,20), for example.
Basically, what I'm saying is that I have a function that tells pixel-by-pixel, what color each pixel in a given row on the canvas is supposed to be. Running this function many times, I can compute a color for every pixel on the canvas.
The process of running computeRow 900 times takes about 0.5 seconds.
However, the drawing of the image takes much longer than that.
What I've done is I've written a function called drawRow that takes an array of 900 numbers as the input and draws them on the canvas. drawRow takes lots longer to run than computeRow! How can I fix this?
drawRow is dead simple. It looks like this:
function drawRow(rowNumber, result /* array */) {
var plot, context, columnNumber, color;
plot = document.getElementById('plot');
context = plot.getContext('2d');
// Iterate over the results for each column in the row, coloring a single pixel on
// the canvas the correct color for each one.
for(columnNumber = 0; columnNumber < width; columnNumber++) {
color = colors[result[columnNumber]];
context.fillStyle = color;
context.fillRect(columnNumber, rowNumber, 1, 1);
}
}

I'm not sure exactly what you are trying to do, so I apologize if I am wrong.
If you are trying to write a color to each pixel on the canvas, this is how you would do it:
var ctx = document.getElementById('plot').getContext('2d');
var imgdata = ctx.getImageData(0,0, 640, 480);
var imgdatalen = imgdata.data.length;
for(var i=0;i<imgdatalen/4;i++){ //iterate over every pixel in the canvas
imgdata.data[4*i] = 255; // RED (0-255)
imgdata.data[4*i+1] = 0; // GREEN (0-255)
imgdata.data[4*i+2] = 0; // BLUE (0-255)
imgdata.data[4*i+3] = 255; // APLHA (0-255)
}
ctx.putImageData(imgdata,0,0);
This is a lot faster than drawing a rectangle for every pixel. The only thing you would need to do is separate you color into rgba() values.

If you read the color values as strings from an array for each pixel it does not really matter what technique you use as the bottleneck would be that part right there.
For each pixel the cost is split on (roughly) these steps:
Look up array (really a node/linked list in JavaScript)
Get string
Pass string to fillStyle
Parse string (internally) into color value
Ready to draw a single pixel
These are very costly operations performance-wise. To get it more efficient you need to convert that color array into something else than an array with strings ahead of the drawing operations.
You can do this several ways:
If the array comes from a server try to format the array as a blob / typed array instead before sending it. This way you can copy the content of the returned array almost as-is to the canvas' pixel buffer.
Use a web workers to parse the array and pass it back as a transferable object which you them copy into the canvas' buffer. This can be copied directly to the canvas - or do it the other way around, transfer the pixel buffer to worker, fill there and return.
Sort the array by color values and update the colors by color groups. This way you can use fillStyle or calculate the color into an Uint32 value which you copy to the canvas using a Uint32 buffer view. This does not work well if the colors are very spread but works ok if the colors represent a small palette.
If you're stuck with the format of the colors then the second option is what I would recommend primarily depending on the size. It makes your code asynchronous so this is an aspect you need to deal with as well (ie. callbacks when operations are done).
You can of course just parse the array on the same thread and find a way to camouflage it a bit for the user in case it creates a noticeable delay (900x900 shouldn't be that big of a deal even for a slower computer).
If you convert the array convert it into unsigned 32 bit values and store the result in a Typed Array. This way you can iterate your canvas pixel buffer using Uint32's instead which is much faster than using byte-per-byte approach.

fillRect is meant to be used for just that - filling an area with a single color, not pixel by pixel. If you do pixel by pixel, it is bound to be slower as you are CPU bound. You can check it by observing the CPU load in these cases. The code will become more performant if
A separate image is created with the required image data filled in. You can use a worker thread to fill this image in the background. An example of using worker threads is available in the blog post at http://gpupowered.org/node/11
Then, blit the image into the 2d context you want using context.drawImage(image, dx, dy).