We Keep Coding

Using d3.js, how can I display data faster on my chart?

In my code, I am loading a JSON with 508 entries on a line chart. This JSON contains data emitted by some machines, and the keys are the names of the machines.
This is the structure of my JSON:
{
"AF3":3605.1496928113393,
"AF4":-6000.4375230516,
"F3":1700.3827875419374,
"F4":4822.544985821321,
"F7":4903.330735023786,
"F8":824.4048714773611,
"FC5":3259.4071092472655,
"FC6":4248.067359141752,
"O1":3714.5106599153364,
"O2":697.2904723891061,
"P7":522.7300768483767,
"P8":4050.79490288753,
"T7":2939.896657485737,
"T8":9.551935316881588
}
I am currently reading the data with the help of a counter called cont, however, the code that I'm using takes too long to draw the graph:
data.length=508
if (data.length>cont)
cont++`
for (var name in groups) {
var group = groups[name]
group.data.push(aData[cont][name])
group.path.attr('d', line)
console.log(cont)
}
As you can see in the gif above, my code is taking too long to plot all the data points. I would like to draw all the data elements of my data set (in this case 508) without delay, for example:
data=[{508 elements}];
tick(data)=> draw the points in the graph at the same time, by dataset.
data2=[{50 elements}];
tick(data)=> draw the points in the graph at the same time, by dataset.
Where tick is the name of the function that would draw the coordinates, without losing the sense of animation.
How can do it?
Here is a link to my code:
http://plnkr.co/edit/y8h9zs1CpLU1BZRoWZi4?p=preview

It seems to me that your problem is the fact that the graph is synchronous - "duration" is both used for animation and for graph shifting. Essentially, changing duration will avail nothing.
You can introduce a time multiplier. Then try dividing duration by two, and using a multiplier of 2. Your actual data duration is now duration*timeMultiplier (you might want to change the names to make it less confusing, or use a timeDivider in the animation).
// Shift domain
x.domain([now - (limit - 2) * duration * timeMultiplier, now - duration * timeMultiplier])
// Slide x-axis left
axis.transition()
.duration(duration)
.ease('linear')
.call(x.axis);
// Slide paths left
var t = paths.attr('transform', null)
.transition()
.duration(duration)
.ease('linear')
t.attr('transform', 'translate(' + x(now - (limit - 1) * duration * timeMultiplier) + ')')
.each('end', tick)
Another thing you might try is to add the points two at a time, i.e. you skip the shift on odd ticks, and shift double the amount on even ticks. This reduces the overhead at the expense of making the animation a bit jaggier (but not very much, because it also plays faster).

Angular-Chart-JS - Line chart with different fill colors according to points' range

I'm using Angular-Chart-js for my website to display some types of graphs, one of them is a line chart.
I would like the line chart to be color-filled, but with different colors according to the y-axis' value. Like in this photo:
I've tried to have different data arrays in the "data" array of the graph, the first one has all the values, the second one has all but the ones painted in green (on the right), the third is the same array only until the purple range etc. and then have for each dataset its own color, but eventually I get a graph with a single color according to the last dataset color.
What am I missing? Is there any way to accomplish that?
Thanks.

Unfortunately, you cannot achieve this with the current chart.js configuration options. The reason is because the line chart backgroundColor option (the option that controls the color of the line chart fill) only accepts a single value.
After digging through the current chart.js 2.5 source, I found that it is possible to extend the line element's draw() method and force chart.js to use a canvas linear gradient for the fill (instead of just a single color). With a little bit of math, we can convert the x position of each point into a linear gradient color stop position and build a gradient.
With this enhancement, you can now pass in an array of colors to the line chart backgroundColor option to achieve varying colored fill regions. Here is an example of what a resulting chart would look like.
Here is how to actually do it (with a working example at the bottom)
First, we must extend Chart.elements.Line and overwrite it's draw() method so that we can build the linear gradient based upon the position of each point, use it as the line fill, and then draw the line.
// save the original line element so we can still call it's
// draw method after we build the linear gradient
var origLineElement = Chart.elements.Line;
// define a new line draw method so that we can build a linear gradient
// based on the position of each point
Chart.elements.Line = Chart.Element.extend({
draw: function() {
var vm = this._view;
var backgroundColors = this._chart.controller.data.datasets[this._datasetIndex].backgroundColor;
var points = this._children;
var ctx = this._chart.ctx;
var minX = points[0]._model.x;
var maxX = points[points.length - 1]._model.x;
var linearGradient = ctx.createLinearGradient(minX, 0, maxX, 0);
// iterate over each point to build the gradient
points.forEach(function(point, i) {
// `addColorStop` expects a number between 0 and 1, so we
// have to normalize the x position of each point between 0 and 1
// and round to make sure the positioning isn't too percise
// (otherwise it won't line up with the point position)
var colorStopPosition = roundNumber((point._model.x - minX) / (maxX - minX), 2);
// special case for the first color stop
if (i === 0) {
linearGradient.addColorStop(0, backgroundColors[i]);
} else {
// only add a color stop if the color is different
if (backgroundColors[i] !== backgroundColors[i-1]) {
// add a color stop for the prev color and for the new color at the same location
// this gives a solid color gradient instead of a gradient that fades to the next color
linearGradient.addColorStop(colorStopPosition, backgroundColors[i - 1]);
linearGradient.addColorStop(colorStopPosition, backgroundColors[i]);
}
}
});
// save the linear gradient in background color property
// since this is what is used for ctx.fillStyle when the fill is rendered
vm.backgroundColor = linearGradient;
// now draw the lines (using the original draw method)
origLineElement.prototype.draw.apply(this);
}
});
Then, we have to also extend the line chart to ensure that the line element used by the chart is the one that we extended above (since this property is already set at load time)
// we have to overwrite the datasetElementType property in the line controller
// because it is set before we can extend the line element (this ensures that
// the line element used by the chart is the one that we extended above)
Chart.controllers.line = Chart.controllers.line.extend({
datasetElementType: Chart.elements.Line,
});
With this done, we can now pass in an array of colors to the line chart backgroundColor property (instead of just a single value) to control the line fill.
Here is a codepen example that demonstrates all that has been discussed.
Caveats:
This approach could break in future chart.js releases since we are messing with the internals.
I'm not familiar with angular-chart.js, so I cannot provide insight on how to integrate the above chart.js changes into the angular directive.

If you would like to have this capability with angular2 and ng2-charts there maybe a less "hacked" way to do this but this is how I was able to apply Jordan's code to make it work:
Downgrade ng2-chart's dependency on Chart.js from 2.7.x to 2.5.
- From your project's directory: npm install chart.js#2.5 --save
Inside node_modules/chart.js/src/charts:
- Add Jordan's code to Chart.line.js( inside the export ) after the Chart.line function
Rebuild Chart.js/dist:
- run npm install
run gulp build
If you get an error from socket.io code, then you will need to upgrade those dependencies to a more current version of socket.io, I believe Karma might have an old version of socket.io that you could upgrade to 2.0.
Anyway this worked for me. It is not fully tested and it is definitely a "hack" but I did not want to learn Chart.js 2.7 to figure out why Jordan's code would not work with it. Which is definitely the more "proper" way to do it. I suppose it should be integrated as a "plugin".

i decided to do the chartJS 2.5 approach but use the extension above vs modifying the chartjs code itself..
i have to work on some performance optimization, as my charts have over 4000 values.
getting the color array built with the right values (sparse alternate color, maybe for 200 in 4000 values) and then having the extension read it to build the linear gradient is very time consuming. buries the raspberry PI I am using for the chart display system.
I finally decided that to reduce the processing time, I needed to eliminate any extra processing of the list of points.. mine collect, mine creating the color array , and chart building the linear grandient...
so, now I create the linearGradient edges as I go thru the data (all in one pass).. the gradient is an array of structures, that have offset from start of data array, and the color to be applied to that edge, basically does what the original extension does.. so, reduce 800 points to 40 edges. or 800 points to 1 edge (start)...
so, here is my updated extend function.. my app has charts with all three color types,. single fixed, array of colors and the array of edges.. all the other routines above are unchanged
// save the original line element so we can still call it's
// draw method after we build the linear gradient
var origLineElement = Chart.elements.Line;
// define a new line draw method so that we can build a linear gradient
// based on the position of each point
Chart.elements.Line = Chart.Element.extend({
draw: function() {
var vm = this._view;
var backgroundColors = this._chart.controller.data.datasets[this._datasetIndex].backgroundColor;
var points = this._children;
var ctx = this._chart.ctx;
var minX = points[0]._model.x;
var maxX = points[points.length - 1]._model.x;
var linearGradient = ctx.createLinearGradient(minX, 0, maxX, 0);
// if not a single color
if( typeof backgroundColors != 'string' ){
// but is array of colors
if( typeof backgroundColors[0] === 'string' ) {
// iterate over each point to build the gradient
points.forEach(function(point, i) { // start original code
// `addColorStop` expects a number between 0 and 1, so we
// have to normalize the x position of each point between 0 and 1
// and round to make sure the positioning isn't too percise
// (otherwise it won't line up with the point position)
var colorStopPosition = self.roundNumber((point._model.x - minX) / (maxX - minX), 2);
// special case for the first color stop
if (i === 0) {
linearGradient.addColorStop(0, backgroundColors[i]);
} else {
// only add a color stop if the color is different
if ( backgroundColors[i] !== backgroundColors[i-1]) {
// add a color stop for the prev color and for the new color at the same location
// this gives a solid color gradient instead of a gradient that fades to the next color
linearGradient.addColorStop(colorStopPosition, backgroundColors[i - 1]);
linearGradient.addColorStop(colorStopPosition, backgroundColors[i]);
}
}
}); // end original code
} // end of if for color array
// must be a gradient fence position list
else {
// loop thru the fence positions
backgroundColors.forEach(function(fencePosition){
var colorStopPosition = self.roundNumber(fencePosition.offset / points.length, 2);
linearGradient.addColorStop(colorStopPosition,fencePosition.color)
});
} // end of block for handling color space edges
// save the linear gradient in background color property
// since this is what is used for ctx.fillStyle when the fill is rendered
vm.backgroundColor = linearGradient;
} // end of if for just one color
// now draw the lines (using the original draw method)
origLineElement.prototype.draw.apply(this);
}

What's the idiomatic way to sync transitions so that adjacent shapes move together?

Say I have an arbitrary path like this:
[##########]
I also have a circle like this: o
I want to keep o at the tip of the arbitrary path, so it looks like this:
[##########]o
(Assume o is centered vertically between the top and bottom of the path object) And when the path grows or shrinks, the o should always stay at the tip.
[###############]o
Most importantly, when a transform is applied to the path, the transform should also be applied accordingly to the circle -- they should be in sync when in motion.
I've tried making the circle a path marker, but run into trouble
(a) getting it to only move through the center of the path
(b) getting it to "stick" in the final position
(all examples have it infinitely rotating around the path, like this and this and this)

Calling two transition functions (one for each set of shapes) one after another is usually sufficient, because the time it takes the browser to run through the code is much less than the delay between animation frames.
However, if your animation is sufficiently complex that there is a noticeable lag between the two, or if you are doing a lot of complex calculations that will affect both elements, you could use a custom tween function on one selection, and within it select the other shape and update it (you'll want to select it in your "outer" function, so that your inner function which gets called at every update can just quickly reposition it to match the new value).
Regarding transformations, the easiest way to keep things coordinated is to put both elements in a <g> and transform the group instead of the individual elements.
Putting the ideas together, you could get a transition process something like this:
d3.selectAll("g.groups").transition().delay(time)
.attr("transform", function(d,i){ /* Calculate new transform */ })
.tween("stretch", function(d,i){
/* Select the sub-elements, do all the calculations
then create interpolators for both objects */
var g = d3.select(this);
var path = g.select("path");
var dot = g.select("circle");
var newEndPoint = /*** Calculate final position ***/;
var offset = /*** distance from end point to center of circle ***/;
var pathInterpolator = d3.interpolateString(
path.attr(d),
/*** new path including new end point ***/
);
var dotInterpolator = d3.interpolateObject(
{cx=dot.attr("cx"), cy=dot.attr("cy")},
{cx=newEndPoint.x + offset, cy=newEndPoint.y}
);
return function(t){
/* the function that updates both objects at each tick */
path.attr("d", pathInterpolator(t) );
dot.attr( dotInterpolator(t) );
};
});
How complex your real calculations are will depend on how arbitrary your "arbitrary path" is, of course. Maybe you'll need to calculate both x and y offsets to keep the circle positioned correctly. But that becomes an issue of geometry, not of synchronization. Regardless of what else your path shape does as it transitions, if the end point is an actual point in the path data, it will transition in a direct line, the same as the transition of the circle's coordinates.

Trace path with DOM object

I'm new to javascript and d3js. I would like a DOM object to trace out a path specified by a parametrized curve (x(t),y(t)). Here is an example of such a parametrization:
var theta = [];
for(var i = 0; i <= N; i++){
theta.push(2*Math.PI*i/N);
}
var points = [];
for(var i = 0; i <= N; i++){
points.push([Math.cos(theta[i]),Math.sin(theta[i])]);
}
The above is the parametrization of a curve -- in this case, also a circle -- and I would like my DOM object to follow the trajectory of this curve. [Aside: is there any better way to define points? It seems ridiculous to run a for loop.]
A crude way to achieve the sort of effect I'm looking for is to run a for loop in the update() part of d3. First, I simply append a circle to the svg variable, so that it need not be linked to any data. It is then selected and updated without required enter/exit.
for (var i = 0; i <= N; i++){
svg.selectAll("circle")
.transition()
.attr("cx",points[i][0]+w/2) // w: width
.attr("cy",points[i][1]+h/2) // h: height
.duration(dt) //
.delay(dt*i);
}
[Aside: I've heard queue() would be better, as opposed to calculating the total delay. Comments?] However, the easing property of the transition makes it run in a choppy fashion. I imagine I could specify no easing, but I'm sure there must be a better way to achieve what I want, which is simply for the initial DOM object (the circle) to move smoothly along a specific trajectory.
In the end, I would want to do this for multiple DOM objects which will eventually be linked to data, each with a specific curve to follow. Any tips on how I would do this?
Thanks in advance for any help, and I will gladly take any advice, including references.

Interesting but not terribly practical approach
The SVG spec actually has a number of animation options, including the ability to move an object along a path. The path is defined in the same form as for a <path> element, so you could use the d3.svg.arc functions to create the path.
Once you have a path defined, it is easy to use d3 to add in the animation:
http://fiddle.jshell.net/RnNsE/1/
although you'll want to read up on SVG animation elements and attributes.
However, there is a limitation to this wonderful animation: poor browser support. So if this is for a website, you're going to need to do the animation with d3 and Javascript.
Production-ready approach
The key to getting d3 to create smooth animations for you is to use a custom "tween" function on a transition.
When you do a transition, d3 initializes a tween function for each change on each element, and starts up timer functions to trigger the updates. At each "tick" of the timer, d3 calls the appropriate "tween" function with the information about how far along the transition it is. So if the tick occurs 500ms into a 2000ms transition, the tween function will given the value 0.25 (assuming a linear easing function, other easing functions complicate the relationship between time elapsed and the expected "distance" along the transition).
Now, for most changes the tween function is fairly straightforward, and d3 will figure one out for you automatically. If you change a "cx" value from 100 to 200, then the tween function is going to return 125 when the transition value 25%, 150 when the transition is 50%, and so on. If you change a "fill" value from red to yellow, it will calculate the numerical values of those colours and convert between them.
The value returned by the tween function at each tick is then used to update the attribute or style of the element. Since the updates happen many times a second, it usually results in a smooth animation. For the simple example of changing the "cx" value of a circle, the circle moves in a straight line from the starting point to the end point.
But you don't want it to move in a straight line. You want it to move in a circle (or along any path you choose). So you're going to need to create a custom function that tells the circle where it should be 25% of the way through the transition, and where it should be 50% through the transition, and so on.
And if you're worried you have to figure that out on your own, never fear. Like so, so much with D3, Mike Bostock has done the hard work for you. But even he didn't have to do the hard hard work. His approach uses two built-in Javascript functions for SVG paths, getTotalLength() and getPointAtLength(). The first tells you the total length of the path, the second gives you the coordinates of the point a certain distance from the start of the path.
With those two values, it is straightforward to figure out the coordinates you should be at if you want to be a certain percent of the way along the path: at 25%, you want to be at path.getPointAtLength(0.25*path.getTotalLength() ).
Here's Mike's function that makes that happen:
// Returns an attrTween for translating along the specified path element.
function translateAlong(path) {
var l = path.getTotalLength();
return function(d, i, a) {
return function(t) {
var p = path.getPointAtLength(t * l);
return "translate(" + p.x + "," + p.y + ")";
};
};
}
A little confusing, no? A function that returns a function that returns a function.
That's because when you specify a "tween" for a transition, what you actually have to specify is a "tween factory" -- the function that will return an appropriate tween function for each element in your selection.
Now, in his example he only has one path and one object moving along it, so those extra layers don't get used. But in the general case, your tween factory function would take the arguments d (the data object for that element in the selection), i (the index of that element) and a (the initial value of the attribute or style that you're changing). With those values, you have to return the custom tween function which take a transition state value t (a number between 0 or 1, or possibly a bit beyond 1 for certain easing functions) and computes the attribute value at that state in the transition.
You'll note that this function returns a translation instruction. That's generally going to be an easier way to move an object around, compared to using cx and cy, since you can specify both horizontal and vertical movement in one transform attribute call, so you only need the one tween function to do both.
Here's my example from above, updated to use a d3 tween to move the circles along the path:
http://fiddle.jshell.net/RnNsE/2/
Key code:
circles.transition().ease("linear")
.duration(5000)
.delay(function(d,i){return i*5000;})
.attrTween("transform", createPathTween);
//creates a tween function to translate an element
//along the path that is a sibling to the element
function createPathTween(d, i, a) {
var path = this.parentNode.getElementsByTagName("path")[0];
//i.e., go from this <circle> -> parent <g> -> array of child <path> elements
-> first (and only) element in that array
var l = path.getTotalLength();
return function(t) {
var p = path.getPointAtLength(t * l);
return "translate(" + p.x + "," + p.y + ")";
};
}
My version strips out the outermost layer of nested functions from Mike's version, but it adds in a bit of Javascript to find the correct <path> element for each circle element.
Note that you need an SVG path element in order to use the getTotalLength() and getPointAtLength() functions; however, this path can be invisible (fill:none; stroke:none; in CSS) if you don't want it to show on the screen. And again, while my path definitions are hard-coded, you could use one of d3's arc or line generators to construct it for you.
And just for fun, here's my example with a different easing function:
http://fiddle.jshell.net/RnNsE/3/
Note that I didn't change anything about the tweening function -- all that's changed is the t values that d3 passes in to that function as the transition progresses.
P.S. Here's another good resource on d3 custom tween functions:
http://blog.safaribooksonline.com/2013/07/11/reusable-d3-js-using-attrtween-transitions-and-mv/

How can I make Raphael.js elements "wiggle" on the canvas?

I'm working on a project that uses SVG with Raphael.js. One component is a group of circles, each of which "wiggles" around randomly - that is, slowly moves along the x and y axes a small amount, and in random directions. Think of it like putting a marble on your palm and shaking your palm around slowly.
Is anyone aware of a Raphael.js plugin or code example that already accomplishes something like this? I'm not terribly particular about the effect - it just needs to be subtle/smooth and continuous.
If I need to create something on my own, do you have any suggestions for how I might go about it? My initial idea is along these lines:
Draw a circle on the canvas.
Start a loop that:
Randomly finds x and y coordinates within some circular boundary anchored on the circle's center point.
Animates the circle from its current location to those coordinates over a random time interval, using in/out easing to smooth the effect.
My concern is that this might look too mechanical - i.e., I assume it will look more like the circle is tracing a star pattern, or having a a seizure, or something like that. Ideally it would curve smoothly through the random points that it generates, but that seems far more complex.
If you can recommend any other code (preferably JavaScript) that I could adapt, that would be great too - e.g., a jQuery plugin or the like. I found one named jquery-wiggle, but that seems to only work along one axis.
Thanks in advance for any advice!

Something like the following could do it:
var paper = Raphael('canvas', 300, 300);
var circle_count = 40;
var wbound = 10; // how far an element can wiggle.
var circleholder = paper.set();
function rdm(from, to){
return Math.floor(Math.random() * (to - from + 1) + from);
}
// add a wiggle method to elements
Raphael.el.wiggle = function() {
var newcx = this.attrs.origCx + rdm(-wbound, wbound);
var newcy = this.attrs.origCy + rdm(-wbound, wbound);
this.animate({cx: newcx, cy: newcy}, 500, '<');
}
// draw our circles
// hackish: setting circle.attrs.origCx
for (var i=0;i<circle_count;i++) {
var cx = rdm(0, 280);
var cy = rdm(0, 280);
var rad = rdm(0, 15);
var circle = paper.circle(cx, cy, rad);
circle.attrs.origCx = cx;
circle.attrs.origCy = cy;
circleholder.push(circle);
}
// loop over all circles and wiggle
function wiggleall() {
for (var i=0;i<circleholder.length;i++) {
circleholder[i].wiggle();
}
}
// call wiggleAll every second
setInterval(function() {wiggleall()}, 1000);
http://jsfiddle.net/UDWW6/1/
Changing the easing, and delays between certain things happening should at least help in making things look a little more natural. Hope that helps.

You can accomplish a similar effect by extending Raphael's default easing formulas:
Raphael.easing_formulas["wiggle"] = function(n) { return Math.random() * 5 };
[shape].animate({transform:"T1,1"}, 500, "wiggle", function(e) {
this.transform("T0,0");
});
Easing functions take a ratio of time elapsed to total time and manipulate it. The returned value is applied to the properties being animated.
This easing function ignores n and returns a random value. You can create any wiggle you like by playing with the return formula.
A callback function is necessary if you want the shape to end up back where it began, since applying a transformation that does not move the shape does not produce an animation. You'll probably have to alter the transformation values.
Hope this is useful!

There is a very good set of easing effects available in Raphael.
Here's a random set of circles that are "given" bounce easing.
Dynamically add animation to objects
The full range of easing effects can be found here. You can play around with them and reference the latest documentation at the same time.
Putting calls in a loop is not the thing to do, though. Use callbacks, which are readily available.