We Keep Coding

Updating candlestick data point using restyle in Plotly.js

I am using a WebSocket connection to update a candlestick chart with live data.
Creating the initial candlestick chart is relatively easy:
var candleDiv = document.getElementById('candle-chart');
var data = {
x: x, //Each of these is a single dimension array of the same length
open: open,
close: close,
high: high,
low: low,
type: 'candlestick',
};
var layout = {
datarevision: candleCount,
dragmode: 'zoom',
showlegend: false,
xaxis: {
type: 'date',
range: [x[x.length - 26], x[x.length - 1]], //Only show the last 25 entries so it's not zoomed out too far.
rangeslider: {
visible: false
},
yaxis: {
autorange: true,
}
}
}
data.xaxis = 'x';
data.yaxis = 'y';
data = [data];
Plotly.plot(candleDiv, data, layout);
However, the documentation for the restyle method doesn't talk much to the update of data. More about how the data is displayed. After much tinkering, I found a reasonable workaround of updating the data variable directly:
candleDiv.data[0].open[candleDiv.data[0].open.length - 1] = updatedOpenValue;
candleDiv.data[0].close[candleDiv.data[0].close.length - 1] = updatedCloseValue;
candleDiv.data[0].high[candleDiv.data[0].high.length - 1] = updatedHighValue;
candleDiv.data[0].low[candleDiv.data[0].low.length - 1] = updatedLowValue;
Plotly.restyle(candleDiv, 'data[0]', candleDiv.data[0], [0]);
This works, except that it appears to draw the new candle on the old candle. This becomes particularly distracting when the stick changes from a green (increasing) stick to a red (decreasing) stick.
Is there a correct syntax to achieve what I am attempting to do such that I don't get display issues?
I checked out this link from this post but I couldn't get the method used to work in the context of a candlestick chart.

You may want to look at the Plotly.react method instead of Plotly.restyle: https://plot.ly/javascript/plotlyjs-function-reference/#plotlyreact

Optimising a group of dc.js line graphs

I have a group of graphs visualizing a bunch of data for me (here), based off a csv with approximately 25,000 lines of data, each having 12 parameters. However, doing any interaction (such as selecting a range with the brush on any of the graphs) is slow and unwieldy, completely unlike the dc.js demo found here, which deals with thousands of records as well but maintains smooth animations, or crossfilter's demo here which has 10 times as many records (flights) as I do.
I know the main resource hogs are the two line charts, since they have data points every 15 minutes for about 8 solid months. Removing either of them makes the charts responsive again, but they're the main feature of the visualizations, so is there any way I can make them show less fine-grained data?
The code for the two line graphs specifically is below:
var lineZoomGraph = dc.lineChart("#chart-line-zoom")
.width(1100)
.height(60)
.margins({top: 0, right: 50, bottom: 20, left: 40})
.dimension(dateDim)
.group(tempGroup)
.x(d3.time.scale().domain([minDate,maxDate]));
var tempLineGraph = dc.lineChart("#chart-line-tempPer15Min")
.width(1100).height(240)
.dimension(dateDim)
.group(tempGroup)
.mouseZoomable(true)
.rangeChart(lineZoomGraph)
.brushOn(false)
.x(d3.time.scale().domain([minDate,maxDate]));
Separate but relevant question; how do I modify the y-axis on the line charts? By default they don't encompass the highest and lowest values found in the dataset, which seems odd.
Edit: some code I wrote to try to solve the problem:
var graphWidth = 1100;
var dataPerPixel = data.length / graphWidth;
var tempGroup = dateDim.group().reduceSum(function(d) {
if (d.pointNumber % Math.ceil(dataPerPixel) === 0) {
return d.warmth;
}
});
d.pointNumber is a unique point ID for each data point, cumulative from 0 to 22 thousand ish. Now however the line graph shows up blank. I checked the group's data using tempGroup.all() and now every 21st data point has a temperature value, but all the others have NaN. I haven't succeeded in reducing the group size at all; it's still at 22 thousand or so. I wonder if this is the right approach...
Edit 2: found a different approach. I create the tempGroup normally but then create another group which filters the existing tempGroup even more.
var tempGroup = dateDim.group().reduceSum(function(d) { return d.warmth; });
var filteredTempGroup = {
all: function () {
return tempGroup.top(Infinity).filter( function (d) {
if (d.pointNumber % Math.ceil(dataPerPixel) === 0) return d.value;
} );
}
};
The problem I have here is that d.pointNumber isn't accessible so I can't tell if it's the Nth data point (or a multiple of that). If I assign it to a var it'll just be a fixed value anyway, so I'm not sure how to get around that...

When dealing with performance problems with d3-based charts, the usual culprit is the number of DOM elements, not the size of the data. Notice the crossfilter demo has lots of rows of data, but only a couple hundred bars.
It looks like you might be attempting to plot all the points instead of aggregating them. I guess since you are doing a time series it may be unintuitive to aggregate the points, but consider that your plot can only display 1100 points (the width), so it is pointless to overwork the SVG engine plotting 25,000.
I'd suggest bringing it down to somewhere between 100-1000 bins, e.g. by averaging each day:
var daysDim = data.dimension(function(d) { return d3.time.day(d.time); });
function reduceAddAvg(attr) {
return function(p,v) {
if (_.isLegitNumber(v[attr])) {
++p.count
p.sums += v[attr];
p.averages = (p.count === 0) ? 0 : p.sums/p.count; // gaurd against dividing by zero
}
return p;
};
}
function reduceRemoveAvg(attr) {
return function(p,v) {
if (_.isLegitNumber(v[attr])) {
--p.count
p.sums -= v[attr];
p.averages = (p.count === 0) ? 0 : p.sums/p.count;
}
return p;
};
}
function reduceInitAvg() {
return {count:0, sums:0, averages:0};
}
...
// average a parameter (column) named "param"
var daysGroup = dim.group().reduce(reduceAddAvg('param'), reduceRemoveAvg('param'), reduceInitAvg);
(reusable average reduce functions from the FAQ)
Then specify your xUnits to match, and use elasticY to auto-calculate the y axis:
chart.xUnits(d3.time.days)
.elasticY(true)

Highcharts label grouping

I am making a diagram that should show states in a timeline like diagram. I figured that the best fit for it would be horizontal bar diagram, I added data like this:
series: [{
data: [?],
name: 'state1'
}, {
data: [?],
name: 'state2'
}, {
data: [?],
name: 'state3'
}, {
data: [?],
name: 'state1'
}, {
....
}, {
data: [?],
name: 'state1'
}]
Here's what I'v got:
The chart looks like what I need, but I need to somehow group legends, for example this case there should be only 3 legends: state1, state2, state3.
How can I achieve something like this?

I would suggest a different approach, using the column range series type.
This way you can do this with only three series, with a data point for each time slot, rather than a series for each time slot. This saves from having to manipulate the legend or fake other series level actions.
This is an example I did for an earlier question that demonstrates the method (with only two states):
http://jsfiddle.net/jlbriggs/o9ck2zLn/0/
can easily be adapted to use a datetime axis as well, which it seems you might be going for.

Basing on the answer of #StAlex, I would recommend a similar solution, but without the ECMAScript 5 specific forEach:
var legend = {};
for (var i = 0; i < series.length; i++) {
var bar = series[i];
if (!legend[bar.name]) {
legend[bar.name] = 0;
}
legend[bar.name] += bar.data;
}
I assumed that data is a number. If this is not so, add a comment.

var legends = {};
series.forEach(function(bar){
legends[bar.name] = bar.data;
});

How to zoom into x axis for multiple Flot charts that were created

I'm trying to get zoomin to work for the Flot charts created using following code.
var options = {
yaxis: { min: 0 },
xaxis: { mode: "time" },
series:{
lines: { show: true },
points: { show: true }
},
grid: {
hoverable: true,
clickable: false,
mouseActiveRadius: 30,
backgroundColor: { colors: ["#D1D1D1", "#7A7A7A"] }
},
selection:{mode: "x"}
};
var pdata = [];
for (var key in datasets) {
pdata = [];
pdata.push(datasets[key]);
$.plot( $('<div style="width:1200px;height:600px;"></div>').appendTo('#placeholder'),pdata,options);
$('<h5 align="center">'+datasets[key]['label']+'</h5>').appendTo('#placeholder');
$('<br>').appendTo('#placeholder');
$("#placeholder").UseTooltip();
};
Here I'm creating multiple charts in a loop.
How can I add zoomin feature.
Thank you.

Follow-up to Mark's answer: unique IDs are not really a Flot limitation; that's a requirement of the HTML spec. Browsers generally let you get away with breaking this rule, but it's still not a good idea. Mark's answer is good, but here's one that doesn't require an array-search on every event:
$.each(datasets, function(key, dataset) {
var element = $('<div style="width:1200px;height:600px;"></div>')
.appendTo('#placeholder');
var plot = $.plot(element, [dataset], options);
var plotOptions = plot.getOptions();
element.bind('plotselected', function(event, ranges) {
plotOptions.xaxes[0].min = ranges.xaxis.from;
plotOptions.xaxes[0].max = ranges.xaxis.to;
plot.setupGrid();
plot.draw();
});
};

flot generally expects it's place holder div to have a unique id. You would then use this unique id to assign a specific plotselected event to that plot. The way you have your code structured, though, you are appending the real placeholder div to a parent div as you create your plots. I like your approach so we need to work around flot's limitation.
So, in your plot call give your real placeholder div a class name. This will give us something to bind the plotselected event to. Also you need to save a reference to all the plot objects you've created. I'd just use a global array.
myPlots.push(
$.plot( $('<div class="myPlot" style="width:300px;height:100px;"></div>').appendTo('#placeholder'),pdata,options)
);
Where myPlots is the global array and my class is myPlot.
After this, you can set up the plotselected handler on the jquery selector .myPlots. Next for the tricky part, you need to find your plot object reference inside the handler. The easiest way to do this, I found, is to loop your myPlots array and compare their divs to the div the event happens on:
$(".myPlot").bind("plotselected", function (event, ranges) {
for (var i = 0; i < myPlots.length; i++)
{
var aPlot = myPlots[i];
if (aPlot.getPlaceholder()[0] == event.currentTarget) //this is the correct plot
{
var opts = myPlots[i].getOptions();
opts.xaxes[0].min = ranges.xaxis.from;
opts.xaxes[0].max = ranges.xaxis.to;
myPlots[i].setupGrid();
myPlots[i].draw();
}
}
});
You'll see above I'm redrawing the plot a little different than in the flot examples. I prefer this method since you don't have to remember the data, you adjust the min/max options and you redraw.
Here's a fiddle putting this all together.

Adding data to a highchart chart using an array with IDs

I want to add a series to a highchart scatterplot where I am naming each point in the series. I create a chart in the following way:
var chart; // globally available
makeCharts = function(){
chart = new Highcharts.Chart({
chart: {
renderTo: 'container1',
type: 'scatter'
},
series: [{
name: 'a',
data: [{
'id': 'point1',
'x': 1,
'y': 2
}, {
'id': 'point2',
'x': 2,
'y': 5
}]
}]
});
}
I would like to be able to update the points on the chart using something like:
chart.series[0].setData([{id:['point3', 'point4', 'point5'], y:[0,1,2], x:[1,2,3]}])
but this is not correct. Is it possible to update a chart using this approach where each point has an ID?
EDIT:
Just to clarify, I would like to be able to pass the arrays directly, rather than adding the data point by point using addPoint(). I could loop through an array and use addPoint() doing something like this:
id:['point3', 'point4', 'point5'];
y:[0,1,2];
x:[1,2,3];
for (i=0; i<x.length; i++)
{
chart.series[0].addPoint({
x: x[[i],
y: y[i],
id: id[i]
});
}
However, this is very slow. It's much quicker to add data using the following approach:
chart.series[0].setData([[1,0],[2,1],[3,2]]);
I have found that I can add data like this:
chart.series[0].setData([[1,0, 'point3'],[2,1, 'point4'],[3,2, 'point5']]);
but then the only way that I can access the id when the point is selected, is through this.point.config[2]. With the following approach I am unable to use chart.get('pointID') to identify a point as I did not set the ID. I want to be able to identify the point using just the ID.

Well broadly speaking there are two ways in which you can modify the chart data dynamically
Series.setData() Use this approach when you want to completely replace the existing data with some new data
Series.addPoint() Use this approach when you want to add a subset of the points dynamically. This method is not just for adding one point at a time, if you read the documentation carefully again you will find that this method takes a boolean redraw argument, and the argument detail is as following
redraw: Boolean
Defaults to true. Whether to redraw the chart after
the point is added. When adding more than one point, it is highly
recommended that the redraw option beset to false, and instead
chart.redraw() is explicitly called after the adding of points is
finished.
In your case, since you want to add a few points dynamically, but retaining the existing points, you should go with approach 2. But you need to use it inside a loop, with the redraw being set to false (hence solving the problem of being slow) and then after the loop, call the redraw method explicitly
Code
var id = ['point3', 'point4', 'point5'],
y = [0, 1, 2],
x = [1, 2, 3];
for (var i = 0; i < x.length; i++) {
chart.series[0].addPoint({
x: x[i],
y: y[i],
id: id[i]
},false);
}
chart.redraw();
Adding multiple points dynamically | Highcharts and Highstock # jsFiddle

Try using series.addPoint.
chart.series[0].addPoint({
x: 0,
y: 0,
id: 'anything'
});
But if you need to set data for series, use
chart.series[0].setData([{
x: 0,
y: 0,
id: 'anything'
},{
x: 2,
y: 2,
id: 'another'
}]);

As soon as you can pass your data like this:
chart.series[0].setData([[1,0, 'point3'],[2,1, 'point4'],[3,2, 'point5']]);
(as you stated in question), I can suggest you to use a little hack.
We'll need to add another statement to method applyOptions of Highcharts.Point prototype.
if (typeof options[0] === 'number' && options[2] && typeof options[2] === 'string') this.id = options[2];
Here you can see it in action.