I am trying to get a larger amount of data. Sample data is below
1850/01 -0.845 -0.922 -0.748 -1.038 -0.652 -1.379 -0.311 -1.053 -0.636 -1.418 -0.272
1850/02 -0.043 -0.113 0.047 -0.244 0.159 -0.613 0.528 -0.260 0.177 -0.653 0.569
1850/03 -0.698 -0.794 -0.633 -0.891 -0.506 -1.123 -0.274 -0.910 -0.495 -1.174 -0.229
……….
2016/12 0.795 0.746 0.828 0.756 0.834 0.586 1.005 0.731 0.848 0.575 1.010
2017/01 1.025 0.977 1.067 0.983 1.068 0.786 1.265 0.963 1.084 0.778 1.271
2017/02 1.151 1.098 1.198 1.112 1.191 0.957 1.346 1.089 1.208 0.946 1.352
which starts from 1850 until 2017 and every month. I am processing this dataset to use it in Highcharts like this
$.each(lines, function(index, row) {
var cells = row.split(','),
series = {
type: 'line',
data:[]
};
$.each(cells, function(itemNo, item) {
if (itemNo == 0) {
series.name = item;
} else {
series.data.push(parseFloat(item));
}
});
data.push(series);
});
And I use it in following way
chart = $('#container').highcharts({
chart: {
polar: true
},
series: data
});
This does work however, it is really really slow. How can I improve/enhance its performance so that I the highcharts gets loaded quickly without freezing the browser?
UPDATE
Here is my xAxis
xAxis: {
tickInterval: 1,
min: 0,
max: 12,
categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'],
},
UPDATE 2
yAxis: {
min: -1,
max: 2.2,
endOnTick: false,
title: {
text: 'Temperature (°C)'
},
plotLines: [{
value: 2,
width: 1,
label: {
text: '2°C',
align: 'right',
y: 2
},
color: 'red'
}, {
value: 1.5,
width: 1,
label: {
text: '1.5°C',
align: 'right',
y: 30
},
color: 'red'
}],
},
I think this problem will require a combination of the solutions proposed by others. These include:
Condensing the data: If I understand correctly, then you have 167 years of data and 12 months per year. Each of these will be a series, for a total of >2000 series. I'm not sure this will create an interpretable graph, but it's also likely I misunderstand the nature of your data and how you plan to plot it.
Using the boost.js module of Highcharts: Highcharts normally renders its graphs as SVGs. My understanding of the boost.js module is that it causes some parts of the charts to be rendered on HTML5 canvas elements. The HTML5 canvas is much faster than SVG for large numbers of data points. See an empirical comparison here: SVG vs canvas: how to choose
Setting chart options to minimize resource requirements: I believe that the slowdown you're experiencing is unlikely to be due to the processing of your data. Rather, I think it's almost certainly due primarily to the rendering time required by Highcharts, and browser resources required to monitor events on all of the chart elements. By default, for instance, Highcharts plots allow you to "hover" your mouse over data points to highlight them, and they display tooltips with information about the data point. If you have a plot with thousands of data points, then this requires your browser to handle thousands of mouse events over the chart objects. Turning off this chart features should improve performance. In the demo below, I've turned off tooltips and data point highlighting using the mouse. I've also turned off the legend, to improve visibility of the chart.
You can process and update the data in chunks: In the long run, this will actually take more time than if you were to render the chunk all in one go, because Highcharts has to redraw the chart each time you add a new series. However, it might lead to a better user experience, since the page will appear more responsive. The demo below utilizes this type of approach. It allows you to set the number of lines of data to process per chunk (linesPerChunk) and the time delay between one chunk finishes processing and when you want to begin processing the next chunk (timeBetweenChunks). Ideally, timeBetweenChunks would be set to the time it takes Highcharts to render and display the last chunk, so that the computer alternates between processing data and rendering data, with no unproductive gaps in between. Ideally one could set this adaptively so that it's optimal across computers/users/browsers/etc., but I'm not sure how to do this; any ideas would be welcome. So for the moment it's just set to a constant 100 ms. With 2000 lines of data, 100 lines per chunk of data, and 100 ms between chunks, the whole thing should take ~2 s to load. The key function is plotMoreData(). After processing a chunk and adding the new series to the chart, it calls itself with a delay of timeBetweenChunks using window.setTimeout(plotMoreData, timeBetweenChunks). It then redraws the chart. When plotMoreData gets called the next time, it processes the next chunk, and so on. It stops when it's processed and displayed all the data and also updates the status message.
EDIT:
It seems the Highcharts boost module does not work with polar charts, and this is a known issue. A fix is described here: Polar Scatter Using Boost Module. I was able to get this fix to work by modifying boost.src.js (built from the Highcharts Github repository as follows:
At ~line 1380, I replaced:
if (!settings.useGPUTranslations) {
inst.skipTranslation = true;
x = xAxis.toPixels(x, true);
y = yAxis.toPixels(y, true);
}
with:
if (!settings.useGPUTranslations) {
inst.skipTranslation = true;
// Default (non-Polar) calculation
if( !series.chart.polar ) {
x = xAxis.toPixels(x, true);
y = yAxis.toPixels(y, true);
}
// Handle Polar chart coordinate conversion
else {
var polarPoint = {
plotX: xAxis.translate( x, 0, 0, 0, 1, series.options.pointPlacement, series.type === 'flags' ),
plotY: yAxis.translate( y, 0, 1, 0, 1 )
};
series.toXY( polarPoint );
x = Math.round( polarPoint.plotX );
y = Math.round( polarPoint.plotY );
}
}
This seemed to work. See the demo here: JSFiddle Polar Highcharts Boost Demo
Given the data you are displaying is not updating once a month, I feel like generating the chart for every view is a big waste of resources for your clients.
Indeed, it would be pretty easy for you to generate the chart without changing anything that you are doing now, but then extracting the resulting SVG and serving it simply to your visitors.
For that you simply have to use the getSVG method of HighCharts, that will return the SVG in a stringified form.
I don't really know if you want to have some sort of process to auto update the chart, but you could use a cron job for this purpose. Keep in mind that you would have to do something anyway even with your initial approach to update the data.
Regarding your script, first thing to notice is that you are using $.each, which is pretty bad in term of performance comparing to the vanilla equivalents. As demonstrated by this jsPerf, I get 3,649 Op/s with $.each whereas for loops could get you up to 202,755 Op/s which is insanely faster! Since you also doing a double loop, the difference would be ^2.
But again, since the data is not updated often, this part of the script could be removed entirely and converted into a JSON corresponding to the output of the function, that HighCharts could load directly, skipping the entire processing and transformation of your CSV.
If using HighCharts is not a requirement, you could use react-vis, which is a collection of React components focused around Data Visualization. It's build as an api on top of SVG, .
I've made a demo that you can checkout on CodePen with the same data as you to plot the monthly temperatures since 1850.
const {
HorizontalGridLines,
XAxis,
XYPlot,
YAxis,
LineMarkSeries,
} = reactVis
const color = d3.scaleLinear()
.domain([1850, 2017])
.range(['yellow', 'red'])
const Chart = () => (
<XYPlot width={window.innerWidth - 100} height={window.innerHeight - 100}>
<XAxis title='Month' />
<YAxis title='Temperature' />
<HorizontalGridLines />
{Object.keys(data).map(key => (
<LineMarkSeries color={color(key)} data={data[key]} key={key} />
))}
</XYPlot>
)
ReactDOM.render(<Chart />, document.querySelector('#root'))
I also use the scaleLinear method of d3 to see the change over the years since I thought it would be an interesting information to show, but it can be changed depending of your needs.
It's using SVGs, but we are also working on integration with webgl by the intermediary of deck.gl which would allow for even more optimizations gains, still not ready yet and I'm not sure you would really need that much, but worth noting.
Disclaimer: I currently work for Uber, which made deck.gl and react-vis.
new Array(length)
Specify the array's length rather than creating an empty array. Writing to an existing offset in an array is substantially faster than creating a new offset each time you write an item.
var data = new Array(lines.length); // Specify array length
$.each(lines, function(index, row) {
var cells = row.split(','),
series = {
type: 'line',
data: new Array(cells.length) // Specify array length
};
$.each(cells, function(itemNo, item) {
if (itemNo == 0) {
series.name = item;
} else {
series.data.push(parseFloat(item));
}
});
data.push(series);
});
I'd use underscore.js.
var result = _.map(lines, function (row) {
return { type: 'line', series: _.map(row.data, function (item, index) {
if (index >= 1)
return parseFloat(item);
}), name: row.data[0]}
});
(I'm writing this from my phone on a plane, so my apologies if there is are any typos or things I'm missing. Just trying to help!)
You don't show what is your choice for the xAxis, but if massaging the data in the server side is not an option what I would suggest is:
First do a call to load the data.
Show at spinning wheel where the chart is going to be displayed
Start parsing the dataset but don't load it all at once but in batches and redraw the chart as you parse the rest of the file.
This should avoid the freezing and give the visual indication that the data is being loaded.
Edit: Response to comment
I don't think is any longer an issue loading the data but on how to display the data in a meaningful way. Even if it loads in 2 seconds the result would be thousands of lines overlapping and totally unreadable.
You should default few hand picked values (ex Current month and same month and 1, 5, 10 years ago) and then allow the user to change the selection select up to a reasonable number of years/month (ex Compare July for 1980, 2010 and 2017) then plot those values updating the chart with (chart.series[0].update(....).
There were no columns in the dataset so I assumed it was a column per month but I see now that they seem to be one per row and 11 columns represent some other data.
I don't think for a polar chart doing averages or doing rollups of the data is the best way to go but allowing to compare specific years and month with the exact data, whatever those values are :)
Situation:
I think the best and more appropriate solution would be to process and serve chunks of data, it would synchronize the display of points on your Chart and avoid loading the whole data once on document load, even if your data isn't as large as you described it in your post title.
But after reading your question, your comments and other posts, I can see that you are getting all data from one file, and you aren't processing data on server side so server side chunks is not an option for you.
Alternative:
Well if you can't process your data on server side why don't you process it on client side, HighCharts has two options for you:
Data grouping:
You can follow Highstock Demos › 52,000 points with data grouping.
Lazy Async loading:
You can follow Highstock Demos › 1.7 million points with async loading.
The both Demos show how to fetch data by ranges and limit the chart to display only relevant range data points, which will improve time loading performances.
my team came across the similar problem but in our case we had 3 nested loops and when it was processing the data it was blocking the main thread which caused bad user experience. so i tackled this problem by deferring the processing data. which did the desired job of not blocking the main thread and did the processing much faster.
you can also use boost module developed by highchart (https://www.highcharts.com/blog/news/175-highcharts-performance-boost/)
Related
I am trying to build a waterfall chart which will have heavily skewed data(max 45000, min 4), to make the smaller values look significant amount and I am trying to break the yaxis to achieve that. However I am unable to find a generic logic as where exactly to break the yaxis so that the graph looks good for any data. Below is my code.
yAxis: {
title: {
text: 'USD'
},
// calculate percentage of first value for breaking
breaks: [{
from: data[0]*0.05, // break starts at 5% of 7311
to: data[0] * 0.97 // and ends at 97% of 7311
}],
events: {
pointBreak: pointBreakColumn
}
},
http://jsfiddle.net/Saibabu276/4cdrqnbj/61/
Also I am getting axis broken on the second bar in below case for the reason I couldn't figure out. Please Help!!
Setting the yAxis.type to logarithimic is the best way to work with skewed data.
Logarithmic axes can be useful when dealing with data with spikes or large value gaps, as they allow variance in the smaller values to remain visible.
More information with type scale use in charts, you can read on the Highcharts blog.
I'm new to JavaScript and HighCharts. I'm sure this is a very simple question, but I got lost.
I want to create a scatter chart with three lines. I need to read data from a text file, and the file looks like this:
x y1 y2 y3
1.02 1.00 6.70 8.19
2.04 1.00 13.30 8.19
3.06 1.00 13.50 8.19
4.08 1.00 9.60 8.19
5.10 1.00 14.60 8.19
6.12 1.00 19.20 8.57
So I need to plot three line with (x and y1), (x and y2), (x and y3)
And this is my HighCharts code:
<script type="text/javascript">
$(document).ready(function() {
var chart1 = new Highcharts.Chart(options);
});
var options = {
chart: {
renderTo: 'container',
type: 'scatter',
zoomType: 'xy'
},
title: {
text: 'Demo'
},
xAxis: {
title: {
enabled: true,
text: 'Time, ns'
},
startOnTick: true,
endOnTick: true,
showLastLabel: true
},
yAxis: {
title: {
text: 'Value'
}
},
series: []
};
$.get('///plot.txt', function(data) {
var lines = data.toString().split('\n');
$.each(lines, function(lineNo, line) {
var item = line.split()});
options.series[0].data[0].push(parseFloat(item[0]));
options.series[0].data[1].push(parseFloat(item[1]));
options.series[1].data[0].push(parseFloat(item[0]));
options.series[1].data[1].push(parseFloat(item[2]));
options.series[2].data[0].push(parseFloat(item[0]));
options.series[2].data[1].push(parseFloat(item[3]));
}, 'text')
var chart1 = new Highcharts.Chart(options);
</script>
I feel I messed up the entire code. I'm sorry but I never wrote JavaScript before. Any help would be hugely appreciated. Thank you in advance.
that's what you want to get : http://jsfiddle.net/z28vy/
Now a few comments
Read doc
You have to look at the HighChart documentation that is quite good, with live example on jsfiddle. Some are quite close to your use case (displaying data from data got with an AJAX call.)
What's peculiar in your need is to get raw text data, which force you into annoying parsing stuff.
Understand what you are doing
Even if your code could end up working, it looks like you are messing a bit with the sync/async story of your javascript. If you are a beginer that's a lot of things to learn at once. Not to mention you are trying to stick to jQuery style with anonymous functions...
Indent !
First thing, I do not know if it is just your post on here, or if you actually write code like that, but indent it properly ! It will show you a lot of problems at a glance. Especially when you are writing enclosed code (for example the success callback function of your ajax call.)
Arrays
Then, just a bit of logic : you have to know that although dynamic, arrays in javascript cannot have random access (read or write) on any non assigned slot. So when you are doing
options.series[0].data[0].push(parseFloat(item[0]));
you should have previously set options.series[0] which you didn't since your options object defines series as an empty array:
series: []
You can do that at the time of your ajax success method, or statically at options definition, depending on the flexibility you need in the number of series accross your use. I prefer you stay simple at first and do :
series: [{
name: 'Serie 1',
data: []
},{
name: 'Serie 2',
data: []
},{
name: 'Serie 3',
data: []
}]
Like that you can access each of your 3 series like you did... well except that you have the same problem with data[0] which does not exist neither for the same reason. Anyway do not bother much because...
First use your API as it goes
The way you add points to your series is far too complex anyway. series has an addPoint() method, just use it ! So instead of
options.series[0].data[0].push(parseFloat(item[0]));
options.series[0].data[1].push(parseFloat(item[1]));
Just do
options.series[0].addPoint([parseFloat(item[0]), parseFloat(item[1])]);
it's already easier to read :)
jQuery is not magical, it is just logical
Now the problem you have is your use of jquery $.each() I do not know if you just did not understand it or if you started to use it, then decided to hard-write your data handling to move on.
$.each(lines, function(lineNo, line) {
var item = line.split()});
options.series[0].data[0].push(parseFloat(item[0]));
options.series[0].data[1].push(parseFloat(item[1]));
As you will clearly see if you start indenting and separate things, you do nothing more than splitting each line for no purpose there.
Just use what you get with your split. If we say we are putting the current serie number in serieIdx :
$.each(lines, function(lineNo, line) {
var item = line.split(' ');
if(item.length==4 && !isNaN(parseFloat(item[0]))) { // skip unwanted line such as header or empty line
chart1.series[serieIdx].addPoint([parseFloat(item[0]), parseFloat(item[serieIdx+1])], false);
}
});
That's it. You just to have to iterate through your 3 series and you are good to go.
A side note about addPoint of serie in HighCharts
Be careful that if you call addPoint with invalid data (such as an array of anything instead of numbers) there is no visible error raised, but it breaks something anyway. In my case, before I added the test
if(item.length==4...
And since I had also at first a '\n' at the end of the last line, the upper split gave me one last empty string, which obviously ended up as en empty items array after the inner split. Which triggered an addPoint([NaN, NaN]) that purely made the lines between dots disappear for the whole graph. Be careful to that !
About the jsfiddle sample
jsfiddle obviously does not allow AJAX get but provide a trick instead : POST data in a json that jsfiddle server will resend back in the answer after the given delay (in my example I put 3 seconds.)
http://jsfiddle.net/z28vy/
I haven't read previous answer, since is pretty long, however I always advice to read tutorial from Highcharts: http://www.highcharts.com/docs/working-with-data/preprocessing-data-from-a-file-csv-xml-json
I currently have a page with some Highcharts graphics, around 22, and they are subject to filters selectable by the user. Each filter interaction removes the current series and adds new ones.
I started with two approaches, the first consisted in having a loop that went through all saved instance objects, removing each serie,
while currentChart.series.length > 0
currentChart.series[currentChart.series.length - 1].remove()
and then, after another loop that goes through arrays to search for the filter data and
currentChart.addSeries
name: operatorName
data: data
type: options.chartType
but this implementation turned out to be quite heavy.
I then implemented a timeout to let the plugin breathe a little and don't stack as much, by setting a 1ms timeout between each loop I got better performance and could see the animations but It makes it a bit sluggish. Setting a higher timeout makes the page hard to navigate and filter changes would stack, as the entire graphics update would take much time.
In my experience, when I have a large number of charts or datapoints, I generally turn off animation. Animation adds a lot over processing overhead, and without it highcharts generally handles a lot of data very well.
I guess you could try removing the series and adding the new one for each chart at a time, rather than removing all and then adding all. I doublt this will make much different to your page responsiveness though.
chart: {
animation: false
},
plotOptions: {
series: {
animation: false
}
},
while currentChart.series.length > 0
currentChart.series[0].destroy()
ex: http://jsfiddle.net/5B9c7/1/
I'm using Highcharts for a project in which I have to display two series with about a thousand points each. The x-axis represents a date, and the y-axis a quantity. In addition, each point has an associated list of namesMy data is day-by-day without gaps, with a structure such as
var mydata = [ ...
{x: theDate, y: theValue, names: theNames},
... ]
where theNames is an array of strings. I can access these in the tooltip formatter through this.points.point.names, given that the range displayed on the chart is small enough. If I change the x-axes so that the start date and end date are more than roughly a year apart, then the tooltip is not rendered at all.
One of the possible avenues that I have tried but failed with so far is setting the turboThreshold limit to the length of the longest series plus 1. Setting this lets me at least display a graph when mydata.length > 1000 (the default value). However, this only displays the tooltip if the x-axis range is less than 261. Otherwise, the tooltip disappears entirely, as does the point.data object where I'm getting the name from.
I'm also not great at JavaScript, but I was wondering if there were a way to separate the names of the points from the array containing them (in my examples, myData1 and myData2) and somehow access those names from the tooltip function without going through the current point.
Here is the link to the jsFiddle demonstrating this issue.
All help is appreciated!
The problem is in dataGrouping, when disabled works fine: http://jsfiddle.net/34tfg/1/
DataGrouping is method in Highcharts to approximate points and display them when width of the chart is not enough, e.g. how to display 10 000points in a chart of width 1 000px -> 10 points in a one pixel..? And when dataGrouping is used, new points are created, so all your custom options like 'names' etc. are lost (at least not accessible).
Code:
plotOptions: {
line: {
dataGrouping: {
enabled: false
},
turboThreshold: 10000
}
},
My chart is displaying data day-by-day (user and system sent SMS/Newsletters). So time part is not relevant and unnecessary. Here is an example: http://jsfiddle.net/uaxZP/1/ where time "12:00" is displayed.
How can I force Highcharts to never display the time part in a "datetime" axis?
EDIT: Highcharts will display dates w/wo times based on chart or screen size. I've updated jsfiddle removing some data and now it's displaying also time parts.
You can intercept the x-axis label function and change it's output. In my example, I've changed it to render short dates:
http://jsfiddle.net/uaxZP/3/
{ xAxis:
labels: {
formatter: function() {
return Highcharts.dateFormat("%b %e", this.value);
}
}
}
The xAxis.labels.formatter property allows control over this. You also may notice I'm using Highcharts.dateFormat, which is a utility function for rendering dates. This is not mandatory, but it's a nice built in feature. Documentation on the xAxis formatter is here:
http://www.highcharts.com/ref/#xAxis-labels--formatter
The easiest way to do is using "minTickInterval"
xAxis: {
minTickInterval: 24 * 3600 * 1000
}
http://api.highcharts.com/highcharts#xAxis.minTickInterval