Sync an html5 video with an highchart.js line chart - javascript

I have a one minute long video of a train moving and some data series about his position, onboard accelerometer etc.
My goal is to make line charts (with Highcharts.js) of those measurements that are dynamically drawn with the video at a rate of 20points per second. The charts have to move with the video, so that if the user go back also the charts go at the same frame and so on.
I was wondering if there's a way to attach an event to the video progress bar and redraw the chart every x milliseconds and/or every time that the user play/stop the video

Connecting timeupdate event for a video element with setData method for a Highcharts series should be enough.
Example:
let currentDataIndex = 0;
const video = document.getElementById("video1");
const chart = Highcharts.chart('container', {
series: [{
// Highcharts mutate original data array, so use a copy
data: data[currentDataIndex].slice()
}]
});
const updateChartData = index => {
chart.series[0].setData(data[index].slice());
currentDataIndex = index;
};
video.addEventListener('timeupdate', e => {
const second = Math.floor(video.currentTime);
if (second !== currentDataIndex) {
updateChartData(second);
}
});
Live demo: http://jsfiddle.net/BlackLabel/8a6yvjs5/
API Reference: https://api.highcharts.com/class-reference/Highcharts.Series#setData
Docs:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/currentTime
https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/timeupdate_event

Related

LightningJs - Find if chart is ready

I have x and y data fetched via ajax , after I fetch I add series like below
series.add({ x: xVal, y: yVal})
But problem is out of 5 times , 3 times chart not loading. I think it is because I try to add series before chart is ready. Is there any callback to know if chart is ready and then I can add the x and y to series ?
At least as of now, LCJS renders synchronously with animation frames so you can find out when the Chart has rendered its first frame (and is pretty much "ready") with requestAnimationFrame:
const chart = lightningChart().ChartXY()
// ... Chart application code ...
// Get informed when Chart has rendered.
requestAnimationFrame( () => console.log('ready') )
But you really shouldn't need to wait for the Chart before adding data to it.
Currently there is no callback to know when the chart is ready.
The behavior you described shouldn't be possible, as long as you're always creating the chart first, before the series - and creating the series before adding a point to it.
Please make sure the order of creation is Chart -> Series -> Series.add

VideoTexture is not playing in VR

At first the texture works fine and the video plays as expected, but when VR is entered via VRDisplay.requestPresent it stops. Why is this and how to fix it?
The VR display has its own render loop. Usually needsUpdate is automatically set to true on every animation frame by three.js, but this is only true for the default display.
To fix this, get the VR display from the vrdisplayconnect event and create your own update loop. E.g.
let display = e.display;
let displayUpdateLoop = () =>
{
// May get a warning if getFrameData is not called.
let frameData = new VRFrameData();
display.getFrameData(frameData);
videoTexture.needsUpdate = true;
// Stop loop if no longer presenting.
if (display.isPresenting)
display.requestAnimationFrame(displayUpdateLoop);
}
display.requestAnimationFrame(displayUpdateLoop);

How do I prevent my javascript locking up the browser?

I have a WebSocket that receives data at a rate of ~50hz. Each time an update is pushed to the browser, it turns the JSON data it's had published to it into some pretty charts.
$(document).ready(function() {
console.log('Connecting web socket for status publishing')
allRawDataPublisher = new ReconnectingWebSocket("ws://" + location.host + '/raw/allrawdata');
var rawUnprocessedData = [];
for (i = 0; i < 256; i++)
{
rawUnprocessedData.push({x:i, y:0});
}
var unprocessedRawChart = new CanvasJS.Chart("rawUnprocessedData",{
title :{ text: "Raw Unprocessed Data"},
axisX: { title: "Bin"},
axisY: { title: "SNR"},
data: [{ type: "line", dataPoints : rawUnprocessedData},{ type: "line", dataPoints : noiseFloorData}]
});
var updateChart = function (dps, newData, chart) {
for (i = 0; i < 256; i++)
{
dps[i].y = newData[i];
}
chart.render();
};
allRawDataPublisher.onmessage = function (message) {
jsonPayload = JSON.parse(message.data);
var dataElements = jsonPayload["Raw Data Packet"]
updateChart(rawUnprocessedData, dataElements["RAW_DATA"].Val, unprocessedRawChart)
};
unprocessedRawChart.render();
});
This works great when my laptop is plugged into a power socket but if I unplug the power, my laptop drops it's processing power (and the same issue occurs on lower-specc'd tablets, phones etc). When there's less processing power available, the browser (Chrome) completely locks up.
I'm guessing the javascript is receiving updates faster than the browser can render them and consequently locking the tab up.
If the browser is unable to update at the requested rate, I would like it to drop new data until it's ready to render a new update. Is there a standard way to check the browser has had enough time to render an update and drop new frames if that's not the case?
*[Edit]
I did some digging with Chrome's profiler which confirms that (as expected) it's re-drawing the chart that is taking the bulk of the processing power.
You can do work between frames by using window.requestAnimationFrame.
The callback passed to this function will be called at a maximum of 60 times a second - or whichever number matches the refresh rate of your display.
It's also guaranteed to be called before the next repaint - and after the previous repaint has finished.
From MDN window.requestAnimationFrame()
The window.requestAnimationFrame() method tells the browser that you wish to perform an animation and requests that the browser call a specified function to update an animation before the next repaint. The method takes a callback as an argument to be invoked before the repaint.
Here's an example on how it's used:
function renderChart () {
// pull data from your array and do rendering here
console.log('rendering...')
requestAnimationFrame(renderChart)
}
requestAnimationFrame(renderChart)
However, it's better to render changes to your graph in batches, instead of doing rendering work for every single datum that comes through or on every frame.
Here's a Fiddle using Chart.js code that:
Pushes data to an Array every 100ms (10 Hz)
Renders data, in batches of 4 - every 1000ms (1 Hz)
const values = []
const ctx = document.getElementById('chartContainer').getContext('2d');
const chart = new Chart(ctx, {
type: 'line',
data: {
labels: ['start'],
datasets: [{
label: 'mm of rain',
data: [1],
borderWidth: 1
}]
}
});
// Push 1 item every 100ms (10 Hz), simulates
// your data coming through at a faster
// rate than you can render
setInterval(() => {
values.push(Math.random())
}, 100)
// Pull last 4 items every 1 second (1 Hz)
setInterval(() => {
// splice last 4 items, add them to the chart's data
values.splice(values.length - 4, 4)
.forEach((value, index) => {
chart.data.labels.push(index)
chart.data.datasets[0].data.push(value)
})
// finally, command the chart to update once!
chart.update()
}, 1000)
Do note that the above concept needs to handle exceptions appropriately, otherwise the values Array will start accumulating so much data that the process runs out of memory.
You also have to be careful in the way you render your batches. If your value render rate is slower than the rate at which you fill the values Array, you will eventually run into memory issues.
Last but not least: I'm not really convinced you ever need to update a piece of data faster than 2 Hz, as I doubt the human brain can make useful interpretations at such a fast rate.

Moving navigator in Highcharts

I would like to fetch data for example from this website: https://www.highcharts.com/stock/demo/lazy-loading
To get the data with high resolution, I have to set the min max to small value and then I have to go over the complete diagram.
It is possible to get data from the website with "Highcharts.charts[0].series[0]"
But I didn't find a possibility to programable changing the navigators, to get other data ranges.
Is it possible to move the navigator to a specific position?
You can use Axis.setExtremes function for this.
Live demo: http://jsfiddle.net/kkulig/am3cgo5w/
API reference: https://api.highcharts.com/class-reference/Highcharts.Axis#setExtremes
Thank you for your answear. Here is my code if somebody needs:
I saw, that the redraw event will be called 5 times till the diagram is finishd redrawn
counter = 1
// Get chart
chart = $('#highcharts-graph').highcharts()
// add event
Highcharts.addEvent(chart, 'redraw', function (e) {
console.log(counter);
counter = counter+1;
if(counter>5){
console.log('finish loaded')
function2()
}
})
var newSelectedMin = 0
var newSelectedMax = 10
chart.xAxis[0].setExtremes(newSelectedMin, newSelectedMax);

Queue getting multiple video frames

I'm creating a PDF output tool using jsPDF but need to add multiple pages, each holding a canvas image of a video frame.
I am stuck on the logic as to the best way to achieve this as I can't reconcile how to queue the operations and wait on events to achieve the best result.
To start I have a video loaded into a video tag and can get or set its seek point simply with:
video.currentTime
I also have an array of video seconds like the following:
var vidSecs = [1,9,13,25,63];
What I need to do is loop through this array, seek in the video to the seconds defined in the array, create a canvas at these seconds and then add each canvas to a PDF page.
I have a create canvas from video frame function as follows:
function capture_frame(video_ctrl, width, height){
if(width == null){
width = video_ctrl.videoWidth;
}
if(height == null){
height = video_ctrl.videoHeight;
}
canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
var ctx = canvas.getContext('2d');
ctx.drawImage(video_ctrl, 0, 0, width, height);
return canvas;
}
This function works fine in conjunction with the following to add an image to the PDF:
function addPdfImage(pdfObj, videoObj){
pdfObj.addPage();
pdfObj.text("Image at time point X:", 10, 20);
var vidImage = capture_frame(videoObj, null, null);
var dataURLWidth = 0;
var dataURLHeight = 0;
if(videoObj.videoWidth > pdfObj.internal.pageSize.width){
dataURLWidth = pdfObj.internal.pageSize.width;
dataURLHeight = (pdfObj.internal.pageSize.width/videoObj.videoWidth) * videoObj.videoHeight;
}else{
dataURLWidth = videoObj.videoWidth;
dataURLHeight = videoObj.videoHeight;
}
pdfObj.addImage(vidImage.toDataURL('image/jpg'), 'JPEG', 10, 50, dataURLWidth, dataURLHeight);
}
My logic confusion is how best to call these bits of code while looping through the vidSecs array as the problem is that setting the video.currentTime needs the loop to wait for the video.onseeked event to fire before code to capture the frame and add it to the PDF can be run.
I've tried the following but only get the last image as the loop has completed before the onseeked event fires and calls the frame capture code.
for(var i = 0; i < vidSecs.length; i++){
video.currentTime = vidSecs[i];
video.onseeked = function() {
addPdfImage(jsPDF_Variable, video);
};
}
Any thoughts much appreciated.
This is not a real answer but a comment, since I develop alike application and got no solution.
I am trying to extract viddeo frames from webcam live video stream and save as canvas/context, updated every 1 - 5 sec.
How to loop HTML5 webcam video + snap photo with delay and photo refresh?
I have created 2 canvases to be populated by setTimeout (5000) event and on test run I don't get 5 sec delay between canvas/contextes, sometimes, 2 5 sec. delayed contextes get populated with image at the same time.
So I am trying to implement
Draw HTML5 Video onto Canvas - Google Chrome Crash, Aw Snap
var toggle = true;
function loop() {
toggle = !toggle;
if (toggle) {
if (!v.paused) requestAnimationFrame(loop);
return;
}
/// draw video frame every 1/30 frame
ctx.drawImage(v, 0, 0);
/// loop if video is playing
if (!v.paused) requestAnimationFrame(loop);
}
to replace setInterval/setTimeout to get video and video frames properly synced
"
Use requestAnimationFrame (rAF) instead. The 20ms makes no sense. Most video runs at 30 FPS in the US (NTSC system) and at 25 FPS in Europe (PAL system). That would be 33.3ms and 40ms respectively.
"
I am afraid HTML5 provided no quality support for synced real time live video processing via canvas/ context, since HTML5 offers no proper timing since was intended to be event/s controlled and not run as real time run app code ( C, C++ ...).
My 100+ queries via search engine resulted in not a single HTML5 app I intend to develop.
What worked for me was Snap Photo from webcam video input, Click button event controlled.
If I am wrong, please correct me.
Two approaches:
create a new video element for every seek event, code provided by Chris West
reuse the video element via async/await, code provided by Adrian Wong

Categories