Based on the web audio analyser API, I am creating an audio animation that draws images based on the real time frequency spectrum (like the classical bar graphics that move to the frequency of the sound, except that it is not bars that are drawn but something more complex).
It works fine, my only issue is that I am not able to stop the image at a precise time.
When I want to have it stopped at let's say 5 seconds, then some times it stops at 5.000021, or 5.000013, or 5.0000098, ...
and the problem is that the frequency spectrum (and so my image based on this frequency spectrum) is not the same at 5.000021, or 5.000013, or 5.0000098, ...
This means that the user when he wants to see the image corresponding to 5s, every time he sees a slightly different image, and I would like to have only one image corresponding to 5s (often the image is only slightly different at every try, but sometimes the differences are quite huge).
Here are extracts of my code:
var ctx = new AudioContext();
var soundmp3 = document.getElementById('soundmp3');
soundmp3.src = URL.createObjectURL(this.files[0]);
var audioSrc = ctx.createMediaElementSource(soundmp3);
var analyser = ctx.createAnalyser();
analyser.fftSize = 2048;
analyser.smoothingTimeConstant = 0.95;
audioSrc.connect(analyser);
audioSrc.connect(ctx.destination);
var frequencyData = new Uint8Array(analyser.frequencyBinCount);
function renderFrame() {
if(framespaused) return;
drawoneframe();
requestAnimationFrame(renderFrame);
};
function drawoneframe(){
analyser.getByteFrequencyData(frequencyData);
// drawing of my image ...
};
function gotomp3(timevalue){
soundmp3.pause();
newtime = timevalue;
backtime = newtime - 0.2000;
soundmp3.currentTime = backtime;
soundmp3.play();
function boucle(){
if(soundmp3.currentTime >= timevalue){
if(Math.abs(soundmp3.currentTime-newtime) <= 0.0001){
drawoneframe();
soundmp3.pause();
soundmp3.currentTime = timeatgetfrequency;
return;
} else {
soundmp3.pause();
soundmp3.currentTime = backtime;
soundmp3.play();
};
}
setTimeout(boucle, 1);
};
boucle();
};
document.getElementById("fieldtime").onkeydown = function (event) {if (event.which == 13 || event.keyCode == 13) {
gotomp3(document.getElementById("fieldtime").value);
return false;
}
return true;
};
Code explanation: if the user enters a value in the "fieldtime" (= newtime) and hits enter, then the I go first 0.2s back, start playing and stop when the currentTime is very close to newtime (I have to go back first, because when I go directly to newtime and stop immediately afterwards then the analyser.getByteFrequencyData does not yet have the values at newtime). With boucle() I manage to get it stopped at very precise times: if newtime = 5, then the time when drawoneframe(); is called is 5.000xx but the problem is that every time the user enters 5 as newtime, the image that is shown is slightly different.
So my question: has someone an idea how I could achieve that every time the user enters the same time as newtime, the image will be exactly the same ?
I am not quite aware at which times the soundmp3.currentTime is updated ? With a samplerate of 44.1kHz, I guess it is something like every 0.0227ms, but does this mean that it is updated exactly at 0, 0.0227ms, 0.0454ms, ...or just approximately ?
I thought about smoothing the analyser results, so that there are less variations for small time variations. Setting analyser.smoothingTimeConstant to a value close to 1 is not sufficient. But maybe there is another way to do it.
I do not need high precision results, I just would like that if a user wants to see the image corresponding to x seconds, then each time he enters x seconds, he sees exactly the same image.
Thank you very much for any help.
Mireille
Related
I have an ASCII art "pathfinding visualizer" which I am modeling off of a popular one seen here. The ASCII art displays a n by m size board with n*m number of nodes on it.
My current goal is to slowly change the appearance of the text on the user-facing board, character by character, until the "animation" is finished. I intend to animate both the "scanning" of the nodes by the pathfinding algorithm and the shortest path from the start node to the end node. The animation, which is just changing text in a series of divs, should take a few seconds. I also plan to add a CSS animation with color or something.
Basically the user ends up seeing something like this, where * is the start node, x is the end node, and + indicates the path:
....
..*.
..+.
.++.
.x..
.... (. represents an empty space)
After doing some research on both setTimeout, promises and other options, I can tell you:
JavaScript really isn't designed to allow someone to delay code execution in the browser.
I've found lots of ways to freeze the browser. I also tried to set a series of promises set to resolve after setTimeout(resolve, milliseconds) occurs, where the milliseconds steadily increases (see below code). My expectation was that numOfAnimationsForPath number of promises would be set and trigger a change in the appearance of the board when each one resolved (forming the appearance of a path). But, they all seem to resolve instantly (?) as I see the path show as soon as I click the "animate" button
const shortestPathAndScanningOrder = dijkstras(grid);
promisesRendering(1000, shortestPathAndScanningOrder[0].path, shortestPathAndScanningOrder[1])
function promisesRendering(animationDelay, algoPath, scanTargets) {
const numOfAnimationsForScanning = scanTargets.length;
const numOfAnimationsForPath = algoPath.length;
for (let i = 1; i < numOfAnimationsForPath - 1; i++) {
const xCoordinate = algoPath[i][0];
const yCoordinate = algoPath[i][1];
renderAfterDelay(animationDelay * i).then(renderNode(xCoordinate, yCoordinate, "path"))
}
}
function renderAfterDelay(milliseconds) {
return new Promise(resolve => setTimeout(resolve, milliseconds))
}
function renderNode(x, y, type) {
if (type === "scan") {
const targetDiv = getLocationByCoordinates(x, y);
targetDiv.innerHTML = VISITED_NODE;
} else if (type === "path") {
const targetDiv = getLocationByCoordinates(x, y);
targetDiv.innerHTML = SHORTEST_PATH_NODE;
} else {
throw "passed incorrect parameter to 'type' argument"
}
}
In my other attempt, I tried to generate pathLength number of setTimeouts as in:
const shortestPathAndScanningOrder = dijkstras(grid);
renderByTimer(10000, shortestPathAndScanningOrder[0].path, shortestPathAndScanningOrder[1])
function renderByTimer(animationDelay, algoPath, scanTargets) {
const numOfAnimations = algoPath.length;
for (let i = 1; i < numOfAnimations - 1; i++) {
const xCoordinate = algoPath[i][0];
const yCoordinate = algoPath[i][1];
setTimeout(i * animationDelay, updateCoordinatesWithTrailMarker(xCoordinate, yCoordinate))
}
}
...but this also resulted in the path being "animated" instantly instead of over a few seconds as I want it to be.
I believe what I want is possible because the Pathfinding Visualizer linked at the start of the post animates its board slowly, but I cannot figure out how to do it with text.
So basically:
If anyone knows how I can convince my browser to send an increasing delay value a series of function executions, I'm all ears...
And if you think it can't be done, I'd like to know that too in the comments, just so I know I have to choose an alternative to changing the text slowly.
edit: a friend tells me setTimeout should be able to do it... I'll update this w/ a solution if I figure it out
Edit2: Here is the modified version of #torbinsky's code that ended up doing the job for me...
function renderByTimer(algoPath, scanTargets) {
const numOfAnimations = algoPath.length - 1; // - 1 because we don't wanna animate the TARGET_NODE at the end
let frameNum = 1;
// Renders the current frame and schedules the next frame
// This repeats until we have exhausted all frames
function renderIn() {
if (frameNum >= numOfAnimations) {
// end recursion
console.log("Done!")
return
}
// Immediately render the current frame
const xCoordinate = algoPath[frameNum][0];
const yCoordinate = algoPath[frameNum][1];
frameNum = frameNum + 1;
updateCoordinatesWithTrailMarker(xCoordinate, yCoordinate);
// Schedule the next frame for rendering
setTimeout(function () {
renderIn(1000)
}, 1000);
}
// Render first frame
renderIn()
}
Thanks #torbinsky!
This should absolutely be doable using setTimeout. Probably the issue is that you are immediately registering 10,000 timeouts. The longer your path, the worse this approach becomes.
So instead of scheduling all updates right away, you should use a recursive algorithm where each "frame" schedules the timeout for the next frame. Something like this:
const shortestPathAndScanningOrder = dijkstras(grid);
renderByTimer(10000, shortestPathAndScanningOrder[0].path, shortestPathAndScanningOrder[1])
function renderByTimer(animationDelay, algoPath, scanTargets) {
const numOfAnimations = algoPath.length;
// Renders the current frame and schedules the next frame
// This repeats until we have exhausted all frames
function renderIn(msToNextFrame, frameNum){
if(frameNum >= numOfAnimations){
// end recursion
return
}
// Immediately render the current frame
const xCoordinate = algoPath[frameNum][0];
const yCoordinate = algoPath[frameNum][1];
updateCoordinatesWithTrailMarker(xCoordinate, yCoordinate);
// Schedule the next frame for rendering
setTimeout(msToNextFrame, function(){
renderIn(msToNextFrame, frameNum + 1)
});
}
// Render first frame
renderIn(1000, 1)
}
Note: I wrote this code in the StackOverflow code snipppet. So I was not able to test it as I did not have the rest of your code to fully run this. Treat it more like pseudo-code even though it probably works ;)
In any case, the approach I've used is to only have 1 timeout scheduled at any given time. This way you don't overload the browser with 1000's of timeouts scheduled at the same time. This approach will support very long paths!
This is a general animation technique and not particularly unique to ASCII art except that old-school ASCII art is rendered one (slow) character at a time instead of one fast pixel frame at a time. (I'm old enough to remember watching ASCII "movies" stream across hard-wired Gandalf modems at 9600bps to a z19 terminal from the local mainframe...everything old is new again! :) ).
Anyhow, queueing up a bunch of setTimeouts is not really the best plan IMO. What you should be doing, instead, is queueing up the next event with either window.requestAnimationFrame or setTimeout. I recommend rAF because it doesn't trigger when the browser tab is not showing.
Next, once you're in the event, you look at the clock delta (use a snapshot of performance.now()) to figure what should have been drawn between "now" and the last time your function ran. Then update the display, and trigger the next event.
This will yield a smooth animation that will play nicely with your system resources.
I'm trying to gradually change the frequency amount of my lowpass filter, but instead of happening gradually, it happens instantly.
This code should start at a frequency, exponentially decrease to 200 at 1 second in, then stop at 2 seconds in. Instead it stays the same until 1 second where it instantly jumps to the lower frequency.
var context = new (window.AudioContext || window.webkitAudioContext)();
var oscillator = context.createOscillator();
var now = context.currentTime;
//lowpass node
var lowPass = context.createBiquadFilter();
lowPass.connect(context.destination);
lowPass.frequency.value = 500;
lowPass.Q.value = 0.5;
lowPass.frequency.exponentialRampToValueAtTime(200, now + 1);
oscillator.connect(lowPass);
oscillator.start(now);
oscillator.stop(now + 2);
edit: I just realized it does actually work in chrome. But I mainly use firefox, can I just not use webaudio yet?
The AudioParam interface has 2 modes:
A. Immediate
B. Automated
In mode A you simply set the value property of the parameter.
In mode B, if you want to 'ramp' from 500 to 200 you have
to use an automation event first to set the value to 500, eg:
frequency.setValueAtTime(500, 0)
A startTime parameter of zero applies the value immediately
according to the specs.
What you are trying is to intermingle both modes, but the latter
does not take the value of the first into account.
I'm using the Web Audio API to play an MP3 file. I'm unaware of how to schedule events to happen at certain intervals, namely at 0.25, 0.5, 0.75 1 second intervals for quarter notes when using a source buffer (i.e., not creating the sound like certain guides suggest: http://middleearmedia.com/timed-rhythms-with-web-audio-api-and-javascript/).
My context is setup like the following:
function setupAudioNodes() {
javascriptNode = context.createScriptProcessor(2048, 1, 1);
javascriptNode.connect(context.destination);
analyser = context.createAnalyser();
analyser.smoothingTimeConstant = 0.3;
analyser.fftSize = 32;
sourceNode = context.createBufferSource();
sourceNode.connect(analyser);
analyser.connect(javascriptNode);
sourceNode.connect(context.destination);
}
I can retrieve the current time using context.currentTime where context is an audio context, but assigning a callback on that doesn't happen regularly.
I've tried something trivial such as:
if ( context.currentTime.toFixed(2).split(".")[1] === .25 ) {
runFunc();
}
However, this doesn't not happen at regular intervals. If I print out the output it doesn't always print out the same time intervals which means that this approach doesn't appear to work well.
Another approach I've tried is using setInterval which I know is far from ideal for lots of reasons but that doesn't work either and will call the function once following the normal interval pace (1 second) and then breaks down into calling it many times.
javascriptNode.onaudioprocess = function() {
var array = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteFrequencyData(array);
setInterval(myFunc(array), 1000); // first time ok, after it runs many times a second
}
My last, somewhat naive approach, is to create a range of acceptable values. The application I'm working with doesn't need to be extremely precise so I'm hoping I can listen for ranges such as 0.26 - 0.28. This approach almost works except that it still calls the function several times but several times only every second.
if (time >= 0 && time <= 25 ) {
myFunc(array);
}
Where am I going wrong?
You can run a checking function very often, for example every 10ms and check if specified time has exceeded. If yes, run the callback and clear the interval:
var intervalId = setInterval(function() {
if (time >= 25) {
runFunc();
clearInterval(intervalId);
}
}, 10);
To demonstrate this (I'm working on a much more compressed Fiddle), open this in chrome, and move Jay-Z using your arrow keys and catch about 4 - 5 (sometimes more!) cakes.
You will notice that there is a massive cupcake on the left side of the screen now.
I update the cakes' positions in my handleTick function, and add new cakes on a time interval. Here are both of those:
/*This function must exist after the Stage is initialized so I can keep popping cakes onto the canvas*/
function make_cake(){
var path = queue.getItem("cake").src;
var cake = new createjs.Bitmap(path);
var current_cakeWidth = cake.image.width;
var current_cakeHeight = cake.image.height;
var desired_cakeWidth = 20;
var desired_cakeHeight = 20;
cake.x = 0;
cake.y = Math.floor((Math.random()*(stage.canvas.height-35))+1); //Random number between 1 and 10
cake.scaleX = desired_cakeWidth/current_cakeWidth;
cake.scaleY = desired_cakeHeight/current_cakeHeight;
cake.rotation = 0;
cake_tray.push(cake);
stage.addChild(cake);
}
And the setInterval part:
setInterval(function(){
if (game_on == true){
if (cake_tray.length < 5){
make_cake();
}
}
else{
;
}
},500);
stage.update is also called from handleTick.
Here is the entire JS file
Thanks for looking into this. Note once again that this only happens in Chrome, I have not seen it happen on Firefox. Not concerned with other browsers at this time.
Instead of using the source of your item, it might make more sense to use the actual loaded image. By passing the source, the image may have a 0 width/height at first, resulting in scaling issues.
// This will give you an actual image reference
var path = queue.getResult("cake");
I have this complicated loop that calculates various still frames of what I want to show in a canvas element. Each time the frame is calculated it gets displayed I call a timer and wait till I clear it and then the next frames is displayed and so on.
drawing(transform(alone, Box, canvasx.width, canvasx.height), false, "00f", canvasx);
drawing(transform(Lines, Box, canvasx.width, canvasx.height), false, "ff0000", canvasx);
var date = new Date();
var curDate = null;
do {
curDate = new Date();
}
while (curDate - date < 550);
if (alone.length > 0) {
canvasx.width = canvasx.width;
}
if i put a break point in var date line and press play each time, every individual frame get displayed but when I let it run through the canvas is empty while it runs and at the end it displays the last frame.
now if i delete the canvasx.width = canvasx.width; I still get the same behavior only obviously at the end i get all frames drawn one on top of the other.
Obviously its not an animation so i cant call drawing in a setinterval.
does anyone has any idea why;
You have to use setTimeout instead of your loop.
// JavaScript is single thread, and this BLOCKS the re-rendering and display of the canvas
do {
curDate = new Date();
}
while (curDate - date < 550);
Also I'd rather use context.clearRect (See MDC) instead of the resizing, there's no performance difference whatsoever.
Something along these lines should do it:
function drawCanvas() {
if (alone.length > 0) {
// clear canvas
}
drawing(transform(alone, Box, canvasx.width, canvasx.height), false, "00f", canvasx);
drawing(transform(Lines, Box, canvasx.width, canvasx.height), false, "ff0000", canvasx);
setTimeout(drawCanvas, 550);
}
Never try to emulate sleep in JavaScript you will block the whole Browser from doing anything while you're sleeping.