I was reading G. Skinner's take on using an image to detect volume peaks in order to create a custom visualizer display for audio and was wondering If anyone ever tried it using waveform images( like from soundcloud) to detect the peaks( volume/ amplitude/ frequency or whatever the term is) from an audio file. I want to use this as an alternative to using the Web Audio API which is a bit more tedious if you ask me and not supported in older devices( android 4.0).
Is there a way to process the colour data from a waveform image that can use to simulate such tasks?
You can access raw pixel data from an image by using a canvas element and the Canvas API. You can render an image into a a canvas like so:
// getting the waveform image by loading it over HTTP
var waveformImage = new Image();
waveformImage.onload = onImageLoad;
waveformImage.src = 'path/to/waveform/image/file.jpg';
function onImageLoad() {
// create a canvas element
var canvas = document.createElement("canvas");
// get the rendering context for the canvas
var renderingContext = canvas.getContext("2d");
// render the image to the canvas with drawImage()
renderingContext.drawImage(waveformImage, 0, 0);
// retrieve the raw pixel data using the width and height
// properties of the image (which is this, since we are attaching
// this method directly to waveformImage.onload
var imageData = renderingContext.getImageData(0, 0, this.width, this.height);
// process imageData from here, imageData is an ImageData instance
// see: https://developer.mozilla.org/en-US/docs/Web/API/ImageData
}
This example is largely based on the MDN example here. Once you have your raw image data you can use whatever algorithms your research provides to extract amplitude or whatever audio information you need from the waveform.
Related
The picture on the HTML page is represented as a canvas, but I have problems getting the source image name of this canvas, using javascript.
Page: https://www.futbin.com/22/squad-builder
I found this image in the page resources:
But how to get it programmically? I mean, if i have:
var canvas = document.getElementById('field-area')
How to get the file name? Result should be "field_1920x1080.png?v=8"
You can use HTMLCanvasElement.toDataURL to get the image drawn on the canvas base64 encoded
var canvas = document.getElementById('field-area');
var b64url = canvas.toDataURL('image/png');
This will return the url for the image itself, i don't think you can get the Asset name directly from a canvas element without inspecting the code on the website
EJBEAN is right, the canvas element doesn't know what resources were used to draw.
Why don't you use dataset to keep an eye on what you used?
var canvas = document.getElementById('field-area');
// ... draw resource here
canvas.dataset.resource = "field_1920x1080.png";
It's an easy way to keep data stored for any element on your page.
I am building an web application for my experiment purpose. The aim here is to capture ~15-20 frames per second from the webcam and send it to the server. Once the frame is captured, it is converted to base64 and added to an array. After certain time, it is sent back to the server. Currently I am using imageCapture.takePhoto() to achieve this functionality. I get blob as a result which is then changed to base64. The application runs for ~5 seconds and during this time, frames are captured and sent to the server.
What are the more efficient ways to capture the frames through webcam to achieve this?
You can capture still images directly from the <video> element used to preview the stream from .getUserMedia(). You set up that preview, of course, by doing this sort of thing (pseudocode).
const stream = await navigator.getUserMedia(options)
const videoElement = document.querySelector('video#whateverId')
videoElement.srcObject = stream
videoElement.play()
Next, make yourself a canvas object and a context for it. It doesn't have to be visible.
const scratchCanvas = document.createElement('canvas')
scratchCanvas.width = video.videoWidth
scratchCanvas.height = video.videoHeight
const scratchContext = scratchCanvas.getContext('2d')
Now you can make yourself a function like this.
function stillCapture(video, canvas, context) {
context.drawImage( video, 0, 0, video.videoWidth, video.videoHeight)
canvas.toBlob(
function (jpegBlob) {
/* do something useful with the Blob containing jpeg */
}, 'image/jpeg')
}
A Blob containing a jpeg version of a still capture shows up in the callback. Do with it whatever you need to do.
Then, invoke that function every so often. For example, to get approximately 15fps, do this.
const howOften = 1000.0 / 15.0
setInterval (stillCapture, howOften, videoElement, scratchCanvas, scratchContext)
All this saves you the extra work of using .takePhoto().
I have a PNG image (in my database) that I want to load into my page in my canvas. I have no idea how to do it. I couldn't figure out how to load the image as a ImageBitmap or anything compatible with canvas. How do I give canvas the bytes and tell it that it's using the png format/set the mime to png?
I would split this into two parts: (1) You'll want to load an image into something an ImageBitmap can use, and (2) you'll want to create the ImageBitmap to put this into your canvas.
This could look something like this:
const fileName = 'resources/image.png';
const fetched = await fetch(fileName); // Retrieve an image file.
const blob = await fetched.blob(); // Get a blob to represent this image.
const imageBitmap = await createImageBitmap(blob); // Create an imageBitmap from this blob.
const canvas = document.getElementById('canvas'); // Make sure your HTML contains a <canvas> element.
const context = canvas.getContext('2d');
context.drawImage(imageBitmap, 0,0); // Draw this image onto the canvas.
HTML
<img id="myImage" src="imageSrc">
<canvas id ="myCanvas" width="width" height="height"></canvas>
JS
var c = document.getElementById("myCanvas");
var ctx = c.getContext("2d");
var img = document.getElementById("myImage");
ctx.drawImage(img, 0, 0);
This is the simplest way. The drawImage() method can take multiple image sources, not just an HTML image element. For example, you do not need to put the image in the DOM as I demonstrated. I recommend you read the documentation: https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage
I'd like to resize uploaded images in browser. I am using canvas to draw the uploaded image to canvas and then resize it, and use the result from toDataURL method.
Simplified code (without upload parts) looks something like this:
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d', { alpha: false} );
// img src contains data url of uploaded image
ctx.drawImage(img, 0, 0, imgWidth, imgHeight);
var dataUrl = canvas.toDataURL('image/png');
The problem is dataUrl contains alpha channel, although context was created with alpha set to false.
Is it possible to get data url without the alpha channel?
If not, I considered using one of Javascript image libraries, but most of them rely on canvas.
Also, I could encode the image using data in canvas, but I'd rather not do that :)
alpha:false is only used in WebGL. It's ignored when creating a 2d context.
But you can export the canvas in jpg format where your unwanted alpha is eliminated:
// export a full-quality jpg dataUrl
canvas.toDataURL("image/jpeg", 1.0);
I need to take HTML5 canvas output as video or swf png sequence.
I found the following link on stackoverflow for capturing images.
Capture HTML Canvas as gif/jpg/png/pdf?
But can anyone suggest if we want the output to be video or swf of png sequence?
EDIT:
Ok now I understood how to capture the canvas data to store on server, I tried it and it is working fine if I use only shapes, rectangle or some other graphic, but not if I draw external images on canvas element.
Can anyone tell me how to capture canvas data completely whether we use graphic or external images for drawing on canvas?
I used the following code:
var cnv = document.getElementById("myCanvas");
var ctx = cnv.getContext("2d");
if(ctx)
{
var img = new Image();
ctx.fillStyle = "rgba(255,0,0,0.5)";
ctx.fillRect(0,0,300,300);
ctx.fill();
img.onload = function()
{
ctx.drawImage(img, 0,0);
}
img.src = "my external image path";
var data = cnv.toDataURL("image/png");
}
after taking the canvas data into my "data" variable I created a new canvas and draw the captured data on that, the red rectangle drawn on the second canvas but that external image doesn't.
Thanks in advance.
I would suggest:
Use setInterval to capture the contents of your Canvas as a PNG data URL.
function PNGSequence( canvas ){
this.canvas = canvas;
this.sequence = [];
};
PNGSequence.prototype.capture = function( fps ){
var cap = this;
this.sequence.length=0;
this.timer = setInterval(function(){
cap.sequence.push( cap.canvas.toDataURL() );
},1000/fps);
};
PNGSequence.prototype.stop = function(){
if (this.timer) clearInterval(this.timer);
delete this.timer;
return this.sequence;
};
var myCanvas = document.getElementById('my-canvas-id');
var recorder = new PNGSequence( myCanvas );
recorder.capture(15);
// Record 5 seconds
setTimeout(function(){
var thePNGDataURLs = recorder.stop();
}, 5000 );
Send all these PNG DataURLs to your server. It'll be a very large pile of data.
Using whatever server-side language you like (PHP, Ruby, Python) strip the headers from the data URLs so that you are left with just the base64 encoded PNGs
Using whatever server-side language you like, convert the base64 data to binary and write out temporary files.
Using whatever 3rd party library you like on the server, convert the sequence of PNG files to a video.
Edit: Regarding your comment of external images, you cannot create a data URL from a canvas that is not origin-clean. As soon as you use drawImage() with an external image, your canvas is tainted. From that link:
All canvas elements must start with their origin-clean set to true.
The flag must be set to false if any of the following actions occur:
[...]
The element's 2D context's drawImage() method is called with an HTMLImageElement or an HTMLVideoElement whose origin is not the same as that of the Document object that owns the canvas element.
[...]
Whenever the toDataURL() method of a canvas element whose origin-clean flag is set to false is called, the method must raise a SECURITY_ERR exception.
Whenever the getImageData() method of the 2D context of a canvas element whose origin-clean flag is set to false is called with otherwise correct arguments, the method must raise a SECURITY_ERR exception.
To start out, you want to capture the pixel data from the canvas on a regular interval (using JavaScript timers probably). You can do this by calling context.getImageData on the canvas's context. That will create a series of images that you can turn into a video stream.
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#pixel-manipulation