How to send multiple textures to a fragment shader in WebGL? - javascript

So in the javascript portion of my code, here is the snippet that actually sends an array of pixels to the vertex and fragment shaders- but I am only working with 1 texture when I get to those shaders- is there anyway that I can send two textures at a time? if so, how would I 'catch' both of them on the GLSL side of the codee?
if (it > 0){
gl.activeTexture(gl.TEXTURE1);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.activeTexture(gl.TEXTURE0);
gl.bindFramebuffer(gl.FRAMEBUFFER, FBO2);}
else{
gl.activeTexture(gl.TEXTURE1);
gl.bindTexture(gl.TEXTURE_2D, texture2);
gl.activeTexture(gl.TEXTURE0);
gl.bindFramebuffer(gl.FRAMEBUFFER, FBO);}
gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4);

You reference multiple textures in GLSL by declaring multiple sampler uniforms
uniform sampler2D u_myFirstTexture;
uniform sampler2D u_mySecondTexture;
...
vec4 colorFrom1stTexture = texture2D(u_myFirstTexture, someUVCoords);
vec4 colorFrom2ndTexture = texture2D(u_mySecondTexture, someOtherUVCoords);
You can specific which texture units those 2 samplers use by calling gl.uniform1i as in
var location1 = gl.getUniformLocation(program, "u_myFirstTexture");
var location2 = gl.getUniformLocation(program, "u_mySecondTexture");
...
// tell u_myFirstTexture to use texture unit #7
gl.uniform1i(location1, 7);
// tell u_mySecondTexture to use texture unit #4
gl.uniform1i(location2, 4);
And you setup texture units by using gl.activeTexture and gl.bindTexture
// setup texture unit #7
gl.activeTexture(gl.TEXTURE7); // or gl.TEXTURE0 + 7
gl.bindTexture(gl.TEXTURE_2D, someTexture);
...
// setup texture unit #4
gl.activeTexture(gl.TEXTURE4); // or gl.TEXTURE0 + 4
gl.bindTexture(gl.TEXTURE_2D, someOtherTexture);
...

Related

WebGL warning: drawArraysInstanced: TEXTURE_2D at unit 0 is incomplete: The dimensions of `level_base` are not all positive

I am trying to use a texture on webgl.
the code for importing this image is:
var image = new Image(); // Create the image object
image.onload = loadTexture(gl, n, texture, u_Sampler, image);
image.crossOrigin = "";
image.src = 'beacon.png';
function loadTexture(gl, n, texture, u_Sampler, image) {
gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, 1); // Flip the image's y axis
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, image);
gl.uniform1i(u_Sampler, 0);
gl.clear(gl.COLOR_BUFFER_BIT); // Clear <canvas>
gl.drawArrays(gl.TRIANGLE_STRIP, 0, n); // Draw the rectangle
}
including the line image.crossOrigin = "" stops the CORS error from showing in firefox (but not in chrome?) but firefox does give three other warnings when the page tries to load:
unreachable code after return statement
^ this happens even when all return statements are removed from the program.
Exceeded 16 live WebGL contexts for this principal, losing the least recently used one.
and
WebGL warning: drawArraysInstanced: TEXTURE_2D at unit 0 is incomplete: The dimensions of `level_base` are not all positive.
the texture doesn't load, the canvas appears as a black square
You generally need to run a server for images in WebGL.
If your image is on the same domain then you should not set crossDomain. If, it's on a different domain then you should, though the server still needs to give permission to use the image. See this
Otherwise there's a few issues with the code
It's not actually setting a onload callback on the image
this line
image.onload = loadTexture(gl, n, texture, u_Sampler, image);
is functionally equivilent to
const result = loadTexture(gl, n, texture, u_Sampler, image);
image.onload = result;
The line needs to be something like this so that loadTexture
is called after the image loads, not before
image.onload = () => loadTexture(gl, n, texture, u_Sampler, image);
You might need to set wrapping to CLAMP_TO_EDGE
// these two lines needed in WebGL1 but not WebGL2 if the image is not
// power of 2 in both dimension
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
Your title suggests your using WebGL2 by mentioning drawAraysInstanced but the code you posted does not use that and you didn't tag webgl2
const vs = `
void main() {
gl_Position = vec4(0, 0, 0, 1);
gl_PointSize = 100.0;
}
`;
const fs = `
precision mediump float;
uniform sampler2D u_Sampler;
void main() {
gl_FragColor = texture2D(u_Sampler, gl_PointCoord.xy);
}
`;
const gl = document.querySelector('canvas').getContext('webgl');
const prg = twgl.createProgram(gl, [vs, fs]);
const u_Sampler = gl.getUniformLocation(prg, 'u_Sampler');
gl.useProgram(prg);
const texture = gl.createTexture();
const n = 4;
var image = new Image(); // Create the image object
image.onload = () => loadTexture(gl, n, texture, u_Sampler, image);
image.crossOrigin = "";
image.src = 'https://i.imgur.com/ZKMnXce.png';
function loadTexture(gl, n, texture, u_Sampler, image) {
gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, 1); // Flip the image's y axis
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
// these two lines needed in WebGL1 but not WebGL2
// if the image is not power-of-2 in both dimensions
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, image);
gl.uniform1i(u_Sampler, 0);
gl.clear(gl.COLOR_BUFFER_BIT); // Clear <canvas>
//gl.drawArrays(gl.TRIANGLE_STRIP, 0, n); // Draw the rectangle
gl.drawArrays(gl.POINTS, 0, 1);
}
<script src="https://twgljs.org/dist/4.x/twgl.min.js"></script>
<canvas></canvas>
As for the other warnings.
unreachable code after return statement
You didn't post enough code to find that warning but it means exactly what it says. For example
function foo() {
return;
console.log('here');
}
will generate that warning
Exceeded 16 live WebGL contexts for this principal, losing the least recently used one.
This is arguably a bug in Firefox though if the code is bad (like if it's creating canvases and contexts in a loop) then there are legit cases for this warning . In general though it comes up reloading a page 17 times like during web development. Hopefully Firefox will fix it so it only shows the warning when it's legit.

How to add interpolation using webgl vertex and fragment shaders to the image

Hello everyone,
I am trying to render a image using webgl shaders and I have successfully done that using webgl samples but the issue is that when i increase the size of image the quality of image is not good. I want to upscale and interpolate the image using vertex and fragment shader.Here is my sample
"use strict";
function main() {
var image = new Image();
requestCORSIfNotSameOrigin(image, "https://upload.wikimedia.org/wikipedia/commons/5/57/Pneumothorax_CT.jpg")
image.src = "https://upload.wikimedia.org/wikipedia/commons/5/57/Pneumothorax_CT.jpg";
image.width = 1000;
image.height = 1000;
image.onload = function() {
render(image);
}
}
function render(image) {
// Get A WebGL context
/** #type {HTMLCanvasElement} */
var canvas = document.getElementById("canvas");
var gl = canvas.getContext("webgl");
if (!gl) {
return;
}
// setup GLSL program
var program = webglUtils.createProgramFromScripts(gl, ["2d-vertex-shader", "2d-fragment-shader"]);
// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");
var texcoordLocation = gl.getAttribLocation(program, "a_texCoord");
// Create a buffer to put three 2d clip space points in
var positionBuffer = gl.createBuffer();
// Bind it to ARRAY_BUFFER (think of it as ARRAY_BUFFER = positionBuffer)
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
// Set a rectangle the same size as the image.
setRectangle(gl, 0, 0, image.width, image.height);
// provide texture coordinates for the rectangle.
var texcoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
0.0, 0.0,
1.0, 0.0,
0.0, 1.0,
0.0, 1.0,
1.0, 0.0,
1.0, 1.0,
]), gl.STATIC_DRAW);
// Create a texture.
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
// Set the parameters so we can render any size image.
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
// Upload the image into the texture.
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
// lookup uniforms
var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
webglUtils.resizeCanvasToDisplaySize(gl.canvas);
// Tell WebGL how to convert from clip space to pixels
gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
// Clear the canvas
gl.clearColor(0, 0, 0, 0);
gl.clear(gl.COLOR_BUFFER_BIT);
// Tell it to use our program (pair of shaders)
gl.useProgram(program);
// Turn on the position attribute
gl.enableVertexAttribArray(positionLocation);
// Bind the position buffer.
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
// Tell the position attribute how to get data out of positionBuffer (ARRAY_BUFFER)
var size = 2; // 2 components per iteration
var type = gl.FLOAT; // the data is 32bit floats
var normalize = false; // don't normalize the data
var stride = 0; // 0 = move forward size * sizeof(type) each iteration to get the next position
var offset = 0; // start at the beginning of the buffer
gl.vertexAttribPointer(
positionLocation, size, type, normalize, stride, offset)
// Turn on the teccord attribute
gl.enableVertexAttribArray(texcoordLocation);
// Bind the position buffer.
gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
// Tell the position attribute how to get data out of positionBuffer (ARRAY_BUFFER)
var size = 2; // 2 components per iteration
var type = gl.FLOAT; // the data is 32bit floats
var normalize = false; // don't normalize the data
var stride = 0; // 0 = move forward size * sizeof(type) each iteration to get the next position
var offset = 0; // start at the beginning of the buffer
gl.vertexAttribPointer(
texcoordLocation, size, type, normalize, stride, offset)
// set the resolution
gl.uniform2f(resolutionLocation, gl.canvas.width, gl.canvas.height);
// Draw the rectangle.
var primitiveType = gl.TRIANGLES;
var offset = 0;
var count = 6;
gl.drawArrays(primitiveType, offset, count);
}
function setRectangle(gl, x, y, width, height) {
var x1 = x;
var x2 = x + width;
var y1 = y;
var y2 = y + height;
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
x1, y1,
x2, y1,
x1, y2,
x1, y2,
x2, y1,
x2, y2,
]), gl.STATIC_DRAW);
}
main();
// This is needed if the images are not on the same domain
// NOTE: The server providing the images must give CORS permissions
// in order to be able to use the image with WebGL. Most sites
// do NOT give permission.
// See: http://webglfundamentals.org/webgl/lessons/webgl-cors-permission.html
function requestCORSIfNotSameOrigin(img, url) {
if ((new URL(url)).origin !== window.location.origin) {
img.crossOrigin = "";
}
}
#import url("https://webglfundamentals.org/webgl/resources/webgl-tutorials.css");
body {
margin: 0;
}
canvas {
width: 100vw;
height: 100vh;
display: block;
}
<div style="height:700px; width:700px; overflow:scroll;">
<canvas id="canvas"></canvas>
</div>
<!-- vertex shader -->
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
attribute vec2 a_texCoord;
uniform vec2 u_resolution;
varying vec2 v_texCoord; void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
// pass the texCoord to the fragment shader
// The GPU will interpolate this value between points.
v_texCoord = a_texCoord;
}
</script>
<!-- fragment shader -->
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
// our texture
uniform sampler2D u_image;
// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;
void main() {
// Look up a color from the texture.
gl_FragColor = texture2D(u_image, v_texCoord);
}
</script>
<script src="https://webglfundamentals.org/webgl/resources/webgl-utils.js"></script>
I need interpolation when image zoomed or if set by maximum height like AMI exmaple provided below Check This sample
It's not clear what you want to happen.
First off you set gl.NEAREST as your filtering. WebGL has several kind of filtering covered here. Setting them to gl.LINEAR would be better but only
a little
The problem is WebGL 1.0 doesn't support mips for images that are not power of 2 dimensions (2, 4, 8, 16, 32, 128, 256, 512, 1024, etc...). That page describes what mips are used for (interpolation) but mips can only be used on images that are power of 2 dimensions. The image you're trying to display is not power of 2 dimensions, it's 954 × 687 .
You have a few different options.
Download the image, edit to be power of 2 in both dimensions in a photo editing application. Then call gl.generateMipmap to generate mips for interpolation as described in that page
Copy the image to canvas that's a power of 2 in size then upload the canvas as a texture
Create a texture that's the next largest power of 2 then upload your image
function nearestGreaterOrEqualPowerOf2(v) {
return Math.pow(2, Math.ceil(Math.log2(v)));
}
const newWidth = nearestGreaterOrEqualPowerOf2(image.width);
const newHeight = nearestGreaterOrEqualPowerOf2(image.height);
// first make an empty texture of the new size
const level = 0;
const format = gl.RGBA;
const type = gl.UNSIGNED_BYTE;
const border = 0;
gl.texImage2D(gl.TEXTURE_2D, level, format, newWidth, newHeight, border,
format, type, null);
// then upload the image into the bottom left corner of the texture
const xoffset = 0;
const yoffset = 0;
gl.texSubImage2D(gl.TEXTURE_2D, level, xoffset, yoffset, format, type, image);
// now because the texture is a power of 2 in both dimensions you can
// generate mips and turn on maximum filtering
gl.generateMipmap(gl.TEXTURE_2D);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
You have a new issue though in all these cases which is that the image is now just using a portion of the texture. You'd have to adjust your texture coordinates either using a texture matrix or by adjusting your texture coordinates directly.
// compute needed texture coordinates to show only portion of texture
var u = newWidth / image.width;
var v = newHeight / image.height;
// provide texture coordinates for the rectangle.
var texcoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
0, 0,
u, 0,
0, v,
0, v,
u, 0,
u, v,
]), gl.STATIC_DRAW);
"use strict";
function main() {
var image = new Image();
requestCORSIfNotSameOrigin(image, "https://upload.wikimedia.org/wikipedia/commons/5/57/Pneumothorax_CT.jpg")
image.src = "https://upload.wikimedia.org/wikipedia/commons/5/57/Pneumothorax_CT.jpg";
image.onload = function() {
render(image);
}
}
function render(image) {
// Get A WebGL context
/** #type {HTMLCanvasElement} */
var canvas = document.getElementById("canvas");
var gl = canvas.getContext("webgl");
if (!gl) {
return;
}
// setup GLSL program
var program = webglUtils.createProgramFromScripts(gl, ["2d-vertex-shader", "2d-fragment-shader"]);
// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");
var texcoordLocation = gl.getAttribLocation(program, "a_texCoord");
function nearestGreaterOrEqualPowerOf2(v) {
return Math.pow(2, Math.ceil(Math.log2(v)));
}
const newWidth = nearestGreaterOrEqualPowerOf2(image.width);
const newHeight = nearestGreaterOrEqualPowerOf2(image.height);
// Create a buffer to put three 2d clip space points in
var positionBuffer = gl.createBuffer();
// Bind it to ARRAY_BUFFER (think of it as ARRAY_BUFFER = positionBuffer)
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
// Set a rectangle fit in the canvas at the same aspect as the image.
const drawWidth = canvas.clientWidth;
const drawHeight = canvas.clientWidth / drawWidth * image.height;
setRectangle(gl, 0, 0, drawWidth, drawHeight);
// compute needed texture coordinates to show only portion of texture
var u = newWidth / image.width;
var v = newHeight / image.height;
// provide texture coordinates for the rectangle.
var texcoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
0, 0,
u, 0,
0, v,
0, v,
u, 0,
u, v,
]), gl.STATIC_DRAW);
// Create a texture.
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
// first make an empty texture of the new size
{
const level = 0;
const format = gl.RGBA;
const type = gl.UNSIGNED_BYTE;
const border = 0;
gl.texImage2D(gl.TEXTURE_2D, level, format, newWidth, newHeight, border,
format, type, null);
// then upload the image into the bottom left corner of the texture
const xoffset = 0;
const yoffset = 0;
gl.texSubImage2D(gl.TEXTURE_2D, level, xoffset, yoffset, format, type, image);
}
// now because the texture is a power of 2 in both dimensions you can
// generate mips and turn on maximum filtering
gl.generateMipmap(gl.TEXTURE_2D);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
// lookup uniforms
var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
webglUtils.resizeCanvasToDisplaySize(gl.canvas);
// Tell WebGL how to convert from clip space to pixels
gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
// Clear the canvas
gl.clearColor(0, 0, 0, 0);
gl.clear(gl.COLOR_BUFFER_BIT);
// Tell it to use our program (pair of shaders)
gl.useProgram(program);
// Turn on the position attribute
gl.enableVertexAttribArray(positionLocation);
// Bind the position buffer.
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
// Tell the position attribute how to get data out of positionBuffer (ARRAY_BUFFER)
var size = 2; // 2 components per iteration
var type = gl.FLOAT; // the data is 32bit floats
var normalize = false; // don't normalize the data
var stride = 0; // 0 = move forward size * sizeof(type) each iteration to get the next position
var offset = 0; // start at the beginning of the buffer
gl.vertexAttribPointer(
positionLocation, size, type, normalize, stride, offset)
// Turn on the teccord attribute
gl.enableVertexAttribArray(texcoordLocation);
// Bind the position buffer.
gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
// Tell the position attribute how to get data out of positionBuffer (ARRAY_BUFFER)
var size = 2; // 2 components per iteration
var type = gl.FLOAT; // the data is 32bit floats
var normalize = false; // don't normalize the data
var stride = 0; // 0 = move forward size * sizeof(type) each iteration to get the next position
var offset = 0; // start at the beginning of the buffer
gl.vertexAttribPointer(
texcoordLocation, size, type, normalize, stride, offset)
// set the resolution
gl.uniform2f(resolutionLocation, gl.canvas.width, gl.canvas.height);
// Draw the rectangle.
var primitiveType = gl.TRIANGLES;
var offset = 0;
var count = 6;
gl.drawArrays(primitiveType, offset, count);
}
function setRectangle(gl, x, y, width, height) {
var x1 = x;
var x2 = x + width;
var y1 = y;
var y2 = y + height;
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
x1, y1,
x2, y1,
x1, y2,
x1, y2,
x2, y1,
x2, y2,
]), gl.STATIC_DRAW);
}
main();
// This is needed if the images are not on the same domain
// NOTE: The server providing the images must give CORS permissions
// in order to be able to use the image with WebGL. Most sites
// do NOT give permission.
// See: http://webglfundamentals.org/webgl/lessons/webgl-cors-permission.html
function requestCORSIfNotSameOrigin(img, url) {
if ((new URL(url)).origin !== window.location.origin) {
img.crossOrigin = "";
}
}
#import url("https://webglfundamentals.org/webgl/resources/webgl-tutorials.css");
body {
margin: 0;
}
canvas {
width: 100vw;
height: 100vh;
display: block;
}
<div style="height:700px; width:700px; overflow:scroll;">
<canvas id="canvas"></canvas>
</div>
<!-- vertex shader -->
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
attribute vec2 a_texCoord;
uniform vec2 u_resolution;
varying vec2 v_texCoord; void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
// pass the texCoord to the fragment shader
// The GPU will interpolate this value between points.
v_texCoord = a_texCoord;
}
</script>
<!-- fragment shader -->
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
// our texture
uniform sampler2D u_image;
// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;
void main() {
// Look up a color from the texture.
gl_FragColor = texture2D(u_image, v_texCoord);
}
</script>
<script src="https://webglfundamentals.org/webgl/resources/webgl-utils.js"></script>
Already really informative and good answers and comments here.
Also please take into account, that the link you provided using high quality images with high-res and excellent quality, at least with no compression artifacts.
Unpacked ~21mb in NIFTI Data Format.
using ami.js to uppack it:
https://github.com/FNNDSC/ami
Using the exampel by gman with a good image resolution that fits with your screen resolution, should give you a descend result.
Yes, their are some algorithms to fix a bad images quality and deal with image compression artifacts, but (and i don't whant to repeat the comments here) generally speaking once the information is lost, it is gone.

copyTexImage2D — GL ERROR :GL_INVALID_OPERATION : glCopyTexImage2D:

I’m trying to figure out how to use gl.copyTexImage2D function.
I have the following code (unwieldy though):
//--------- SETUP (not important) ---------------
//I use budo package to easily run browserify
var createContext = require('webgl-context');
var createShader = require('gl-shader');
//↓ here is webgl setup usual routine, using floats and simple one-triangle vertex shader
var gl = createContext({width: 2, height: 2});
gl.getExtension('OES_texture_float');
gl.getExtension('OES_texture_float_linear');
var shader = createShader(gl, `
precision mediump float;
attribute vec2 position;
varying vec2 uv;
void main (void) {
gl_Position = vec4(position, 0, 1);
uv = vec2(position.x * 0.5 + 0.5, position.y * 0.5 + 0.5);
}
`, `
precision mediump float;
uniform sampler2D image;
varying vec2 uv;
void main (void) {
gl_FragColor = texture2D(image, uv);
}
`);
//fullscreen triangle
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, -1, 3, 3, -1]), gl.STATIC_DRAW);
shader.attributes.position.pointer();
//textures
var outTexture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, outTexture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 2, 2, 0, gl.RGBA, gl.FLOAT, new Float32Array([1,1,1,1, 0,0,0,1, 0,0,0,1, 0,0,0,1]));
var sourceTexture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, sourceTexture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 2, 2, 0, gl.RGBA, gl.FLOAT, new Float32Array([0,0,0,1, 1,1,1,1, 0,0,0,1, 0,0,0,1]));
//--------------- MAIN PART (important) ---------------
//then I setup custom framebuffer ↓
var framebuffer = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, outTexture, 0);
gl.bindTexture(gl.TEXTURE_2D, sourceTexture);
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer);
gl.drawArrays(gl.TRIANGLES, 0, 3);
//here ↓ I am expecting to copy framebuffer’s output, which is `outTexture`, to `sourceTexture`
gl.bindTexture(gl.TEXTURE_2D, sourceTexture);
gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, w, h, 0);
//then I try to render shader again, with it’s own output as input
gl.drawArrays(gl.TRIANGLES, 0, 3);
//when I try to read pixels here ↓ I get the error below
gl.readPixels(0, 0, w, h, gl.RGBA, gl.FLOAT, pixels);
The error: [.CommandBufferContext]GL ERROR :GL_INVALID_OPERATION : glCopyTexImage2D:
Cannot figure out what do I do wrong.
The error is from gl.copyTexImage2D not from gl.readPixels. The reason you don't see it until calling gl.readPixels is because WebGL is a command driven language. Commands are not executed until they have to be for various reasons. gl.flush will force the commands to be executed at some point. gl.readPixels also forces the commands to be executed since the results of the commands needed to be used to read the pixels.
As for the error you need to provide more code. The code as is works through gl.copyTexImage2D which means the error you're getting from that has to do with some code your not showing. Either you created your textures wrong or w and h or funky values or something
Trying it out myself below it works but pointed out another error. You can't read floats with gl.readPixels in WebGL. Switching to UNSIGNED_BYTE works
var gl = document.getElementById("c").getContext("webgl");
var w = 300;
var h = 150;
var programInfo = twgl.createProgramInfo(gl, ["vs", "fs"]);
gl.useProgram(programInfo.program);
var arrays = {
position: [-1, -1, 0, 1, -1, 0, -1, 1, 0, -1, 1, 0, 1, -1, 0, 1, 1, 0],
};
var bufferInfo = twgl.createBufferInfoFromArrays(gl, arrays);
twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
// make a renderable npot texture
function createRenderableTexture(gl, w, h) {
var tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
return tex;
}
var outTexture = createRenderableTexture(gl, w, h);
var framebuffer = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, outTexture, 0);
// render something to it
gl.clearColor(0,1,0,1); // green
gl.clear(gl.COLOR_BUFFER_BIT);
// copy the framebuffer to the texture
var sourceTexture = createRenderableTexture(gl, w, h)
gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, w, h, 0);
// draw to canvas
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
// clear to red
gl.clearColor(1,0,0,1);
gl.clear(gl.COLOR_BUFFER_BIT);
// Since we cleared to red and the texture is filled with green
// the result should be green
gl.drawArrays(gl.TRIANGLES, 0, 6);
var pixels = new Uint8Array(w * h * 4);
gl.readPixels(0, 0, w, h, gl.RGBA, gl.UNSIGNED_BYTE, pixels);
log("pixel 0: "
+ pixels[0] + ","
+ pixels[1] + ","
+ pixels[2] + ","
+ pixels[3]);
function log(msg) {
var div = document.createElement("pre");
div.appendChild(document.createTextNode(msg));
document.body.appendChild(div);
}
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vs" type="notjs">
attribute vec4 position;
varying vec2 v_uv;
void main() {
gl_Position = position;
v_uv = position.xy * 0.5 + 0.5;
}
</script>
<script id="fs" type="notjs">
precision mediump float;
varying vec2 v_uv;
uniform sampler2D u_texture;
void main() {
gl_FragColor = texture2D(u_texture, v_uv);
}
</script>
<canvas id="c"></canvas>
On the other hand FLOAT textures should not work. They worked for me but as far as I can tell that's a bug on my system not yours.
Specifically FLOAT textures are not allowed by default in WebGL. You can turn them on by enabling the OES_texture_float extension but that extension specifically says
Should CopyTexImage2D, CopyTexSubImage{2D|3D} be supported for textures with half-float and float channels?
RESOLUTION: No.
I filed a bug for Chrome
You can try to work around that issue by implementing copyTexImage2D yourself. Basically attach the texture you want to copy as a source texture to some shader and quad. Attach the texture you want to copy to to a framebuffer object. Since it's a FLOAT texture you need to call checkFramebufferStatus and check it returns FRAMEBUFFER_COMPLETE to make sure your GPU/Driver actually supports floating point textures as attachments. Then draw your quad with a simple shader effectively copying your texture from src to dst.

How to render binary data on canvas using WebGl?

I am using PNaCl ffmpeg to open, read and decode RTSP stream. I am now having raw video frames which I need to transfer to WebGl to render on the canvas.
How can I render binary data on the canvas?
I am running the following code: I presume that I should get a grey canvas after running this code, because I am passing RGBA values of (120,120,120,1) to the synthetic data.
var canvas = document.getElementById('canvas');
var gl = initWebGL(canvas); //function initializes webgl
initViewport(gl, canvas); //initializes view port
console.log('viewport initialized');
var data = [];
for (var i = 0 ; i < 256; i++){
data.push(120,120,120,1.0);
}
console.log(data);
var pixels = new Uint8Array(data); // 16x16 RGBA image
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(
gl.TEXTURE_2D, // target
0, // mip level
gl.RGBA, // internal format
16, 16, // width and height
0, // border
gl.RGBA, //format
gl.UNSIGNED_BYTE, // type
pixels // texture data
);
console.log('pixels');
console.log(pixels);
<canvas id="canvas"></canvas>
I should get a grey 16x16 box being represented on the canvas, but I am not getting that. What additional steps do I need to take to correctly render the 2D bitmap on the canvas?
PS. I am taking help from this article.
Console output:
As pointed out in the comments, alpha in WebGL in the type of texture you're creating is 0 to 255. You're putting in 1.0 which = 1/255 or an alpha of 0.004
But on top of that you say
I am running the following code: I presume that I should get a grey canvas after running this code
That code is not enough for WebGL. WebGL requires you to supply a vertex shader and fragment shader, vertex data for vertices and then call either gl.drawArrays or gl.drawElements to render something. The code you provided doesn't do those things and without those things we can't tell what else you're doing.
You're also only supplying mip level 0. You either need to supply mips or set texture filtering so only the first level is used otherwise the texture is unrenderable (you'll get a warning about it in the JavaScript console of most browsers).
Here's a working example
var canvas = document.getElementById('canvas');
var gl = canvas.getContext("webgl");
var data = [];
for (var i = 0 ; i < 256; i++){
data.push(120,120,120,255);
}
var pixels = new Uint8Array(data); // 16x16 RGBA image
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(
gl.TEXTURE_2D, // target
0, // mip level
gl.RGBA, // internal format
16, 16, // width and height
0, // border
gl.RGBA, //format
gl.UNSIGNED_BYTE, // type
pixels // texture data
);
gl.generateMipmap(gl.TEXTURE_2D); // you need to do this or set filtering
// compiles and links the shaders and looks up uniform and attribute locations
var programInfo = twgl.createProgramInfo(gl, ["vs", "fs"]);
var arrays = {
position: [
-1, -1, 0,
1, -1, 0,
-1, 1, 0,
-1, 1, 0,
1, -1, 0,
1, 1, 0,
],
};
// calls gl.createBuffer, gl.bindBuffer, gl.bufferData for each array
var bufferInfo = twgl.createBufferInfoFromArrays(gl, arrays);
var uniforms = {
u_texture: texture,
};
gl.useProgram(programInfo.program);
// calls gl.bindBuffer, gl.enableVertexAttribArray, gl.vertexAttribPointer
twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
// calls gl.activeTexture, gl.bindTexture, gl.uniformXXX
twgl.setUniforms(programInfo, uniforms);
// calls gl.drawArrays or gl.drawElements
twgl.drawBufferInfo(gl, gl.TRIANGLES, bufferInfo);
canvas { border: 1px solid black; }
<script id="vs" type="notjs">
attribute vec4 position;
varying vec2 v_texcoord;
void main() {
gl_Position = position;
// Since we know we'll be passing in -1 to +1 for position
v_texcoord = position.xy * 0.5 + 0.5;
}
</script>
<script id="fs" type="notjs">
precision mediump float;
uniform sampler2D u_texture;
varying vec2 v_texcoord;
void main() {
gl_FragColor = texture2D(u_texture, v_texcoord);
}
</script>
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<canvas id="canvas"></canvas>

Speeding-up Indexed Color images operations, pseudo-palette shifting and color cycling techniques in HTML5 canvas [duplicate]

Currently, I'm using 2D canvas context to draw an image generated (from pixel to pixel, but refreshed as a whole buffer in once after a generated frame) from JavaScript at about a 25fps rate. The generated image is always one byte (integer / typed array) per pixel and a fixed palette is used to generate RGB final result. Scaling is also needed to adopt to the size of the canvas (ie: going to fullscreen) and/or at user request (zoom in/out buttons).
The 2D context of canvas is OK for this purpose, however I'm curious if WebGL can provide better result and/or better performance. Please note: I don't want to put pixels via webGL, I want to put pixels into my buffer (which is basically Uint8Array), and use that buffer (in once) to refresh the context. I don't know too much about WebGL, but using the needed generated image as some kind of texture would work somehow for example? Then I would need to refresh the texture at about 25fps rate, I guess.
It would be really fantastic, if WebGL support the colour space conversion somehow. With 2D context, I need to convert 1 byte / pixel buffer into RGBA for the imagedata in JavaScript for every pixel ... Scaling (for 2D context) is done now by altering the height/width style of the canvas, so browsers scales the image then. However I guess it can be slower than what WebGL can do with hw support, and also (I hope) WebGL can give greater flexibility to control the scaling, eg with the 2D context, browsers will do antialiasing even if I don't want to do (eg: integer zooming factor), and maybe that's a reason it can be quite slow sometimes.
I've already tried to learn several WebGL tutorials but all of them starts with objects, shapes, 3D cubes, etc, I don't need any - classical - object to render only what 2D context can do as well - in the hope that WebGL can be a faster solution for the very same task! Of course if there is no win here with WebGL at all, I would continue to use 2D context.
To be clear: this is some kind of computer hardware emulator done in JavaScript, and its output (what would be seen on a PAL TV connected to it) is rendered via a canvas context. The machine has fixed palette with 256 elements, internally it only needs one byte for a pixel to define its colour.
You can use a texture as your palette and a different texture as your image. You then get a value from the image texture and use it too look up a color from the palette texture.
The palette texture is 256x1 RGBA pixels. Your image texture is any size you want but just a single channel ALPHA texture. You can then look up a value from the image
float index = texture2D(u_image, v_texcoord).a * 255.0;
And use that value to look up a color in the palette
gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
Your shaders might be something like this
Vertex Shader
attribute vec4 a_position;
varying vec2 v_texcoord;
void main() {
gl_Position = a_position;
// assuming a unit quad for position we
// can just use that for texcoords. Flip Y though so we get the top at 0
v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
}
Fragment shader
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
void main() {
float index = texture2D(u_image, v_texcoord).a * 255.0;
gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
Then you just need a palette texture.
// Setup a palette.
var palette = new Uint8Array(256 * 4);
// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
palette[index * 4 + 0] = r;
palette[index * 4 + 1] = g;
palette[index * 4 + 2] = b;
palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue
// upload palette
...
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA,
gl.UNSIGNED_BYTE, palette);
And your image. It's an alpha only image so just 1 channel.
// Make image. Just going to make something 8x8
var image = new Uint8Array([
0,0,1,1,1,1,0,0,
0,1,0,0,0,0,1,0,
1,0,0,0,0,0,0,1,
1,0,2,0,0,2,0,1,
1,0,0,0,0,0,0,1,
1,0,3,3,3,3,0,1,
0,1,0,0,0,0,1,0,
0,0,1,1,1,1,0,0,
]);
// upload image
....
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA,
gl.UNSIGNED_BYTE, image);
You also need to make sure both textures are using gl.NEAREST for filtering since one represents indices and the other a palette and filtering between values in those cases makes no sense.
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
Here's a working example:
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");
// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
gl,
["vshader", "fshader"],
["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);
// Setup a unit quad
var positions = [
1, 1,
-1, 1,
-1, -1,
1, 1,
-1, -1,
1, -1,
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
// Setup a palette.
var palette = new Uint8Array(256 * 4);
// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
palette[index * 4 + 0] = r;
palette[index * 4 + 1] = g;
palette[index * 4 + 2] = b;
palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue
// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);
// Make image. Just going to make something 8x8
var image = new Uint8Array([
0,0,1,1,1,1,0,0,
0,1,0,0,0,0,1,0,
1,0,0,0,0,0,0,1,
1,0,2,0,0,2,0,1,
1,0,0,0,0,0,0,1,
1,0,3,3,3,3,0,1,
0,1,0,0,0,0,1,0,
0,0,1,1,1,1,0,0,
]);
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
canvas { border: 1px solid black; }
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
attribute vec4 a_position;
varying vec2 v_texcoord;
void main() {
gl_Position = a_position;
// assuming a unit quad for position we
// can just use that for texcoords. Flip Y though so we get the top at 0
v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
}
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
void main() {
float index = texture2D(u_image, v_texcoord).a * 255.0;
gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
To animate just update the image and then re-upload it into the texture
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA,
gl.UNSIGNED_BYTE, image);
Example:
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");
// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
gl,
["vshader", "fshader"],
["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);
// Setup a unit quad
var positions = [
1, 1,
-1, 1,
-1, -1,
1, 1,
-1, -1,
1, -1,
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
// Setup a palette.
var palette = new Uint8Array(256 * 4);
// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
palette[index * 4 + 0] = r;
palette[index * 4 + 1] = g;
palette[index * 4 + 2] = b;
palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue
// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);
// Make image. Just going to make something 8x8
var width = 8;
var height = 8;
var image = new Uint8Array([
0,0,1,1,1,1,0,0,
0,1,0,0,0,0,1,0,
1,0,0,0,0,0,0,1,
1,0,2,0,0,2,0,1,
1,0,0,0,0,0,0,1,
1,0,3,3,3,3,0,1,
0,1,0,0,0,0,1,0,
0,0,1,1,1,1,0,0,
]);
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
var frameCounter = 0;
function render() {
++frameCounter;
// skip 3 of 4 frames so the animation is not too fast
if ((frameCounter & 3) == 0) {
// rotate the image left
for (var y = 0; y < height; ++y) {
var temp = image[y * width];
for (var x = 0; x < width - 1; ++x) {
image[y * width + x] = image[y * width + x + 1];
}
image[y * width + width - 1] = temp;
}
// re-upload image
gl.activeTexture(gl.TEXTURE0);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA,
gl.UNSIGNED_BYTE, image);
gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
}
requestAnimationFrame(render);
}
render();
canvas { border: 1px solid black; }
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
attribute vec4 a_position;
varying vec2 v_texcoord;
void main() {
gl_Position = a_position;
// assuming a unit quad for position we
// can just use that for texcoords. Flip Y though so we get the top at 0
v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
}
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
void main() {
float index = texture2D(u_image, v_texcoord).a * 255.0;
gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
Of course that assumes your goal is to do the animation on the CPU by manipulating pixels. Otherwise you can use any normal webgl techniques to manipulate texture coordinates or whatever.
You can also update the palette similarly for palette animation. Just modify the palette and re-upload it
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA,
gl.UNSIGNED_BYTE, palette);
Example:
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");
// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
gl,
["vshader", "fshader"],
["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);
// Setup a unit quad
var positions = [
1, 1,
-1, 1,
-1, -1,
1, 1,
-1, -1,
1, -1,
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
// Setup a palette.
var palette = new Uint8Array(256 * 4);
// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
palette[index * 4 + 0] = r;
palette[index * 4 + 1] = g;
palette[index * 4 + 2] = b;
palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue
// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);
// Make image. Just going to make something 8x8
var width = 8;
var height = 8;
var image = new Uint8Array([
0,0,1,1,1,1,0,0,
0,1,0,0,0,0,1,0,
1,0,0,0,0,0,0,1,
1,0,2,0,0,2,0,1,
1,0,0,0,0,0,0,1,
1,0,3,3,3,3,0,1,
0,1,0,0,0,0,1,0,
0,0,1,1,1,1,0,0,
]);
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
var frameCounter = 0;
function render() {
++frameCounter;
// skip 3 of 4 frames so the animation is not too fast
if ((frameCounter & 3) == 0) {
// rotate the 3 palette colors
var tempR = palette[4 + 0];
var tempG = palette[4 + 1];
var tempB = palette[4 + 2];
var tempA = palette[4 + 3];
setPalette(1, palette[2 * 4 + 0], palette[2 * 4 + 1], palette[2 * 4 + 2], palette[2 * 4 + 3]);
setPalette(2, palette[3 * 4 + 0], palette[3 * 4 + 1], palette[3 * 4 + 2], palette[3 * 4 + 3]);
setPalette(3, tempR, tempG, tempB, tempA);
// re-upload palette
gl.activeTexture(gl.TEXTURE1);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA,
gl.UNSIGNED_BYTE, palette);
gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
}
requestAnimationFrame(render);
}
render();
canvas { border: 1px solid black; }
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
attribute vec4 a_position;
varying vec2 v_texcoord;
void main() {
gl_Position = a_position;
// assuming a unit quad for position we
// can just use that for texcoords. Flip Y though so we get the top at 0
v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
}
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
void main() {
float index = texture2D(u_image, v_texcoord).a * 255.0;
gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
Slightly related is this tile shader example
http://blog.tojicode.com/2012/07/sprite-tile-maps-on-gpu.html
Presumably you're building up a javascript array that's around 512 x 512 (PAL size)...
A WebGL fragment shader could definitely do your palette conversion pretty nicely. The recipe would go something like this:
Set up WebGL with a "geometry" of just two triangles that span your viewport. (GL is all triangles.) This is the biggest bother, if you're not already GL fluent. But it's not that bad. Spend some quality time with http://learningwebgl.com/blog/?page_id=1217 . But it will be ~100 lines of stuff. Price of admission.
Build your in-memory frame buffer 4 times bigger. (I think textures always have to be RGBA?) And populate every fourth byte, the R component, with your pixel values. Use new Float32Array to allocate it. You can use values 0-255, or divide it down to 0.0 to 1.0. We'll pass this to webgl as a texture. This one changes every frame.
Build a second texture that's 256 x 1 pixels, which is your palette lookup table. This one never changes (unless the palette can be modified?).
In your fragment shader, use your emulated frame buffer texture as a lookup into your palette. The first pixel in the palette is accessed at location (0.5/256.0, 0.5), middle of the pixel.
On each frame, resubmit the emulated frame buffer texture and redraw. Pushing pixels to the GPU is expensive... but a PAL-sized image is pretty small by modern standards.
Bonus step: You could enhance the fragment shader to imitate scanlines, interlace video, or other cute emulation artifacts (phosphor dots?) on modern high-resolution displays, all at no cost to your javascript!
This is just a sketch. But it will work. WebGL is a pretty low-level API, and quite flexible, but well worth the effort (if you like that kind of thing, which I do. :-) ).
Again, http://learningwebgl.com/blog/?page_id=1217 is well-recommended for overall WebGL guidance.

Categories