Multi-dimensional input in Keras.js - javascript

I am looking to implement my Keras model into my website with the Keras.js library. One problem with this library is that when inputting data in javascript, only a Float32Array() is allowed as an input. This type of array is 1D, and in my Keras model, the input is 3D. I have asked on the Keras.js issues page and found some potential solutions such as adding an embedding layer, but that requires a specific input shape, but I would like to have any 3D input work, as it did when I trained the model. The model structure is simple, there is an input layer which takes an array of dimension mxnx3 (it is an image of unknown size with r, g, and b channels) and a Conv2D layer, which then outputs an mxnx1 array. I know the model works because it can give a good output based on an input, so the only problem I have is with the transition to Keras.js. Here is the JS code that I have at the moment.
function predictImageWithCNN(data) { //'data' is mxnx3 array
var final = [];
//instantiating model from json and buf files
var model = new KerasJS.Model({
filepaths: {
model: 'dist/model.json',
weights: 'dist/model_weights.buf',
metadata: 'dist/model_metadata.json'
},
gpu: true //MAY NEED TO CHANGE (NOT SURE REALLY)
});
//Ready the model.
model.ready()
.then(function() {
//This matches our input data with the input key (b/c Sequential Model)
var inputData = {
'input_1': new Float32Array(data)
};
// make predictions based on inputData
return model.predict(inputData);
})
.then(function(outputData) {
//Here we take the outputData and parse it to get a result.
var out = outputData['output']; //Gets output data
console.log(out);
//TODO: Put in code to assign outputData to 'final' so we can then convert it
// This should not be too hard to do.
})
.catch(function(err) {
console.log(err);
// handle error
});
return final; // should return nxmx1 array of vals 0-1.
}
If anyone had any suggestions for how to resolve this, that would be very much appreciated. Thanks! :)

I had the same problem with an LSTM. The way I got around this was by training using a flattened version of the data but using a reshape layer as the first layer to get it to the shape I needed for my LSTM. eg.
model = Sequential()
model.add(Reshape((40,59),input_shape=(1,2360)))
model.add(LSTM(128, input_shape=(maxlen, len(chars))))
model.add(Dense(len(chars)))
model.add(Activation('softmax'))
optimizer = RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)
Then in Keras.JS I can feed in the flattened version from Float32Array

Related

Why does large eojson not get displayed on Here Map?

I'm trying to parse and display a geojson representation of the UK postcode districts on a Here Map using the JavaScript API within a VueJs application.
The code is relatively simple - districtGeojson is the JSON document. The following function is called after the map has been initialised and displayed:
processGeojson() {
const reader = new H.data.geojson.Reader(districtGeojson, {
disableLegacyMode: true
});
reader.parse();
const layer = reader.getLayer();
this.shapes = layer;
console.log(layer.isValid(7));
try {
this.map.addLayer(layer);
} catch (err) {
console.log('err adding layer', err);
}
}
As you can see, there's a console.log() in there to do some kind of checking on the layer's validity at the default zoom level and it returns true.
All I see is the map flicker briefly and then the plain map is shown. Is there a way to get some feedback from the API on what is going wrong, it seems to just fail silently - addLayer throws no exception?
If necessary, I can share the JSON document but as it's large (5Mb) I wanted to see if there was anything obviously wrong with this code first.
So, it appears there may be an error in the documentation here: https://developer.here.com/documentation/maps/3.1.17.0/api_reference/H.data.geojson.Reader.html#getLayer
which states:
var reader = new H.data.geojson.Reader('/path/to/geojson/file.json');
reader.parse();
// Assumption: map already exists
map.addLayer(reader.getLayer());
Whereas what just worked for me was to call parseData() rather than parse(), passing the source data. I can do:
var reader = new H.data.geojson.Reader('');
reader.parseData(districtGeojson);
// Assumption: map already exists
map.addLayer(reader.getLayer());
But not
var reader = new H.data.geojson.Reader('./Districts.json');
reader.parse();
// Assumption: map already exists
map.addLayer(reader.getLayer());
Which begs the question of why the file path is required in the constructor as I cannot get the reader to parse data that is passed as a path in the constructor.

How to truncate coco ssd model in tensorflow.js?

ML / Tensorflow beginner.
I'm having trouble trying to get one of the layers from the coco ssd model imported as a package in a React application. I'm following the Pacman tensorflow.js example to retrain the model.
const modelPromise = cocoSsd.load();
Promise.all([modelPromise])
.then(cocoModel => {
console.log(cocoModel[0]);
var cocoModel = cocoModel[0].model;
console.log(cocoModel);
const layer = cocoModel.getLayer('conv_pw_13_relu');
this.truncatedCocoModel = tf.model({inputs: cocoModel.inputs, outputs:
layer.output});
})
.catch(error => {
console.error(error);
});
In the const layer line I get the error message that 'cocoModel.getLayer is not a function'. The Pacman example is using the mobilenet model which I guess has this function.
What are my options here? I looked around using the browser console but I can't find this function anywhere and looking online didn't help much (is there any place online where I can see the whole structure of the cocoSSD model by Google?)
Using the npm package https://cdn.jsdelivr.net/npm/#tensorflow-models/coco-ssd, you cannot retrieve any layer.
load returns an instance of ObjectDetection which does not have the getLayer property.
If you want to retrieve the layer, you would have to load the graph model as described here

How to read webgl GL.bufferData in javascript

i want to read back the data stored in the GL.bufferData array in javascript.
Here is my code
var TRIANGLE_VERTEX = geometryNode["triangle_buffer"];
GL.bindBuffer(GL.ARRAY_BUFFER, TRIANGLE_VERTEX);
GL.bufferData(GL.ARRAY_BUFFER,new Float32Array(vertices),GL.STATIC_DRAW);
is it possible in webgl to read back the bufferdata in GPU?
if possible then please explain me with a sample code.
How to know the memory size(filled and free) of the Gpu in webgl at run time and how to debug the shader code and data in GPU in webgl.
It is not directly possible to read the data back in WebGL1. (see below for WebGL2). This is limitation of OpenGL ES 2.0 on which WebGL is based.
There are some workarounds:
You could try to render that data to a texture then use readPixels to read the data.
You'd have to encode the data into bytes in your shader because readPixels in WebGL can only read bytes
You can wrap your WebGL to store the data yourself something like
var buffers = {};
var nextId = 1;
var targets = {};
function copyBuffer(buffer) {
// make a Uint8 view of buffer in case it's not already
var view = new Uint8Buffer(buffer.buffer);
// now copy it
return new UintBuffer(view);
}
gl.bindBuffer = function(oldBindBufferFn) {
return function(target, buffer) {
targets[target] = new Uint8Buffer(buffer.buffer);
oldBindBufferFn(target, buffer);
};
}(gl.bindBuffer.bind(gl));
gl.bufferData = function(oldBufferDataFn) {
return function(target, data, hint) {
var buffer = targets[target];
if (!buffer.id) {
buffer.id = nextId++;
}
buffers[buffer.id] = copyBuffer(data);
oldBufferDataFn(target, data, hint);
};
}(gl.bufferData.bind(gl)));
Now you can get the data with
data = buffers[someBuffer.id];
This is probably what the WebGL Inspector does
Note that there are a few issues with the code above. One it doesn't check for errors. Checking for errors would make it way slower but not checking for error will give you incorrect results if your code generates errors. A simple example
gl.bufferData(someBuffer, someData, 123456);
This would generate an INVALID_ENUM error and not update the data in someBuffer but our code isn't checking for errors so it would have put someData into buffers and if you read that data it wouldn't match what's in WebGL.
Note the code above is pseudo code. For example I didn't supply a wrapper for gl.bufferSubData.
WebGL2
In WebGL2 there is a function gl.getBufferSubData that will allow you to read the contents of a buffer. Note that WebGL2, even though it's based on OpenGL ES 3.0 does not support gl.mapBuffer because there is no performant and safe way to expose that function.

Realtime data and Openlayers 3: Batch moving features

I have a hashtable I am trying to use in an vector layer so this is what I made:
var make = [], remove = [];
for (var key in data) {
var val = data[key];
if (featureCache[key]) {
if (val._ts > featureCache[key]._ts) {
var geom = featureCache[key].getGeometry();
The problem starts here. From what I have found out so far I have two options:
geom.setCoordinates(latlng([val.Latitude, val.Longitude]));
geom.flatCoordinates = latlng([val.Latitude, val.Longitude]);
However the first one bubbles up and makes the map render, this is a problem because I have over 1500 features and I plan on using more.
The second line gives me assertion failed, and I dont know how to debug it.
featureCache[key]._ts = val._ts;
}
} else {
make.push(featureCache[key] =
new ol.Feature({
geometry: new ol.geom.Point(latlng([val.Latitude,val.Longitude]))
})
);
featureCache[key]._ts = val._ts;
}
}
source.addFeatures(make);
This needs to be run every other second or so because I want the data to be as realtime as possible.
And if this is a really stupid way do do it, I would like to know that too.
I am not sure how you could do it with the new API, but you could try this:
clone an old geometry and update it, do this for each
add the clone to a collection that OL is not aware of (avoids the map render / updates)
once that collection is ready, remove the old collection from OL and set the new (batch)
Doing this for several thousand items should work pretty fast on 'modern' browsers as you set it only once. The key point is to avoid triggering updates while you update each entry.
Maybe there are ways to avoid the bubbling/events to allow for batch updates (no triggering of map render for each change) ?

Modify Cubism.js Graphite Title Text

Using cubism.js I'm snagging graphite data and creating multiple graphs on page. Looks amazing but I cannot figure out how to modify the default title/text of each graph. Very limited JS experience.
This might be a cubusm.js, d3.js, or general javascript question, I'm not sure. Since graphite nests data within sometimes deep folders structures, I'd like to be able to simplify default the string a bit. Example of text I want to modify ('servers.apt1.loadavg.05', 'servers.apt2.loadavg.05', etc): http://i.imgur.com/4FqwhjA.png
How do I modify the title/text of each graph's Graphite data? Getting "servers.apt1.loadavg.05, want "apt1" displayed.
var context = cubism.context()
.step( 1 * 30 * 1000 )
.size(960);
var graphite = context.graphite("http://graphite.example.com");
graphFind = 'servers.*.loadavg.05'
graphite.find(graphFind, function(error, results) {
// Map find results to array and set to graphite.metric object type
var metrics = results.sort().map(function(i) {
return graphite.metric(i);
});
// loop through array and print stuff to "graphs" div
for (var i=0;i<metrics.length;i++){
d3.select("#graphs").call(function(div) {
div.append("div")
.selectAll(".horizon")
.data([metrics[i]])
.enter()
.append("div")
.attr("class", "horizon")
.call(context.horizon());
});
}
});
According to the documentation, you can do this with the horizon.title() function. So in your case, it would be something like
.call(context.horizon().title("your title here"));
You can also specify this as a function, e.g.
.call(context.horizon().title(function(d) { return d.title; }));

Categories