Unable to load python trained model in tensorflow.js - javascript

I am getting the following errors when I am trying to load a Model, which I trained in Python, when I use the loadModel() function tensorflow.js:
Failed to load resource: net::ERR_NAME_NOT_RESOLVED
Uncaught (in promise) TypeError: Failed to fetch
Below is the predict.js file
console.log ("hello");
let model;
(async function () {
model = await tf.loadModel("http://keras_model/model.json");
$(".progress-bar").hide();
console.log("it works");
})();
The directory structure:
main
-dataset (contains images for training the model)
-training_scripts (python scripts to train the model)
-user_interface
--server.js (server made using node.js(and express))
--static (this folder contains the trained keras model)
--index.html (html file to be served)
--predict.js
--keras_model(this folder contains the model.json file)
Any help will be appreciated!!

If you want to load local files in tfjs you need to use the file type file:/// and for this to work you need the node extension of tfjs. You can load it by installing and requiring node-fetch into your program.

You can also use the fileSystem handler exposed in tfjs-node like this:
const tf = require("#tensorflow/tfjs");
const tfn = require("#tensorflow/tfjs-node");
const handler = tfn.io.fileSystem("./path/to/your/model.json");
const model = await tf.loadModel(handler);

Related

ASP .NET Core .glb file not found whereas the file was copy

Hello
I want to import a .glb file but when I call it in JS I get an error which tell me that the file isn't found whereas the file was copied.
Here's my files:
wwwroot
/-Assets
/---Element
/---Objects3D
/-----hub.glb
Here, the error:
Failed to load resource: the server responded with a status of 404 () :44353/Assets/Objects3D/hub.glb:1
Have you an idea ?
Thank you !
Ok so with some little investigation I was able to solve my issue, this might help you solve your problem too.
In the server side when setting the UseStaticFiles you should add some options to allow UnkownType files to be accessed.
There are two ways you can solve this. The first is especifying the type of the file that you wish to provide that way. In this case you should do it this way on you Startup.cs file, Configure function:
StaticFileOptions options = new StaticFileOptions { ContentTypeProvider = new FileExtensionContentTypeProvider() };
((FileExtensionContentTypeProvider)options.ContentTypeProvider).Mappings.Add(new KeyValuePair<string, string>(".glb", "model/gltf-buffer"));
app.UseStaticFiles(options);
The second option (and more appropriate if you have more than 1 Unkown file type) is to allow any UnkownTypeFile using a flag on the same StaticFileOptions at the same Configure function:
StaticFileOptions options = new StaticFileOptions { ContentTypeProvider = new FileExtensionContentTypeProvider() };
options.ServeUnknownFileTypes = true;
app.UseStaticFiles(options);
Also, make sure you are able to access the subdomain directories on you server. If you are not, then you should also include this in your ConfigureServices function in the Startup.cs file:
services.AddDirectoryBrowser();
As well as the line bellow on your Configure function at the same file:
app.UseFileServer(enableDirectoryBrowsing: true);
Note: This solution is for a server using .Net Core 3.1. If you are using a different version or .Net Framework there might be a different fix.

Electron doesn't add json file to application

This is my first electron/node application, I m trying to use a json file as a datastore. so I created a simple one index.json under the app folder next to index.js|css|html
I installed a npm package jsonfile that is loading just fine
When I try to load my json file the EOF is rised claiming that there is no json file, and I can see that using the DevTools source tab that my json file is not there ( not loaded )
I tried force reload from electron app menu.
Here is my files code that is reading my json
const jsonfile = require('jsonfile')
const file = '/index.json';
var json;
jsonfile.readFile(file)
.then(obj => json = obj)
.catch(error => console.error(error))
------------ Edit
correcting the path name to index.json or ./index.json rises the same issue
You can use the native fs (filesystem) module.
let path = "index.json"
const fs = require('fs');
const json = JSON.parse(fs.readFileSync(path));
Thanks for your support
For me the issue was more about file system handling than electron.
All I did is to chmod my project folder to assure that I will be able to read and write into the index.json datastore
sudo chmod -R 777 /opt/workspaces/electron/myElectronPrpjectFolder
Then for a better path resolution I used the basic idea used in electron archtype, It more error safe
const path = require('path')
const file = path.join(__dirname,'index.json');
var json;
var html = "";// The returned object.
$(document).ready(function () {
jsonfile.readFile(file)
.then(obj => {
json = JSON.parse(JSON.stringify(obj));
console.log(JSON.stringify(json))
parseIssues(json.children);
document.getElementById('a').innerHTML = html;
})
.catch(error => console.error(error))
});
You can see that I m using JQuery in this snippet but it also works without JQuery.
in resume, better path resolve policy with granted priveleges on folder.
Thanks

Load Tensorflow js model from local file system in javascript

I have converted a keras model to tensorflow json format and saved it locally in my computer. I am trying to load that json model in a javascript code using the below command
model = await tf.loadModel('web_model')
But the model is not getting loaded.
Is there a way to load tensorflow json model from local file system?
I know you're trying to load your model in a browser but if anybody lands here that's trying to do it in Node, here's how:
const tf = require("#tensorflow/tfjs");
const tfn = require("#tensorflow/tfjs-node");
const handler = tfn.io.fileSystem("./path/to/your/model.json");
const model = await tf.loadLayersModel(handler);
LoadModel uses fetch under the hood. And fetch cannot access the local files directly. It is meant to be used to get files served by a server. More on this here.
To load a local file with the browser, there is two approaches, asking the user to upload the file with
<input type="file"/>
Or serving the file by a server.
In these two scenarios, tf.js provides way to load the model.
Load the model by asking the user to upload the file
html
<input type="file" id="upload-json"/>
<input type="file" id="upload-weights"/>
js
const uploadJSONInput = document.getElementById('upload-json');
const uploadWeightsInput = document.getElementById('upload-weights');
const model = await tfl.loadModel(tf.io.browserFiles(
[uploadJSONInput.files[0], uploadWeightsInput.files[0]]));
Serving the local files using a server
To do so, one can use the following npm module http-server to serve the directory containing both the weight and the model. It can be installed with the following command:
npm install http-server -g
Inside the directory, one can run the following command to launch the server:
http-server -c1 --cors .
Now the model can be loaded:
// load model in js script
(async () => {
...
const model = await tf.loadFrozenModel('http://localhost:8080/model.pb', 'http://localhost:8080/weights.json')
})()
const tf = require('#tensorflow/tfjs');
const tfnode = require('#tensorflow/tfjs-node');
async function loadModel(){
const handler = tfnode.io.fileSystem('tfjs_model/model.json');
const model = await tf.loadLayersModel(handler);
console.log("Model loaded")
}
loadModel();
This worked for me in node. Thanks to jafaircl.
If you're using React with create-react-app, you can keep your saved model files in your public folder.
For example, say you want to use the blazeface model. You would
Download the .tar.gz model from that web page.
Unpack the model into your app's public directory. So now you have the files from the .tar.gz file in a public subdir:
%YOUR_APP%/public/blazeface_1_default_1/model.json
%YOUR_APP%/public/blazeface_1_default_1/group1-shard1of1.bin
Load the model in your React app using
tf.loadGraphModel(process.env.PUBLIC_URL + 'blazeface_1_default_1/model.json'
You could try:
const model = await tf.models.modelFromJSON(myModelJSON)
Here it is in the tensorflow.org docs
Check out our documentation for loading models: https://js.tensorflow.org/api/latest/#Models-Loading
You can use tf.loadModel takes a string which is a URL to your model definition which needs to get served over HTTP. This means you need to start an http-server to serve those files (it will not allow you to make a request to your filesystem because of CORS).
This package can do that for you: npmjs.com/package/http-server
You could use insecure chrome instance:
C:\Program Files (x86)\Google\Chrome\Application>chrome.exe --disable-web-security --disable-gpu --user-data-dir=C:/Temp
Than you could add this script to redefine fetch function
async function fetch(url) {
return new Promise(function(resolve, reject) {
var xhr = new XMLHttpRequest
xhr.onload = function() {
resolve(new Response(xhr.responseText, {status: 200}))
}
xhr.onerror = function() {
reject(new TypeError('Local request failed'))
}
xhr.open('GET', url)
xhr.send(null)
})
}
After that be shure that you use the right model loader
my comment about loader issue
BUT your weights will be incorrect - as I understand there are some encoding problems.
If you are trying to load it in server side, use #tensorflow/tfjs-node instead of #tensorflow/tfjs and update to 0.2.1 or higher version to resolve this issue.
I am using React js for loading model (for image classification and more machine learning stuff)
Tensorflow.js do not support an Api to read a previously model trained
const file= new Blob()
file.src=modelJSON
const files= new Blob()
files.src=modelWeights
console.log(files)
const model= await tf.loadLayersModel(tf.io.browserFiles([file, files]));
[![enter image description here][1]][1]
You be able to create an APi in Express.js for servering your model (model.json and weigths.bin) if you use a web app (for a tensorflow.lite you could use a opencv.readTensorflowmodel(model.pb, weight.pbtxt)
References: How to load tensorflow-js weights from express using tf.loadLayersModel()?
const classifierModel = await tf.loadLayersModel(
"https://rp5u7.sse.codesandbox.io/api/pokeml/classify"
);
const im = new Image()
im.src =imagenSample//'../../../../../Models/ShapesClassification/Samples/images (2).png';
const abc= this.preprocessImage(im);
const preds = await classifierModel.predict(abc)//.argMax(-1);
console.log('<Response>',preds,'Principal',preds.shape[0],'DATA',preds.dataSync())
const responde=[...preds.dataSync()]
console.log('Maxmimo Valor',Math.max.apply(Math, responde.map(function(o) { return o; })))
let indiceMax = this.indexOfMax(responde)
console.log(indiceMax)
console.log('<<<LABEL>>>',this.labelsReturn(indiceMax))
If you are using Django, you should:
create a directory static in your app and put your model there.
load that static directory to the template where you want to use your model:
var modelPath = "{% static 'sampleModel.json' %}">
Don't forget to also load tensorflow.js library:
<script src="https://cdn.jsdelivr.net/npm/#tensorflow/tfjs"></script>
Now you can load your model:
<script>model = await tf.loadGraphModel(modelPath)</script>
i found a solution that it works. You can replace the url with a localhost url on xampp, for example (directory = model) http://localhost/model/model.json and after that you have to disable your browser CORS policy. For me i found a chrome extention and removed cors for my specific tab and it worked.
Thank me later!!

File path in Meteor

I'm having trouble identifying the path to a file in the public directory c:\TEMP\todos\.meteor\local\build\programs\server\public\main.py. Meteor complains the file or directory doesn't exist. Already searched the other postings about the similar issue (e.g., Reading files from a directory inside a meteor app) but didn't help.
Here is the error message.
=> Your application has errors. Waiting for file change.
=> Modified -- restarting.
=> Meteor server restarted
W20151206-04:05:57.893(-5)? (STDERR) Error inside the Async.runSync: ENOENT, no such file or directory 'c:\TEMP\todos\.meteor\local\build\programs\server\public'
Client code
Meteor.call('runPython', function(err, response) {
if(err){
} else {
console.log(response);
}
})
Server code
Meteor.startup( function (){
Meteor.methods({
runPython: function (){
var PythonShell = Meteor.npmRequire('python-shell');
var fs = Meteor.npmRequire('fs');
var runPython = Async.runSync(function (done){
var files = fs.readdirSync('./public/');
// PythonShell.run('main.py', function ... was tried first but Meteor complained that "main.py doesn't exist". So below is a different attempt.
var py = _(files).reject(function(fileName){
return fileName.indexOf('.py') <0;
})
PythonShell.run(py, function (err) {
// PythonShell.run(path.join(py,"main.py") ... was also tried but resulted in the same error
if (err) throw err;
console.log('script running failed');
});
})
return "Success";
}
})
})
All files inside the public folder should be read using '/':
var files = fs.readdirSync('/');
More here: http://docs.meteor.com/#/full/structuringyourapp
For server-side only (might be your case and probably a better solution) you can put everything under the private/ folder and access them by using the Assets API: http://docs.meteor.com/#/full/assets_getText
Clearly I was overthinking it. Specifying a full path to the file was all I needed to do.
PythonShell.run('c:\\project\\public\\main.py', function ...
If your application allows moving the Python script to /private instead of /public, you can take advantage of Meteor's Assets:
Assets allows server code in a Meteor application to access static server assets, which are located in the private subdirectory of an application’s tree. Assets are not processed as source files and are copied directly into your application’s bundle.
e.g. If you move your script to /private/scripts/script.py you can generate the absolute path the Meteor way by doing Assets.absoluteFilePath('scripts/script.py').

AWS Lambda Function is returning "Cannot find module 'index'" yet the handler in the config is set to index

As my title explains I am getting the following error:
{
"errorMessage": "Cannot find module 'index'",
"errorType": "Error",
"stackTrace": [
"Function.Module._resolveFilename (module.js:338:15)",
"Function.Module._load (module.js:280:25)",
"Module.require (module.js:364:17)",
"require (module.js:380:17)"
]
}
I have tried both solutions provided in creating-a-lambda-function-in-aws-from-zip-file and simple-node-js-example-in-aws-lambda
My config currently looks like:
and my file structure is:
and my index.js handler function looks like :
exports.handler = function(event, context) {
What else could be causing this issue aside from what was stated in those two answers above? I have tried both solutions and I have also allocated more memory to the function just incase thats why it couldn't run.
EDIT -
For the sake of trying, I created an even simpler version of my original code and it looked like this:
var Q = require('q');
var AWS = require('aws-sdk');
var validate = require('lambduh-validate');
var Lambda = new AWS.Lambda();
var S3 = new AWS.S3();
theHandler = function (event, context) {
console.log =('nothing');
}
exports.handler = theHandler();
And yet still does not work with the same error?
Try zipping and uploading the contents of the folder lambda-create-timelapse. Not the folder itself.
If this was unclear for anyone else, here are the steps:
Step 1
Navigate to the folder of your project, and open that folder so that you are inside the folder:
Step 2
Select all of the images you want to upload into to Lambda:
Step 3
Right-click and compress the files you have selected:
This will give you a .zip file, which is the file you need to upload to Lambda:
There are a lot of ways to automate this, but this is the manual procedure.
I ran into this problem a few times myself, and this indeed has to do with zipping the folder instead of just the contents like you're supposed to.
For those working from the terminal...
While INSIDE of the directory where the .js files are sitting, run the following:
zip -r ../zipname.zip *
The * is instructing the client to zip all the contents within this folder, ../zipname.zip is telling it to name the file zipname.zip and place it right outside of this current directory.
I had the same problem sometime ago - I reformatted the code.
function lambdafunc1(event, context) {
...
...
...
}
exports.handler = lambdafunc1
The problem occurs when the handler cannot be located in the zip at first level. So anytime you see such error make sure that the file is at the first level in the exploded folder.
To fix this zip the files and not the folder that has the files.
Correct Lambda function declaration can look like this:
var func = function(event, context) {
...
};
exports.handler = func;
You may have other syntax errors that prevent the index.js file from being properly ran. Try running your code locally using another file and using the index.js as your own module.
make sure in your handler following code added
exports.handler = (event, context, callback) => {
...
}
Another reason this can occur is if you don't do an npm install in the folder before packaging and deploying.

Categories