Basic training of ML5 neural network not working - javascript

I am using ML5 to train a Neural Network.
I am loading a CSV file with data from the Titanic. This works when I download the demo file from the ML5 GitHub.
But when I use a different CSV file, and I replace the column names in my code, it stops working. Am I missing something? Is it a problem that my CSV file contains numbers, while the demo file contains strings?
let neuralNetwork
function start() {
const nnOptions = {
dataUrl: "data/titanic.csv",
inputs: ["Pclass", "Sex", "Age", "SibSp"], // labels from my CSV file
outputs: ["Survived"],
task: "classification",
debug: true,
};
neuralNetwork = ml5.neuralNetwork(nnOptions, modelReady);
}
function modelReady() {
neuralNetwork.normalizeData();
neuralNetwork.train({ epochs: 50 }, whileTraining, finishedTraining);
}
// this doesn't get called at all
function whileTraining(epoch, logs) {
console.log(`Epoch: ${epoch} - loss: ${logs.loss.toFixed(2)}`);
}
// this gets called immediately
function finishedTraining() {
console.log("done!");
}
start()
The console immediately shows "done!", but the model is not trained. There is no error message.
The strange thing is, when a label name is incorrect, then I do get an error. So the label names are actually recognised.
Original CSV file, working:
survived,fare_class,sex,age,fare
died,first,male,39,0
died,first,male,29,0
My CSV file, not working:
Survived,Pclass,Sex,Age,SibSp,Parch,Fare,Embarked
0,3,1,22.0,1,0,7.25,1
1,1,0,38.0,1,0,71.2833,2

Just in case anyone runs into this issue: when you are classifying, the label always has to be a string....
Working CSV file:
Survived,Pclass,Sex,Age,SibSp,Parch,Fare,Embarked
yes,3,1,22.0,1,0,7.25,1
no,1,0,38.0,1,0,71.2833,2

I'm not sure if this would help but from what I learned is that all the inputs should be numbers. But if the demo works then it the code should work. When it comes to ML5 you have to pay attention to the CSV file. make sure the output that you are trying to predict is a string. if ur using a csv with only numbers and are trying to predict a number as output then you should change the task: classification to regression. you can also give this code a try if you would like to:

Related

Javascript loading a text or json file into array - to be used for form validation

I have a script that validates a form input based on the input being one of the values within an array. If the input value is not within the array, an alert happens, otherwise code continues. I have this working using an array that is populated within the code as below,
var arr= ["AANDRUS, Bloemfontein",
"AANHOU WEN, Stellenbosch",
"ABBOTSDALE, Western Cape",
"ABBOTSFORD, East London",
"ABBOTSFORD, Johannesburg",
"ABBOTSPOORT, Limpopo",
"ABERDEEN, Eastern Cape"];
but there are +4000 values and the correct way is to create the array in an external file.
I have tried this using var arr = [<?php require_once("townlist.txt")?>]; and then my validation code looks like this
`function quoteValid()
{
var x = document.forms['quoteRequest']['colltown_name'].value;
if ($.inArray(x, arr) == -1) {
alert("Please select Collection Town");
document.forms['quoteRequest']['colltown_name'].focus();
return false;
}
return true;
}`
but this does not load the array and the code does not execute as expected.
So, my validation works using the array within the code, but not with the external file loading. I know there are many examples of similar questions on SO, and I have tried replicating quite a few with no positive result.
I have tried using an external json file and text file without it working. I have tried uploading the file to my server (as I am using it on localhost in development) and using the URL to the file on the server. A sample of my text file is as follows
[{"AANDRUS, Bloemfontein"},
{"AANHOU WEN, Stellenbosch"},
{"ABBOTSDALE, Western Cape"},
{"ABBOTSFORD, East London"},
{"ABBOTSFORD, Johannesburg"},
{"ABBOTSPOORT, Limpopo"},
{"ABERDEEN, Eastern Cape"}]
Any help appreciated.

Download Fortify export data using the API

Objective
To generate CSV "export data" files from within Fortify, then download them.
Steps
Generate the (csv) export data using, that works fine:
https://SERVER:8443/ssc/api/v1/dataExports/action
Download the csv file using:
https://SERVER:8443/ssc/transfer/reportDownload.html - GET
Problem
I'm facing an issue with step 2, export being successfully generated. I can't seem to download them.
I'm requesting :
https://SERVER:8443/ssc/api/v1/fileTokens
with the payload : { fileTokenType : "REPORT_FILE" }
I obtain:
YzFmOWY4ZjMtZjU2MS00ZTU0XXXXXXXXXXXXX
Yet as per the documentation, I should get something like:
7e8d912e-2432-6496-3232-709b05513bf2
As a result when I attempt to GET my file with the following request:
https://SERVER:8443/ssc/transfer/reportDownload.html?mat=YzFmOWY4ZjMtZjU2MS00ZTU0XXXXXXXXXXXXX&id=449741
I get a 500 error.
Questions
What do you guys get when you request: https://SERVER:8443/ssc/api/v1/fileTokens?
Is the documentation not exactly correct?
If not, what do you reckon I am doing wrong?
Found the solution, the correct link is
dataExportDownload.html
instead of:
reportDownload.html

How to read in CSV with d3 v4?

I'm just having a little trouble understanding the documentation for CSV Parse with D3. I currently have:
d3.parse("data.csv",function(data){
salesData = data;
});
But I keep on getting the error:
Uncaught TypeError: d3.parse is not a function
What is this supposed to look like? I'm just a little confused, and the only examples that I could find was something like this.
I also tried something like:
d3.dsv.parse("data.csv",function(data){
salesData = data;
});
and got:
Uncaught TypeError: Cannot read property 'parse' of undefined
Why is this happening? Any help would be greatly appreaciated, thanks!!
There is some misunderstanding here: you're confusing d3.csv, which is a request, with d3.csvParse, which parses a string (and also mixing D3 v3 syntax with D3 v4 syntax). This is the difference:
d3.csv (D3 v4)
The d3.csv function, which takes as arguments (url[[, row], callback]):
Returns a new request for the CSV file at the specified url with the default mime type text/csv. (emphasis mine)
So, as you can see, you use d3.csv when you want to request a given CSV file at a given url.
For example, the snippet below gets the CSV at the url between quotes, which looks like this...
name, parent
Level 2: A, Top Level
Top Level, null
Son of A, Level 2: A
Daughter of A, Level 2: A
Level 2: B, Top Level
... and logs the parsed CSV file, check it:
d3.csv("https://gist.githubusercontent.com/d3noob/fa0f16e271cb191ae85f/raw/bf896176236341f56a55b36c8fc40e32c73051ad/treedata.csv", function(data){
console.log(data);
});
<script src="https://d3js.org/d3.v4.min.js"></script>
d3.csvParse
On the other hand, d3.csvParse (or d3.csv.parse in D3 v3), which takes as arguments (string[, row]):
Parses the specified string, which must be in the delimiter-separated values format with the appropriate delimiter, returning an array of objects representing the parsed rows.
So, you use d3.csvParse when you want to parse a string.
Here is a demo, suppose you have this string:
var data = "foo,bar,baz\n42,33,42\n12,76,54\n13,42,17";
If you want to parse it, you'll use d3.csvParse, not d3.csv:
var data = "foo,bar,baz\n42,33,42\n12,76,54\n13,42,17";
var parsed = d3.csvParse(data);
console.log(parsed);
<script src="https://d3js.org/d3.v4.min.js"></script>
You can get csv data into d3 like the following -
// get the data
d3.csv("data.csv", function(error, data) {
if (error) throw error;
console.log(data);
//format data if required...
//draw chart
});
I also could not get the d3.csv("csv_file.csv", function(data) { //modifying code }
to work.
A classmate recommended using the following, which has worked so far:
d3.csv("data.csv").then(function(data){
//modifying code
}
As noted in the comments below, this is a fix if you're running v5 instead of v4.
Use d3.csv("data.csv", function(data){...}) to get CSV from url and parse, or use d3.csv.parse() to parse a CSV-formatted string.
Here is the code you can use to read csv file using d3.js
<script src="https://d3js.org/d3.v4.min.js"></script>
<script>
d3.csv("csv/cars.csv", function(data) {
console.log(data[0]);
});
</script>
Note that, the csv file name is "cars.csv", and the file is saved in a folder called csv.
console.log(data[0])
will help you to see the data output in the browser debug window. Where, you can also find if there is any error as well.

Three .json files 'StockChart' - highcharts

Please help by taking StockChart. Pinning three .json files as in the example , But does not work. The code is below:
www.jsfiddle.net/d8xwjxg7/2
You have an error in your code:
$.getJSON('http://www.iklimat.pl/'+name()+'.php', function (data) {
Should be:
$.getJSON('http://www.iklimat.pl/'+ name +'.php', function (data) {
Since name is a string not a function.
Also, this will not work in a JSFiddle since you cannot load files not from the site the javascript is running on without the endpoint you are accessing setting the access-allow-control-origin header.
EDIT:
I have sorted out the issue you were having getting data, however there is an issue with that data.
http://jsfiddle.net/d8xwjxg7/5/

Importing data from two csv files using d3js

consider this fiddle link FIDDLE. In this example I would like to use a csv file to load data at line no.33-[data.csv] and data at line no.158-[data1.csv]. I want to use two separate csv files. I tried using a csv file for data at line no.33 with this code
d3.csv("data.csv", function(csvData) {
csvData.forEach(function (d,i) {
data[i] = {
first: +d.first,
second: +d.second
}
});
console.log(data);
I was able to get an output but the charts had moved far away from each other with the following errors : Unexpected value NaN parsing cy attribute. How to load the two datasets in an efficient way using two separate csv files ?
Here is hopefully the final plunker in this ever-growing project :) (A lot of the csv work here has been guided by the great Lars...as usual, many kudos to him.)
Updated plunker with data on top chart coming from datam.csv.

Categories