Export javascript array to a shape file - javascript

I'm new to javascript and need to create a web app where a user will click a button, and the array of data will export into a shape file. After reading this answer I know that it is possible with an ARCGIS server, but I do not have access to this.
The array in question is a stream of data similar to the following
var array = [
[17, 70, "mark", "let", "test", "test"],
[18, 50, "marj", "get", "test", "test"],
ETC...]
I've also read about shp-write but I don't know where to start. Would anyone be able to give me any examples of how to do this, or pointers where to start? Thanks.

You should just convert your data from a simple array to an array of points and an array of features, like:
let points = [
[17, 70],
[18, 50],
...
];
let features = [
{col1: "mark", col2: "let", col3: "test", col4: "test"},
{col1: "marj", col2: "get", col3: "test", col4: "test"},
...
];
And then call the write function provided by scp-write, providing your callback function to write the resulting file (check the examples for the callback function).
let scp = require('scp-write');
scp.write(features, 'POINT', points, callbackFunction);

Related

Best way to store a 2D-array with column and row names

I am trying to find the best way to store in Redux a 2D array with row and column names.
In terms of priorities, I would rank code readability and maintainability above performance. Hence, accessing a value through table[row][col] would be preferable.
I have thought about two solutions:
First, storing it in three distinct variables colNames, rowNames and table. Accessing values would still be doable. But, for example, removing a column would then require to modify both colNames and rowNames. Incrementing some value from row and column names would require to access all three variables.
var rowNames = ["row1", "row2", "row3"];
var colNames = ["col1", "col2", "col3"];
var table = [[1, 2, 3], [4, 5, 6], [7, 8, 9]];
The other way would be to store a single variable table as an Object. It becomes easy to access values. But it is harder to correctly update it in Redux to respect immutable state.
var table = {
row1: {
col1: 1,
col2: 2,
col3: 3
},
row2: {
col1: 4,
col2: 5,
col3: 6
},
row3: {
col1: 7,
col2: 8,
col3: 9
}
};
Some other idea would be to use Immer JS with the second solution to improve code readability.
What is then the best way to store such a value in Redux ?
Solution I chose
I ended up using the second solution with a single variable. Once the modification of this table is properly made in Redux reducers, it makes accessing and displaying said table easy.
I do think your solution is quite nice and adapted to rendering in an html table. However you will have to iterate over Object.keys(table) to go through the rows.
Another solution could be to do somehing like that so you can iterate over objects directly:
var table = [
{
name: "row1",
col1: 1,
col2: 2,
col3: 3
},
{
name: "row2",
col1: 4,
col2: 5,
col3: 6
},
{
name: "row3",
col1: 7,
col2: 8,
col3: 9
}
];
This solution also has it's problems: you have to specifically take the name property out of the object while iterating for rows. You would also have to iterate over Object.keys of the inner object avoiding the treatment of the "name" column.
Hope this help as an overview of your problem :)

Nodejs Multiple map arrays to single Json file

I'm new to nodejs and am trying to write a Json parser that will take data from a Json API, allow me to grab the data I want (some of this data I will transform) and then write to a Json file.
I have discovered the map command which is extracting the raw data and this is giving me an array for example
data['home_team'] = json['data'].map(smdata => smdata.localTeam.data.name);
data['awayid'] = json['data'].map(smdata => smdata.visitorTeam.data.id);
data['away_team'] = json['data'].map(smdata => smdata.visitorTeam.data.name);
And this works perfectly, now I'm left with an array which I want to turn into a Json file, I can of course just run a FOR loop and iterate through each of the entries and write to a file but I wondered if there was a more efficent way to do this
Thanks
Take a look at JSON.stringify to turn your data into a string you can write to a file.
In your comments below, you provide an invalid JS structure, an array with keys - maybe a typo with square braces (array) rather than curly braces (object). Or maybe note that Array map returns an array, so you may wish to look at Array.reduce, depending on your data.
let data = {
islive: [90, 90, null, null],
ls: [1, 1, 1, 1],
time: [90, 90, null, null],
status: ['FT', 'FT', 'NS', 'NS'],
starttime: [1616036700, 1616050800, 1616058000, 1616058900],
matchid: [17706353, 17926675, 17869155, 17926676]
};
// Dump it as JSON:
console.log(JSON.stringify(data));
// Pretty print with 2 space indent:
console.log(JSON.stringify(data, {}, 2));
// Write it to file 'my.json':
const fs = require('fs');
fs.writeFileSync(
'my.json',
JSON.stringify(data)
);
Outputs:
{"islive":[90,90,null,null],"ls":[1,1,1,1],"time":[90,90,null,null],"status":["FT","FT","NS","NS"],"starttime":[1616036700,1616050800,1616058000,1616058900],"matchid":[17706353,17926675,17869155,17926676]}
{
"islive": [
90,
90,
null,
null
],
"ls": [
1,
1,
1,
1
],
"time": [
90,
90,
null,
null
],
"status": [
"FT",
"FT",
"NS",
"NS"
],
"starttime": [
1616036700,
1616050800,
1616058000,
1616058900
],
"matchid": [
17706353,
17926675,
17869155,
17926676
]
}

Array or object in Javascript eval?

I am working on a code optimization where I have several rows (~500-1000) of data with at least 300 columns.
The data is stored as an array of objects:
data = [
{col1: 1, col2: 2, col3: null, col4: 1.3, col5:undefined, col6: 2, ....},
{col1: 3, col2: 4, col3: 3, col4: 1.1, col5:8, col6: 2, ....},
.
.
.
{col1: 1.1, col2: 1.2, col3: 7, col4: 3, col5:4, col6: 2, ....}
]
Then I have another array of objects named calculators, these are user-defined calculations. These calculations can have another calculated variable like 2nd object in the following array.
So I need to maintain the order.
calculators = [
{formula:'col1 + sin(col2)', order:0, colname: 'calc1'}
{formula:'calc1 * col6', order:1, colname: 'calc2'}
.
.
.
{formula: 'any random calculation', order:n, colname:'RandomName'}
]
my current code-flow is,
Extraction of dependent columns from calculators, like col1 and col2
for calc1 and col6 from calc2.
dependentCols = [col1, col2, col6]
Looping through the data array with nested for loops for calculators and columns and setting the value in parser for each
dependent column. parser.set(column, row[column]);
Then formula evaluation and appending that value to the dararow.
The original code I have includes some other stuff to handle sub-columns, units, and other stuff. That's why instead of pasting the original code, here is the rough draft.
const parser = math.parser();
data = _.map(data, row=>{
row = {...row};
_.forEach(calculators, (calculator)=>{
_.forEach(dependentCols, col=>{
// cannot add the whole row as it is because of the sub-column stuff.
parser.set(col, row[col]);
});
try {
const value = parser.eval(calculator.formula);
row[colname] = value;
} catch (exception) {
console.warn(parser, exception, calculator.colname);
}
}
});
return row;
});
}
The original code is definitely working (for a year). Its just very slow, with 450 rows and 20 calculations its taking upto 12 seconds.
So instead of going row by row and effectively cell by cell I am thinking about passing the whole column at the same time to the eval function. But I couldn't find any supporting documentation.
Any help will be appreciated.

Reading CSV in D3 and converting it to array of arrays

I have the following small dataset hardcoded into a variable in my code:
var dataset = [['CENTRAL', 44, 552, 18565],
['NORTHERN', 42, 945, 20092],
['SOUTHERN', 96, 795, 30095],
['PARK', 1, 640, 9341],
['MISSION', 66, 1198, 18542],
['TENDERLOIN', 23, 113, 10735],
['RICHMOND', 9, 561, 9082],
['TARAVAL', 81, 789, 11966],
['INGLESIDE', 5, 1368, 13414],
['BAYVIEW', 7, 985, 14711]];
Now, this dataset is taken directly from a .csv file, that looks like this:
District,Prostitution,VehicleTheft,Totalcrimecount
CENTRAL,44,552,18565
NORTHERN,42,945,20092
SOUTHERN,96,795,30095
PARK,1,640,9341
MISSION,66,1198,18542
TENDERLOIN,23,113,10735
RICHMOND,9,561,9082
TARAVAL,81,789,11966
INGLESIDE,5,1368,13414
BAYVIEW,7,985,14711
However, I'd obviously like to be able to just read in the file, which I've attempted using this:
var dataset_csv = d3.csv("datafile.csv", function(d){
console.log(dataset_csv = d)
});
Which gives me an array of objects that look like this:
[Object, Object, Object, Object, Object, Object, Object, Object, Object, Object]
District: "CENTRAL"
Prostitution: "44"
Totalcrimecount: "18565"
VehicleTheft: "552"
My question is, and it might be trivial, how can I transform those objects into an array data structure like my initial dataset? I've been battling with it for some time now. Any hints greatly appreciated.
Use Array#map to iterate over each object and Object.keys to catch only the keys values.
var dataset_csv = [{District: "CENTRAL", Prostitution: "44", Totalcrimecount: "18565", VehicleTheft: "552"}, {District: "WEST", Prostitution: "22", Totalcrimecount: "4324", VehicleTheft: "53"}, {District: "EAST", Prostitution: "11", Totalcrimecount: "23434" , VehicleTheft: "76"}],
datasetArr = dataset_csv.map(v => Object.keys(v).map(c => Number(v[c]) ? Number(v[c]) : v[c]));
console.log(datasetArr);

JSON understanding and arrays

I am in process of reverse engineering a JS script. Somewhere is says:
var a = [{
name: 'sample1',
data: ["Otu1", "Otu2", "Otu3", "Otu4", "Otu5"],
values: [5, 15, 250, 20, 23]
},{
name: 'sample2',
data: ["Otu1", "Otu5", "Otu6", "Otu7"],
values: [234, 29, 239, 5]
}]
First question: What type of object is it? is it JSON? Or is it an array of JSON objects?
I need to write this in this form:
var b = {
name: 'sample1',
data: ["Otu1", "Otu2", "Otu3", "Otu4", "Otu5"],
values: [5, 15, 250, 20, 23]
}
var c = {
name: 'sample2',
data: ["Otu1", "Otu5", "Otu6", "Otu7"],
values: [234, 29, 239, 5]
}
var a = b + c
Could you please help? Any insights are appreciated. Thank you community !
"First question: What type of object is it? is it JSON? Or is it an array of JSON objects?"
It's an Array of JavaScript Objects. It could be serialized into JSON data, but currently you should just see it as JavaScript code. The notation is similar, but the resulting data is different.
(And actually in your case, for the notation to be JSON-like, you'd need to use double quotes. But even then, you're still creating JavaScript Objects)
"I need to write this in this form: "
For this, you could make an Array of JavaScript Objects like this:
var a = [b, c];
You have an array of Objects here, remember JSON simply means JavaScript Object Notation

Categories