We are looking for ways to bulk upload data into google firestore and found here in Stack Overflow an awesome script to do so, sort of. The problem is that when data is uploaded the collection is fine, document is fine, but nesting some data we can only manage to import as "map" type when we need "array" and "map". Since we are basically newbies trial and error has not been enough. We appreciate if you can take a look at the code and help us with that.
So far, with the following code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<script src="https://www.gstatic.com/firebasejs/7.5.0/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.5.0/firebase-firestore.js"></script>
</head>
<body>
<script>
var config = {
apiKey: 'MY API KEY',
authDomain: 'MY AUTHDOMAIN',
projectId: 'MY PROJECT ID'
};
firebase.initializeApp(config);
var firestore = firebase.firestore();
//Paste from the CSV to JSON converter
const data = [
{
"collection": "sub_cats",
"uid": "Av9vJ0EoFfxhAR2",
"class": "especialidades_medicas",
"id": 0,
"name": "Cardiologo",
"ind": "11",
"district": ""
},
{
"collection": "sub_cats",
"uid": "Av9vJ0EoFfxhAR2",
"class": "especialidades_medicas",
"id": 1,
"name": "Urologo",
"ind": "12",
"district": ""
}
]
var promises = [];
data.forEach(function (arrayItem) {
var docRef = firestore.collection(arrayItem.collection).doc(arrayItem.uid);
var objClass = {};
var objId = {};
objId[arrayItem.id] = { name: arrayItem.name, ind: arrayItem.ind };
objClass[arrayItem.class] = objId;
promises.push(docRef.set(objClass, { merge: true }));
});
</script>
</body>
</html>
we get this:
data structure current / needed
How would you modify this code to get the array / map structure?
Thanks a lot!!!
You will need to use arrayUnion() of fieldValue that can be used with set() or update(). So basically if you use it with update it will "merge" the new value with the existing array values in your document. Inside that you can have your map or any datatype you want.
An example code looks like this:
db.collection("users").doc(props.uid).update({
points: db.FieldValue.arrayUnion({value: pointObj.value, reason: pointObj.reason})
});
Related
Right, so I am trying to wrap my head around editing (appending data) to a JSON file.
The file (users.json) looks like this:
{
"users": {
"id": "0123456789",
"name": "GeirAndersen"
}
}
Now I want to add users to this file, and retain the formatting, which is where I can't seem to get going. I have spent numerous hours now trying, reading, trying again... But no matter what, I can't get the result I want.
In my .js file, I get the data from the json file like this:
const fs = require('fs').promises;
let data = await fs.readFile('./test.json', 'utf-8');
let users = JSON.parse(data);
console.log(JSON.stringify(users.users, null, 2));
This console log shows the contents like it should:
{
"id": "0123456789",
"name": "GeirAndersen"
}
Just to test, I have defined a new user directly in the code, like this:
let newUser = {
"id": '852852852',
"name": 'GeirTrippleAlt'
};
console.log(JSON.stringify(newUser, null, 2));
This console log also shows the data like this:
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
All nice and good this far, BUT now I want to join this last one to users.users and I just can't figure out how to do this correctly. I have tried so many version and iterations, I can't remember them all.
Last tried:
users.users += newUser;
users.users = JSON.parse(JSON.stringify(users.users, null, 2));
console.log(JSON.parse(JSON.stringify(users.users, null, 2)));
console.log(users.users);
Both those console logs the same thing:
[object Object][object Object]
What I want to achieve is: I want to end up with:
{
"users": {
"id": "0123456789",
"name": "GeirAndersen"
},
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
}
When I get this far, I am going to write back to the .json file, but that part isn't an issue.
That's not really a valid data structure, as you're trying to add another object to an object without giving that value a key.
I think what you're really looking for is for 'users' to be an array of users.
{
"users": [
{
"id": "0123456789",
"name": "GeirAndersen"
},
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
]
}
You can easily create an array in JS and the push() new items into your array. You JSON.stringify() that with no issue.
const myValue = {
users: []
};
const newUser = {
'id': '0123456789',
'name': "GeirAndersen'
};
myValue.users.push(newUser);
const strigified = JSON.stringify(myValue);
I have a Firebase realtime database that reads sensor data (updated every 0.3s) and displays it on my webpage. After doing some research I found out about 'pretty-printing'. However, this is not in format I am after. My data right now is displayed like this: {"Moisture":619}.
What I am looking for is: Moisture: 619. As of right now this code is also creating a new {"Moisture":619} every time the value in the database is updated. Ideal would be if the new value is updated, making it so it just changes the value after Moisture, instead of displaying the whole thing again.
My code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="/styles.css">
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-auth.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-database.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-firestore.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-storage.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.11.0/firebase-messaging.js"></script>
<script>
// Initialize Firebase
var config = {
apiKey: "xx",
authDomain: "xx",
databaseURL: "xx",
projectId: "xx",
storageBucket: "xx",
messagingSenderId: "xx",
appId: "xx"
};
firebase.initializeApp(config);
</script>
<script>
var database = firebase.database();
var ref = firebase.database().ref("plant-patrol/Moisture");
ref.once("value")
.then(function(snapshot) {
var key = snapshot.key; // "plant-patrol"
var childKey = snapshot.child().key; // "Moisture"
});
</script>
<script>
var ref = firebase.database().ref();
ref.on("value", function(snapshot) {
console.log(snapshot.val());
var snapshotJSON = JSON.stringify(snapshot.val());
var moisture = snapshotJSON;
document.write(moisture);
}, function (error) {
console.log("Error: " + error.code);
});
</script>
<script src="/script.js" defer></script>
</head>
</html>
You can use JSON.stringify and replace:
const json = {
id: "1",
employee_name: "Tiger Nixon",
employee_salary: "320800",
employee_age: "61",
profile_image: ""
};
document.getElementById("app").innerHTML = JSON.stringify(json, (key, value) => (value || ''), 4).replace(/"([^"]+)":/g, '$1:');
const json = {
id: "1",
employee_name: "Tiger Nixon",
employee_salary: "320800",
employee_age: "61",
profile_image: ""
};
document.getElementById("app").innerHTML = JSON.stringify(json, (key, value) => (value || ''), 4).replace(/"([^"]+)":/g, '$1:');
<div id="app"></div>
You can use pre tag to display formatted json.
const json = {
id: "1",
employee_name: "Tiger Nixon",
employee_salary: "320800",
employee_age: "61",
profile_image: ""
};
document.getElementById("app").innerHTML = JSON.stringify(json, (key, value) => (value || ''), 4).replace(/"([^"]+)":/g, '$1:');
<div><pre id="app"></pre></div>
You can use regex to remove []{}"" characters:
snapshotJSON.replace(/[\[\]\{\}\"]+/g, '')
But you already have the plain value as
snapshot.val()
so why not use this.
JSON.stringify()
converts a javascript object to a JSON formatted string - normally usedfor machine to machine communication. The opposite would be JSON.parse to convert text into a JavaScript object.
You can use prettier for styling your whole code.
Source: https://prettier.io
Npm Link: https://www.npmjs.com/package/prettier
I need help in my one issue. I have write the program in that I am use map in node.js.
I am testing this program using postman by sending JSON structure, however I am not get specific value in console which I am printing.
Please see below code .
async CreateProduceMVPRateAsset(data, callback) {
// Create a new file system based wallet for managing identities.
var ProducePRICE = {};
var MVPRATE = new Map();
var MVPPRICE =[];
var MVPPRICE_BS ={};
var MVPPRICE_LB ={};
var PRODUCENAME = data.PRODUCE
console.log('PRODUCENAME', PRODUCENAME);
var COUNTRY = data.COUNTRY;
console.log('COUNTRY', COUNTRY);
var STATE = data.STATE;
console.log('STATE', STATE);
MVPRATES = data.MVPRATES;
console.log('MVPRATERATE', MVPRATES); // not getting value of MVPRATES from request body
}
JSON structure which is sending using POSTMAN
{
"username": "admin2",
"PRODUCE": "Apple",
"STATE": "MI",
"COUNTRY": "US",
"MVPRATES": {
"fuji": {
"VARIETY": "fuji",
"RATE": [
{
"UNIT": "Bussel",
"CURRENCY": "USD",
"VALUE": 10.25,
"UIDISPLAY": true
}
]
},
"gala": {
"VARIETY": "gala",
"RATE": [
{
"UNIT": "Bussel",
"CURRENCY": "USD",
"VALUE": 10.25,
"UIDISPLAY": true
}
]
}
}
}
output
Any help very appreciable
Thanks
Abhijeet
That's how logs will show up for the non-primitive type of data. Try stringifying the response like:
MVPRATES = data.MVPRATES;
console.log('MVPRATERATE', JSON.stringify(MVPRATES));
This will help you in printing actual values to the logs. A better approach will be to use a logging module like winston and configure all such things and many more.
Sorry to waste you all time I think I miss the var in front of MVPRATES. It should be
var MVPRATES = data.MVPRATES;
I'm extracting an array with 4 objects and each object has an array inside, from my kendo charts datasource, on my Angular project.
The data inside each sub-object varies in size, but it always includes a timestamp, and 1-5 value fields.
I need to export this array to an Excel file (.xls or .xlsx NOT CSV).
So far I managed to download the JSON as a file on its own (both .json and unformatted .xls).
I'd like for each object to be a book and in that book to have a formatting that has the timestamp in the first column, value 1 in another, and so on. The header for the columns should be timestamp, value1 name, etc (I'm translating these on the ui according to user preferences).
How can I build this type of formatted .xls file using angular? I don't know a particular good library for this, that is clear on how to use it in Angular.
Following Nathan Beck's link sugestion, I used AlaSQL. I'm getting correctly formatted columns, just need to adapt my array to have multiple worksheets.
The way we integrate alaSQL into our Angular project is by including the alasql.min.js and xlsx.core.min.js.
Then we call the alasql method in our function
$scope.export = function(){
var arrayToExport = [{id:1, name:"gas"},...];
alasql('SELECT * INTO XLSX("your_filename.xlsx",{headers:true}) FROM ?', arrayToExport);
}
EDIT: Solved the multiple worksheets issues as well. Keep in mind that when using the multiple worksheet method, you have to remove the asterisk and replace the headers: true object in the query with a question mark, passing the options in a separate array. So:
var arrayToExport1 = [{id:1, name:"gas"},...];
var arrayToExport2 = [{id:1, name:"solid"},...];
var arrayToExport3 = [{id:1, name:"liquid"},...];
var finalArray = arrayToExport1.concat(arrayToExport2, arrayToExport3);
var opts = [{sheetid: "gas", headers: true},{sheetid: "solid", headers: true},{sheetid: "liquid", headers: true}];
alasql('SELECT INTO XLSX("your_filename.xlsx",?) FROM ?', [opts, finalArray]);
You can use the XLSX library to convert JSON into XLS file and Download. Just create a service for your AngularJS application then call it as service method having below code.
I found this tutorial having JS and jQuery code but we can refer this code to use in AngularJS
Working Demo
Source link
Method
Include library
<script type="text/javascript" src="//unpkg.com/xlsx/dist/xlsx.full.min.js"></script>
JavaScript Code
var createXLSLFormatObj = [];
/* XLS Head Columns */
var xlsHeader = ["EmployeeID", "Full Name"];
/* XLS Rows Data */
var xlsRows = [{
"EmployeeID": "EMP001",
"FullName": "Jolly"
},
{
"EmployeeID": "EMP002",
"FullName": "Macias"
},
{
"EmployeeID": "EMP003",
"FullName": "Lucian"
},
{
"EmployeeID": "EMP004",
"FullName": "Blaze"
},
{
"EmployeeID": "EMP005",
"FullName": "Blossom"
},
{
"EmployeeID": "EMP006",
"FullName": "Kerry"
},
{
"EmployeeID": "EMP007",
"FullName": "Adele"
},
{
"EmployeeID": "EMP008",
"FullName": "Freaky"
},
{
"EmployeeID": "EMP009",
"FullName": "Brooke"
},
{
"EmployeeID": "EMP010",
"FullName": "FreakyJolly.Com"
}
];
createXLSLFormatObj.push(xlsHeader);
$.each(xlsRows, function(index, value) {
var innerRowData = [];
$("tbody").append('<tr><td>' + value.EmployeeID + '</td><td>' + value.FullName + '</td></tr>');
$.each(value, function(ind, val) {
innerRowData.push(val);
});
createXLSLFormatObj.push(innerRowData);
});
/* File Name */
var filename = "FreakyJSON_To_XLS.xlsx";
/* Sheet Name */
var ws_name = "FreakySheet";
if (typeof console !== 'undefined') console.log(new Date());
var wb = XLSX.utils.book_new(),
ws = XLSX.utils.aoa_to_sheet(createXLSLFormatObj);
/* Add worksheet to workbook */
XLSX.utils.book_append_sheet(wb, ws, ws_name);
/* Write workbook and Download */
if (typeof console !== 'undefined') console.log(new Date());
XLSX.writeFile(wb, filename);
if (typeof console !== 'undefined') console.log(new Date());
Angular directive for exporting and downloading JSON as a CSV. Perform bower install ng-csv-download. Run in plunkr
var app = angular.module('testApp', ['tld.csvDownload']);
app.controller('Ctrl1', function($scope, $rootScope) {
$scope.data = {};
$scope.data.exportFilename = 'example.csv';
$scope.data.displayLabel = 'Download Example CSV';
$scope.data.myHeaderData = {
id: 'User ID',
name: 'User Name (Last, First)',
alt: 'Nickname'
};
$scope.data.myInputArray = [{
id: '0001',
name: 'Jetson, George'
}, {
id: '0002',
name: 'Jetson, Jane',
alt: 'Jane, his wife.'
}, {
id: '0003',
name: 'Jetson, Judith',
alt: 'Daughter Judy'
}, {
id: '0004',
name: 'Jetson, Elroy',
alt: 'Boy Elroy'
}, {
id: 'THX1138',
name: 'Rosie The Maid',
alt: 'Rosie'
}];
});
<!DOCTYPE html>
<html ng-app="testApp">
<head>
<meta charset="utf-8" />
<title>Exporting JSON as a CSV</title>
<script>document.write('<base href="' + document.location + '" />');</script>
<link rel="stylesheet" href="style.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.3.15/angular.min.js"></script>
<script src="csv-download.min.js"></script>
<script src="app.js"></script>
</head>
<body>
<div>Using an Angular directive for exporting JSON data as a CSV download.</div>
<div ng-controller="Ctrl1">
<h2>All Attributes Set</h2>
<csv-download
filename="{{data.exportFilename}}"
label="{{data.displayLabel}}"
column-header="data.myHeaderData"
input-array="data.myInputArray">
</csv-download>
<hr />
<h2>Only Required Attribute Set</h2>
<h3>Optional Attributes Default</h3>
<csv-download
input-array="data.myInputArray">
</csv-download>
</div>
</body>
</html>
I've got the following document named "clients" which includes id, name and list of projects (array of objects):
{
"_id": {
"$oid": "572225d997bb651819f379f7"
},
"name": "ppg",
"projects": [
{
"name": "aaa",
"job_description": "",
"projectID": 20
},
{
"name": "bbbb",
"job_description": "",
"projectID": 21
}
]
}
I would like to update "job_description" of project with given "projectID" like this:
module.exports.saveJobDesc = function(client, idOfProject, textProvided) {
db.clients.update({ name: client},
{ $set: {'projects.0.job_description': textProvided }});
};
But instead of hardcoded index "0" of array I want to find specific project using "projectID". Is there a way to achieve this without changing the structure of collection and/or document?
If you want to update the "job_description" where name="ppg" and project_id=20 then you can use below mongo query:-
db.clients.update({ "name":"ppg","projects.projectID":20 },{$set: {"projects.$.job_description": "abcd"}})
Please let me know if any thing else is required
You cannot update multiple array elements in single update operation, instead you can update one by one which takes time depends upon number of elements in array and number of such documents in collection. see New operator to update all matching items in an array
db.test2.find().forEach( function(doc) {
var projects = doc.projects;
for(var i=0;i<projects.length;i++){
var project = projects[i];
if(project.projectID == 20){
var field = "projects."+i+".job_description";
var query = {};
query[field] = "textasdsd";
db.test2.update({ _id: doc._id},{ $set:query});
}
}
})