We Keep Coding

MongoDB Scheduled Trigger Save Error -- What to Return?

This problem is very annoying. So, I am making a scheduled trigger run every 24 hours. It simply gets items from one collection does some data processing then appends information to another collection. The functioning code works even when the function runs. But it will not let me save because there are "runtime" errors? Even though it was executed perfectly and returned.
Console Error
> result (JavaScript):
EJSON.parse('{"$undefined":true}')
I suppose this has something to do with returning. but when I return null I get this:
> result:
null
> result (JavaScript):
EJSON.parse('null')
when trying to save I get this at the top of the page:
runtime error during function validation
Function Code:
exports = async function() {
const usersCol = context.services.get("SchoologyDashCluster").db("SchoologyDashApp").collection("users");
const gradesCol = context.services.get("SchoologyDashCluster").db("SchoologyDashApp").collection("grades");
var usersCusor = await usersCol.find( ).toArray();
var gradesCusor = await gradesCol.find( ).toArray();
let insert = [];
for (let i = 0; i < usersCusor.length; i++) {
var user = usersCusor[i];
var userSavedGrades = gradesCusor[i].grades
var currentGrades = await getGrades(user.schoologyUID, user.consumerKey, user.secretKey);
var lastGraded = NaN;
let index = gradesCusor[i].grades.length - 1;
while (true) {
if (gradesCusor[i].grades[index].changed == 1) {
lastGraded = index;
break
}
index = index - 1;
}
console.log(lastGraded)
if (userSavedGrades[lastGraded].grades.ga == currentGrades.ga){
currentGrades = { changed : 0, time: new Date().getTime()};
} else {
currentGrades = {changed : 1, grades: currentGrades, time : new Date().getTime()};
}
gradesCol.updateOne(
{"user" : user._id},
{"$push" : {"grades" : currentGrades}}
)
}
// return usersCol.find( );
return null;
};

The answer was simple and now I feel ignorant. Instinctual I put the module imports at the top of the document. However this is incorrect and they need to be placed in the exports function, like so:
exports = function (x,y,z) {
const http = context.http;
return;
}

I am trying to run two functions onLoad, one needs to run first so the second one can populate a boxlist, however, the second one doesn't get thearray

I have two functions that I am trying to run when I load the page. dataRetrieve() gets the data from a firebase collection. populate() is supposed to populate a boxlist with the entries retrieved from dataRetrieve(). The main problem is that it lists the array as empty when I run populate() after dataRetrieve() no matter what I try. The last thing I tried was this:
async function dataRetrieve(){
const getAdmins = firebase.functions().httpsCallable('getAdmins');
// Passing params to data object in Cloud functinon
getAdmins({}).then((results) => {
admins = results;
console.log("admins retrieved");
console.log(admins);
}).then(() => {
populate();
});
}
async function populate(){
let list = document.getElementById("user-list");
//loop through users in out Users object and add them to the list
for (var i = 0; i < admins.length; i++) {
let newItem = document.createElement('option');
newItem.innerHTML = admins[i].first + " " +admins[i].last;
newItem.id = admins[i].uid;
if (i == 0) {
newItem.className = "active";
}
console.log(newItem.innerHTML + " " + newItem.id)
list.appendChild(newItem);
}
updateResponse(list.firstChild);
list.size = admins.length;
console.log(document.getElementById("user-list").size)
//collect all the list items
let listItems = list.querySelectorAll('option');
//loop through the list itmes and add a click listener to each that toggles the 'active' state
for (var i = 0; i < listItems.length; i ++) {
listItems[i].addEventListener('click', function(e) {
if (!e.target.classList.contains('active')) {
for (var i = 0; i < listItems.length; i ++) {
listItems[i].classList.remove('active');
}
e.target.classList.add('active');
updateResponse(e.target);
}
})
}
}
also, admins is a global variable listed at the start of the script:
var admins = [];
I am trying to run all this onload so I can instantly generate the list
I thought that .next would cause it to wait to get the values before running, but even making results a parameter and transferring it directly into the function that way gives an undefined array. I don't understand why the function insists on calling on old data. Pls help.

I'm not sure what updateResponse function does. If it's not returning a promise then I'd make the populate function synchronous first. Also do you really need to use admins array somewhere else apart from populate function that it is a global variable? If not then I'd just pass it as a parameter.
async function dataRetrieve() {
const getAdmins = firebase.functions().httpsCallable('getAdmins');
// Passing params to data object in Cloud function
const results = await getAdmins({})
console.log("admins retrieved");
console.log(results);
// Passing results in populate function
populate(results.data)
// If your function returns an array, pass the array itself
}
function populate(admins) {
let list = document.getElementById("user-list");
//loop through users in out Users object and add them to the list
// Using a for-of loop instead so no need to worry about checking the length here
for (const admin of admins) {
let newItem = document.createElement('option');
newItem.innerHTML = admin.first + " " + admin.last;
newItem.id = admin.uid;
//if (i == 0) {
// newItem.className = "active";
//}
console.log(newItem.innerHTML + " " + newItem.id)
list.appendChild(newItem);
}
updateResponse(list.firstChild);
// rest of the logic
}

I guess you know how to check when the page loads. call the retrieve function when the page is loaded. Then you should call the populate function at the end of the retrieve function. this makes sure that the populate function is called after you get all the data

Async Javascript: Waiting for data to be processed in a for loop before proceeding to a new function

I'm having issues understanding how to work around Javascript's asynchronous behavior in a forEach loop. This issue is quite complex (sorry), but the idea of the loop is as followed:
Loop through every item in an array
Make an HTTP request from a provider script
I then need to multiply every element of the array by a constant
Assign the new array to an item in an object
After the loop, take all the arrays and add them together into one array
The data will be assigned to the indvCoinPortfolioChartData array
I'm looking for any flaws in my event loop. I believe the battle is making this task synchronous, making sure my data is assigned before aggregating data.
The issue
When I'm adding all the arrays together, ONE dataset isn't summed up (I think because it's still being processed after the function is called). There is no error, but it doesn't have all the coin data in the final aggregated array.
This is the issue I see in the aggregatePortfolioChartData function. It begins the for loop with only 2 items in the array, and then later shows 3. The third item was not processed until after the for loop started.
image of console log (logged from aggregatePortfolioChartData function)
debug log when aggregation is successful
var indivCoinPortfolioChartData = {'data': []};
for(var i = 0; i < this.storedCoins.Coins.length; i++)
{
let entry = this.storedCoins.Coins[i];
localThis._data.getChart(entry.symbol, true).subscribe(res => {localThis.generateCoinWatchlistGraph(res, entry);});
localThis._data.getChart(entry.symbol).subscribe(res => {
if(entry.holdings > 0)
{
let data = res['Data'].map((a) => (a.close * entry.holdings));
indivCoinPortfolioChartData.data.push({'coinData': data});
localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
}
else
{
localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
}
});
}
Loaded Coin Data
loadedCoinData(loader, indivCoinPortfolioChartData)
{
this.coinsWithData++;
if(this.coinsWithData === this.storedCoins.Coins.length - 1)
{
loader.dismiss();
this.aggregatePortfolioChartData(indivCoinPortfolioChartData);
}
}
aggregatePortfolioChartData
aggregatePortfolioChartData(indivCoinPortfolioChartData)
{
console.log(indivCoinPortfolioChartData);
var aggregatedPortfolioData = [];
if(indivCoinPortfolioChartData.data[0].coinData)
{
let dataProcessed = 0;
for(var i = 0; i < indivCoinPortfolioChartData.data[0].coinData.length; i++)
{
for(var j = 0; j< indivCoinPortfolioChartData.data.length; j++)
{
let data = indivCoinPortfolioChartData.data[j].coinData[i];
if(data)
{
aggregatedPortfolioData[i] = (aggregatedPortfolioData[i] ? aggregatedPortfolioData[i] : 0) + data;
dataProcessed++;
}
else
{
dataProcessed++;
}
}
if(dataProcessed === (indivCoinPortfolioChartData.data[0].coinData.length) * (indivCoinPortfolioChartData.data.length))
{
console.log(dataProcessed + " data points for portfolio chart");
this.displayPortfolioChart(aggregatedPortfolioData);
}
}
}
}
Thank you for helping me get through this irksome issue.

Using response from one API call to do another API Call

I'm using AngularJS to build a site where one of the functions is to present Billboard(it's a music chart) listings for a specified date.
I want to present the songs in order, together with an image of the song.
First I'm calling this API:
http://billboard.modulo.site/
where I give a date and get a response of the top 10 songs for that date and data about each song.
The response from the Billboard API also includes a spotify id and I want to use that ID and call the Spotify Web API to get an image of that song, to complement the information I present about each song.
This is how it looks like in my controller:
var spotifyID = [];
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data){ //Response is top 10 songs for given date
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
spotifyID[i] = data[i].spotify_id; //data[i].spotify_id returns the ID of the track, as given by the billboard API
}
$scope.getImages();
});
$scope.getImages = function() {
for (var i = 0; i < spotifyID.length; i++) {
if(spotifyID[i] !== null) {
musicService.getSpotify(spotifyID[i]).then(function(data){
$scope.spotifyImg[i] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
});
}
}
console.log($scope.spotifyImg);
}
And in my view it would look something like this:
<div ng-repeat = "song in songlist">
<div>{{ song.rank }}</div>
<div>
<img ng-src=" {{ spotifyImg[$index] }}"/>
</div>
</div>
However, it does not work.
When I'm checking the $scope.spotifyImg array in the console, it is of length 11 and only has one element in index 10 and that is the image of the last song(that is the 10th song).
I'm a bit confused as to why the $scope.spotifyImg array only contains one element in index 10. Also why is the array of length 11 when the spotifyID is of length 10?
Any ideas of how I could solve this?

The problem is that getSpotify is run asynchronous, when the responses to these calls come in, i is probably set to spotifyID.length - 1 which means that all callback functions set the $scope.spotifyImg[spotifyID.length - 1] element.
Try this:
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data){
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
$scope.getImage(data[i].spotify_id, i);
}
});
$scope.getImage = function(id, index) {
musicService.getSpotify(id).then(function(data){
$scope.spotifyImg[index] = data.album.images[0].url;
});
}

create separate function and put the content of for loop inside that function and call that function inside the loop
$scope.getImages = function() {
for (var i = 0; i < spotifyID.length; i++) {
if (spotifyID[i] !== null) {
sampleFunc(i);
}
}
}
function sampleFunc(i) {
musicService.getSpotify(spotifyID[i]).then(function(data) {
$scope.spotifyImg[i] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
});
}
I think reason you get only last index of an array is when you are calling promise inside loop, loop does't wait until the promise returns. it just keep executing.At the time promise returns loop is executed and it;s getting last index of the array. That's why you need to separately call the promise from for loop

You can use IIFE
(function(i){
musicService.getSpotify(spotifyID[i]).then(function (data) {
$scope.spotifyImg[i] = data.album.images[0].url;
});
})(i)
So,your getImages function should be like this.
$scope.getImages = function () {
for (var i = 0; i < spotifyID.length; i++) {
if (spotifyID[i] !== null) {
(function(i){
musicService.getSpotify(spotifyID[i]).then(function (data) {
$scope.spotifyImg[i] = data.album.images[0].url;
});
})(i)
}
}
}

Try using this code
Js code
var spotifyID = [];
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data) { //Response is top 10 songs for given date
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
spotifyID[i] = data[i].spotify_id; //data[i].spotify_id returns the ID of the track, as given by the billboard API
}
$scope.getImages(0);
});
$scope.getImages = function(index) {
if (index == spotifyID.length) {
return;
}
musicService.getSpotify(spotifyID[index]).then(function(data) {
$scope.spotifyImg[index] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
// call recursive here
$scope.getImages(index++);
});
}
}
call your getimages function recursively so that will add you images in array.

Nested for loop with inner functions

In javascript For loop it's works fast and inner functions are not called. I am trying to store images in database using cordova. In my code for loop works fast and finished but I didn't get any base64 images.
for(var i = 0; i < pages.length; i++)
{
var cat_img = res.Catalogue[0].Catalogue_img;
var catalogue_image_id = cat_img[i].catalogue_image_id;
var catalogue_image = cat_img[i].catalogue_image;
getBase64FromImage(catalogue_image,function (baseData64) {
console.log("baseData64===="+baseData64);
insertPageData (catalogue_image, catalogue_image_id);
},function (error) {
console.log("error====="+error);
});
}
Thanks in advance!!

Your callback "function (baseData64)" will be executed only when the image is downloaded and converted to base64. At that point, the value of "calalogue_image" and "catalogue_image_id" will contain the value of the last element in the loop - with "i = pages.length"
That is, your for loop finishes probably before even the first image is downloaded.
Actually I think the valid behaviour is that you should see only the last image, with "i = pages.length" in the database.
Try this
var processImage = function (cImg,cImgId){
var catalogueImage = cImg;
var catalogueImageId = cImgId;
return function (baseData64) {
console.log("baseData64===="+baseData64);
insertPageData (catalogueImage, catalogueImageId);
}
}
for(var i = 0; i < pages.length; i++)
{
var cat_img = res.Catalogue[0].Catalogue_img;
var catalogue_image_id = cat_img[i].catalogue_image_id;
var catalogue_image = cat_img[i].catalogue_image;
getBase64FromImage(catalogue_image,
processImage(catalogue_image, catalogue_image_id) ,
function (error) {
console.log("error====="+error);
}
);
}