We Keep Coding

Javascript multiple promises not working with q and request middle

So this is my dilemma. I have a list of movies, witch I have scraped from a website, then I want to add additional properties to my newly constructed object(json)
Now the omdi api witch I am using supports searching for a movie by title.
Then I make a get request using request and q middlewares. When I receive information from omdb api in the call back I add that data to the object.
Now the next part is where my problem lies. Now I want to return a new Request using data from the previous request. Now I make an new get Request and return it but then() func isin't returning anything. But I don't seem to realize what I am doing wrong.
Here is my code..
var promises = [];
films.forEach(function (film) {
// Get omdbapi information
promises.push(HttpService.getContent(configExternal.omodburl + '?t=' + film.title.trim() + '&y=' + film.year + '&plot=true&tomatoes=true&r=json').then(function (data) {
var result = JSON.parse(data);
if(Boolean(result.Response) === true) {
film.omdb.push(result);
}
var imdbid = result.imdbID;
return HttpService.getContent(configExternal.themoviedburl + imdbid + '/videos?api_key=' + configExternal.themoviedbkey);
}).then(function(data) {
film.trailers = [];
film.trailers.push(JSON.parse(data));
}).catch(function (err) {
logger.error().info('Error getting ' + film.title + ' from omdb, ErrorMessage : ' + err);
}));
});
//--------------------------------
// When all promises have finished
//--------------------------------
Promise.all(promises).then(function (data, err) {
// do stuff with the data
});
And here is my getContent func
var Service = {
getContent: function(url) {
var deferred = q.defer();
request(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
deferred.resolve(body);
} else {
deferred.reject(error);
}
});
return deferred.promise;
}
};

Problem solved. There wasn't anything wrong with the request as Roamer said. But the moviedata base limits by 40 request per 10 sek witch I didn't know :)

Parse database job is not getting all results from http request

I have the following code set up as a job in the Parse Cloud Code for my application.
Parse.Cloud.job("requestLocations", function (request, response) {Parse.Cloud.httpRequest({
url: 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=29.7030428,-98.1364808&radius=900&types=restaurant&key=AIzaSyCTg0x68Q6lrCAo6-A37zkxge81jDEKpvo'
}).then(function (httpResponse) {
// Success
response.success("Success");
var parsedData = JSON.parse(httpResponse.text);
var Location = Parse.Object.extend("Locations");
for (var i = 0; i < parsedData.results.length; i++) {
var restaurant = new Location();
var placeId = parsedData.results[i].place_id;
var name = parsedData.results[i].name;
var vicinity = parsedData.results[i].vicinity;
var point = new Parse.GeoPoint({
latitude: parsedData.results[i].geometry.location.lat,
longitude: parsedData.results[i].geometry.location.lng
});
restaurant.set("placeId", placeId);
restaurant.set("name", name);
restaurant.set("vicinity", vicinity);
restaurant.set("location", point);
restaurant.save(null, {
success: function (location) {
console.log("Object ID: " + location.id);
},
error: function (location, error) {
console.log("Failed to create object, with error code: " + error.message);
}
});
}
}, function (httpResponse) {
// Error
response.error('request failed with response code ' + httpResponse)
});});
As you can see, this HTTP request should return a total of 14 places. Unfortunately, it will only return 9 places and it would also seem that which 9 are return can change. I am assuming there is a problem with the way my function is put together. Can anyone help me remedy this issue. I would like to return as many places as I want based on the radius of the HTTP request.
Thank You

The http request is done right, with a promise that's fulfilled when the request is complete. But your then() block tries to create several objects in a loop, not waiting for them all to finish, and failing to call response.success. Fix it like this...
// break it into understandable chunks, too, so, here's a function
// to build a Locations object from the http data
function locationFromResult(result) {
var Location = Parse.Object.extend("Locations");
var restaurant = new Location();
var placeId = result.place_id;
var name = result.name;
var vicinity = result.vicinity;
var point = new Parse.GeoPoint({
latitude: result.geometry.location.lat,
longitude: result.geometry.location.lng
});
restaurant.set("placeId", placeId);
restaurant.set("name", name);
restaurant.set("vicinity", vicinity);
restaurant.set("location", point);
return restaurant;
}
Parse.Cloud.job("requestLocations", function (request, response) {
var url = 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=29.7030428,-98.1364808&radius=900&types=restaurant&key=AIzaSyCTg0x68Q6lrCAo6-A37zkxge81jDEKpvo';
Parse.Cloud.httpRequest({url: url}).then(function (httpResponse) {
var parsedData = JSON.parse(httpResponse.text);
var locations = parsedData.results.map(function(result) {
return locationFromResult(result);
});
// this is important, saveAll of the new objects before returning
// this can also be accomplished by saving the objects individually and using Parse.Promise.when()
return Parse.Object.saveAll(locations);
}).then(function(result) {
response.success(JSON.stringify(result));
}, function(error) {
response.error(JSON.stringify(error));
});
});

Parse.Cloud.job promise not working

What I am trying to do here are:
Remove all contents in a class first, because every day the events.json file will be updated. I have my first question here: is there a better way to remove all contents from a database class on Parse?
Then I will send a request to get the events.json and store "name" and "id" of the result into a 2D array.
Then I will send multiple requests to get json files of each "name" and "id" pairs.
Finally, I will store the event detail into database. (one event per row) But now my code will terminate before it downloaded the json files.
Code:
function newLst(results) {
var event = Parse.Object.extend("event");
for (var i = 0; i < results.length; i++){
Parse.Cloud.httpRequest({
url: 'https://api.example.com/events/'+ results[i].name +'/'+ results[i].id +'.json',
success: function(newLst) {
var newJson = JSON.parse(newLst.text);
var newEvent = new event();
newEvent.set("eventId",newJson.data.id);
newEvent.set("eventName",newJson.data.title);
newEvent.save(null, {
success: function(newEvent) {
alert('New object created with objectId: ' + newEvent.id);
},
error: function(newEvent, error) {
alert('Failed to create new object, with error code: ' + error.message);
}
});
},
error: function(newLst) {
}
});
}
};
Parse.Cloud.job("getevent", function(request, status) {
var event = Parse.Object.extend("event");
var query = new Parse.Query(event);
query.notEqualTo("objectId", "lol");
query.limit(1000);
query.find({
success: function(results) {
for (var i = 0; i < results.length; i++) {
var myObject = results[i];
myObject.destroy({
success: function(myObject) {
},
error: function(myObject, error) {
}
});
}
},
error: function(error) {
alert("Error: " + error.code + " " + error.message);
}
});
var params = { url: 'https://api.example.com/events.json'};
Parse.Cloud.httpRequest(params).then(function(httpResponse) {
var results = [];
var jsonobj = JSON.parse(httpResponse.text);
for (var i = 0; i < jsonobj.data.length; i++) {
var tmp2D = {"name":"id"}
tmp2D.name = [jsonobj.data[i].name];
tmp2D.id = [jsonobj.data[i].id];
results.push(tmp2D);
}
newLst(results);
}).then(function() {
status.success("run job");
}, function(error) {
status.error(error);
});
});

I think my original answer is correct as a standalone. Rather than make it unreadable with the additional code, here it is made very specific to your edit.
The key is to eliminate passed callback functions. Everything below uses promises. Another key idea is decompose the activities into logical chunks.
A couple of caveats: (1) There's a lot of code there, and the chances that either your code is mistaken or mine is are still high, but this should communicate the gist of a better design. (2) We're doing enough work in these functions that we might bump into a parse-imposed timeout. Start out by testing all this with small counts.
Start with your question about destroying all instances of class...
// return a promise to destroy all instances of the "event" class
function destroyEvents() {
// is your event class really named with lowercase? uppercase is conventional
var query = new Parse.Query("event");
query.notEqualTo("objectId", "lol"); // doing this because the OP code did it. not sure why
query.limit(1000);
return query.find().then(function(results) {
return Parse.Object.destroyAll(results);
});
}
Next, get remote events and format them as simple JSON. See the comment. I'm pretty sure your idea of a "2D array" was ill-advised, but I may be misunderstanding your data...
// return a promise to fetch remote events and format them as an array of objects
//
// note - this differs from the OP data. this will evaluate to:
// [ { "name":"someName0", id:"someId0" }, { "name":"someName1", id:"someId1" }, ...]
//
// original code was producing:
// [ { "name":["someName0"], id:["someId0"] }, { "name":["someName1"], id:["someId1"] }, ...]
//
function fetchRemoteEvents() {
var params = { url: 'https://api.example.com/events.json'};
return Parse.Cloud.httpRequest(params).then(function(httpResponse) {
var results = [];
var remoteEvents = JSON.parse(httpResponse.text).data;
for (var i = 0; i < remoteEvents.length; i++) {
var remoteEvent = { "name": remoteEvents[i].name, "id": remoteEvents[i].id };
results.push(remoteEvent);
}
return results;
});
}
Please double check all of my work above regarding the format (e.g. response.text, JSON.parse().data, etc).
Its too easy to get confused when you mix callbacks and promises, and even worse when you're generating promises in a loop. Here again, we break out a simple operation, to create a single parse.com object based on one of the single remote events we got in the function above...
// return a promise to create a new native event based on a remoteEvent
function nativeEventFromRemoteEvent(remoteEvent) {
var url = 'https://api.example.com/events/'+ remoteEvent.name +'/'+ remoteEvent.id +'.json';
return Parse.Cloud.httpRequest({ url:url }).then(function(response) {
var eventDetail = JSON.parse(response.text).data;
var Event = Parse.Object.extend("event");
var event = new Event();
event.set("eventId", eventDetail.id);
event.set("eventName", eventDetail.title);
return event.save();
});
}
Finally, we can bring it together in a job that is simple to read, certain to do things in the desired order, and certain to call success() when (and only when) it finishes successfully...
// the parse job removes all events, fetches remote data that describe events
// then builds events from those descriptions
Parse.Cloud.job("getevent", function(request, status) {
destroyEvents().then(function() {
return fetchRemoteEvents();
}).then(function(remoteEvents) {
var newEventPromises = [];
for (var i = 0; i < remoteEvents.length; i++) {
var remoteEvent = remoteEvents[i];
newEventPromises.push(nativeEventFromRemoteEvent(remoteEvent));
}
return Parse.Promise.when(newEventPromises);
}).then(function() {
status.success("run job");
}, function(error) {
status.error(error);
});
});

The posted code does just one http request so there's no need for an array of promises or the invocation of Promise.when(). The rest of what might be happening is obscured by mixing the callback parameters to httpRequest with the promises and the assignment inside the push.
Here's a clarified rewrite:
Parse.Cloud.job("getevent", function(request, status) {
var promises = [];
var params = { url: 'https://api.example.com'};
Parse.Cloud.httpRequest(params).then(function(httpResponse) {
var results = [];
var jsonobj = JSON.parse(httpResponse.text);
for (var i = 0; i < jsonobj.data.length; i++) {
// some code
}
}).then(function() {
status.success("run job");
}, function(error) {
status.error(error);
});
});
But there's a very strong caveat here: this works only if ("// some code") that appears in your original post doesn't itself try to do any asynch work, database or otherwise.
Lets say you do need to do asynch work in that loop. Move that work to a promise-returning function collect those in an array, and then use Promise.when(). e.g....
// return a promise to look up some object, change it and save it...
function findChangeSave(someJSON) {
var query = new Parse.Query("SomeClass");
query.equalTo("someAttribute", someJSON.lookupAttribute);
return query.first().then(function(object) {
object.set("someOtherAttribute", someJSON.otherAttribute);
return object.save();
});
}
Then, in your loop...
var jsonobj = JSON.parse(httpResponse.text);
var promises = [];
for (var i = 0; i < jsonobj.data.length; i++) {
// some code, which is really:
var someJSON = jsonobj.data[i];
promises.push(findChangeSave(someJSON));
}
return Parse.Promise.when(promises);

Service function returning empty object

I've got a factory function that won't return a variable I'm trying to set in my controller. I don't get an error though, just the variable won't get set to what it's suppose to.
spApp.factory('SiteService', function ($q){
var rootUrl = window.location.protocol + "//" + window.location.hostname;
var siteMap;
//returns object containing info about all sites within collection
var getSiteMap = function () {
siteMap = {};
var promise = $().SPServices({
operation: "GetAllSubWebCollection",
async: true
});
promise.then(
function (response){
map = {}; //init the map
var web = $(response).find("Web").map(function () {
return $(this).attr('Url');
});
var webTitle = $(response).find("Web").map(function () {
return $(this).attr('Title');
});
// create map
for (var i = 0; i < web.length; i++) {
var item = web[i],
title = webTitle[i],
parts = item.split('/'),
domain = parts.splice(0, 3).join('/'),
current;
if (!map[domain]) map[domain] = {url:domain, title:title ,children:{}};
current = map[domain].children;
for (var index in parts) {
var part = parts[index];
if (!current[part]) {
current[part] = {url:domain+'/'+parts.slice(0,index+1).join('/'), title:title, children:{}};
}
current = current[part].children;
}
}
siteMap = map;
}, function(reason){
alert('FAILED:' + reason);
})
console.log(siteMap);
return siteMap;
}
return{
getSiteMap:getSiteMap
}
});

Try chaining your promises like this:
var getSiteMap = function () {
siteMap = {};
var promise = $().SPServices({
operation: "GetAllSubWebCollection",
async: true
});
return promise.then(function(response){ //return your promise
// all you code
siteMap = map;
return siteMap; //return a value to another .then in the chain
});
}
Use it like this:
SiteService.getSiteMap().then(function(siteMap){
});

The issue you have is that you are working with promises. When you put your console.log outside your then() function, you are logging the variable before it has actually been resolved.
If you put your console.log inside your then() function (after sitemap is assigned), it should show the correct value, but you still won't be able to access it reliably.
I think the simplest way for you to access the siteMap value after it has been populated with data is to pass in a callback function. Eg:
var getSiteMap = function (_callback) {
siteMap = {};
$().SPServices({
operation: "GetAllSubWebCollection",
async: true
}).then(function(response){
// Process the data and set siteMap
// ...
siteMap = map;
// now pass siteMap to the callback
_callback(siteMap);
});
You would then use this in your controller like so:
SiteService.getSiteMap(function(sitemap){
// Do something with your sitemap here
});
Now while this will work, it is just one quick example, and not necessarily the best way. If you don't like callbacks, you could create a second promise that resolves only when siteMap is assigned. Also depending on your use case for getSiteMap(), you may want to cache the value, otherwise the request will be called every time.

Populating an object with ajax in a loop

I need to pull data from a series of .csv files off the server. I am converting the csvs into arrays and I am trying to keep them all in an object. The ajax requests are all successful, but for some reason only the data from the last request ends up in the object. Here is my code:
var populate_chart_data = function(){
"use strict";
var genders = ["Boys","Girls"];
var charts = {
WHO: ["HCFA", "IWFA", "LFA", "WFA", "WFL"],
CDC: ["BMIAGE", "HCA", "IWFA", "LFA", "SFA", "WFA", "WFL", "WFS"]
};
var fileName, fileString;
var chart_data = {};
for (var i=0; i < genders.length; i++){
for (var item in charts){
if (charts.hasOwnProperty(item)){
for (var j=0; j<charts[item].length; j++) {
fileName = genders[i] + '_' + item + '_' + charts[item][j];
fileString = pathString + fileName + '.csv';
$.ajax(fileString, {
success: function(data) {
chart_data[fileName] = csvToArray(data);
},
error: function() {
console.log("Failed to retrieve csv");
},
timeout: 300000
});
}
}
}
}
return chart_data;
};
var chart_data = populate_chart_data();
The console in Firebug shows every ajax request successful, but when I step through the loops, my chart_data object is empty until the final loop. This is my first foray into ajax. Is it a timing issue?

There are two things you need to consider here:
The AJAX calls are asynchronous, this means you callback will only be called as soon as you receive the data. Meanwhile your loop keeps going and queueing new requests.
Since you're loop is going on, the value of filename will change before your callback is executed.
So you need to do two things:
Push the requests into an array and only return when the array completes
Create a closure so your filename doesn't change
.
var chart_data = [];
var requests = [];
for (var j=0; j<charts[item].length; j++) {
fileName = genders[i] + '_' + item + '_' + charts[item][j];
fileString = pathString + fileName + '.csv';
var onSuccess = (function(filenameinclosure){ // closure for your filename
return function(data){
chart_data[filenameinclosure] = csvToArray(data);
};
})(fileName);
requests.push( // saving requests
$.ajax(fileString, {
success: onSuccess,
error: function() {
console.log("Failed to retrieve csv");
},
timeout: 300000
})
);
}
$.when.apply(undefined, requests).done(function () {
// chart_data is filled up
});

I'm surprised that any data ends up in the object. The thing about ajax is that you can't depend on ever knowing when the request will complete (or if it even will complete). Therefore any work that depends on the retrieved data must be done in the ajax callbacks. You could so something like this:
var requests = [];
var chart_data = {};
/* snip */
requests.push($.ajax(fileString, {
/* snip */
$.when.apply(undefined, requests).done(function () {
//chart_data should be full
});