We Keep Coding

How to use caching with the riot-lol-api npm package?

QUESTION:
After reading this:
https://www.npmjs.com/package/riot-lol-api#caching
I am still confused. This is my first time trying to cache api responses.
For example, I do not know what values are available for YOUR_CACHE_STRATEGY and it is not explained in the docs.
Essentially, I would be looking for an example, like how can I cache and serve for 5 min the response from /lol/summoner/v3/summoners/by-name/ ?
CODE:
riotRequest.request(region.toLowerCase(), 'summoner', '/lol/summoner/v3/summoners/by-name/'+encodeURI(player), function(err, data) {
if (!err) {
var summonerID = data.id;
} else {
console.error("ERROR1: "+err);
res.render("page", {errorMessage: "Player not found !"});
}
});

The documentation is not very detailed indeed. What you need to do is basically implement the cache object as specified in the code sample from the doc (the commented area).
Below is an example of caching to an array (in memory). You could also save this array to a file or into a Redis database as suggested in the doc.
//cacheData holds objects of type {key: 123, value: "request data"}
var cacheData = []
var cacheIndex = 0
function deleteFromCache(key) {
for (var i = 0; i < cacheData.length; i++) {
if (cacheData[i].key == key) {
cacheData.splice(i, 1);
return;
}
}
}
var cache = {
get: function(region, endpoint, cb) {
for (var entry of cacheData) {
if (entry.value == data) {
//we have a cache hit
return cb(null, entry.value);
}
}
return cb(null, null);
},
set: function(region, endpoint, cacheStrategy, data) {
var key = cacheIndex++;
var value = data;
cacheData.push({key, value});
//cacheStrategy is a number representing the number of seconds to keep the data in cache
setTimeout(() => {
deleteFromCache(key);
}, cacheStrategy * 1000);
}
};
YOUR_CACHE_STRATEGY is an object that is passed to your set function in the cacheStrategy parameter. They suggest it can be a number representing the lifespan of the cache entry, so I implemented a timer to delete the cache entry after a number of seconds equal to cacheStrategy.
You would call the request using this number:
riotRequest.request(region.toLowerCase(), 'summoner', '/lol/summoner/v3/summoners/by-name/'+encodeURI(player), 30, function(err, data) {//.....
To enable caching you need to pass the cache object to the constructor of RiotRequest :
var riotRequest = new RiotRequest('my_api_key', cache);

How do I use the Request function in node.js within a for loop and have it return output in correct order?

I've been using request to iterate through several XML entries and return every article, date, and url to the console by using cheerio.js. The main issue is that the output will appear in a different order each time, as Request is an asynchronous function. I'm really inexperienced with javascript in general, and was wondering how I could retrieve consistent output (I've been reading about async and promises, I'm just unsure how to implement them).
Here is my code:
var count = 0;
for(var j = 0; j < arrNames.length; j++){
request('http://dblp.org/search/publ/api?q=' + arrNames[j], function(error, response, html){
if (!error && response.statusCode == 200){
var $ = cheerio.load(html, {xmlMode: true});
console.log($('query').text()+'\n');
$('title').each(function(i, element){
var title = $('title').eq(i).text();
var year = Number($('year').eq(i).text());
var url = $('ee').eq(i).text();
if (year >= arrTenures[count]){
console.log(title);
console.log(year);
console.log(url + '\n');
}
});
count++;
}
});
}

Though you've already found a solution, I thought I'd show you how you would do this using ES6 promises (a more modern approach for managing multiple asynchronous operations):
const rp = require('request-promise');
Promise.all(arrNames.map(item => {
return rp('http://dblp.org/search/publ/api?q=' + item).then(html => {
const $ = cheerio.load(html, {xmlMode: true});
return $('title').map(function(i, element){
const title = $(element).text();
const year = Number($('year').eq(i).text());
const url = $('ee').eq(i).text();
return {title, year, url};
}).get();
});
})).then(results => {
// results is an array of arrays, in order
console.log(results);
}).catch(err => {
console.log(err);
});
This offers several advantages:
Promise.all() puts the results in order for you.
rp() checks the 2xx status for you.
This can be chained/sequenced with other asynchronous operations more easily.
Error handling is simpler and low level errors are propagated out to the top level for you automatically.
This is throw-safe (if async errors are thrown, they are captured and propagated).

It looks like you're trying to capture the iteration number of each request, so use forEach and utilize its second parameter, which indicates the iteration index:
arrNames.forEach((q, requestIndex) => {
request('http://dblp.org/search/publ/api?q=' + q, (error, response, html) => {
if (error || response.statusCode == 200) return;
var $ = cheerio.load(html, {
xmlMode: true
});
console.log($('query').text() + '\n');
$('title').each(function(i, element) {
var title = $('title').eq(i).text();
var year = Number($('year').eq(i).text());
var url = $('ee').eq(i).text();
if (year >= arrTenures[requestIndex]) {
console.log(title);
console.log(year);
console.log(url + '\n');
}
});
});
});
As a side note, consistent indentation really improves code readability - you might consider a linter.

On your first try, you might have tried:
if (year >= arrTenures[j]) {
But noticed that didn't work. This is because of scoping issues
You can solve your problem by using an iterator like forEach(), or simply changing your for loop to use let:
for(let j = 0; j < arrNames.length; j++){
Now you can just use j in your check instead of count.
The real question though, is why are arrTenures and arrNames separate arrays? Their information clearly relates to each other, so relying on the array index to couple them seems like a bad idea. Instead, you should try and keep a single array of objects with all related information. For example:
[
{ name: 'some name', tenures: 2 },
{ name: 'another', tenures: 5 }
]

How to wait to render page with express, while API grabs data?

I am trying to load data from the twitter api, getting user information and save that in a temporary array. That array will then be loaded on the page for viewing. The array is getting loaded by the API call, but it doesn't display.
I think I need to use an asynchronous thing like React or Angular, not sure. Would love some input!
function getUserIds (userId) {
T.get('statuses/retweeters/ids', { id: userId }, function (err, data, response) {
for(var i = 0; i < data.ids.length; i++){
ids.push(data.ids[i]);
}
getUserInfo();
});
}
function getUserInfo() {
for(var i = 0; i < ids.length; i++) {
T.get('users/lookup', { user_id: ids[i] }, function (err, data, response) {
names.push(data[0].screen_name);
pics.push(data[0].profile_image_url_https);
console.log(names);
});
}
res.render('display', {names: names, pics:pics});
}

The issue is that you are running ids.length async calls and those will finish some time in the future. You have to render your page only when they are all done. But, your for loop is synchronous so you are calling res.render() before any of them have finished. In addition, your T.get() calls may finish in any order (if that matters).
I would normally use promises for coordinating multiple asynchronous operations since it is a very, very good tool for that. But, if you aren't using promises, here's a simple technique to test when you have all your results back:
function getUserInfo() {
var names = [];
var pics = [];
for(var i = 0; i < ids.length; i++) {
T.get('users/lookup', { user_id: ids[i] }, function (err, data, response) {
if (err) {
// decide what to display if you get an API error
names.push("unknown due to API error");
} else {
names.push(data[0].screen_name);
pics.push(data[0].profile_image_url_https);
console.log(names);
}
if (names.length === ids.length) {
res.render('display', {names: names, pics:pics});
}
});
}
}
As I said above, this does not necessarily collect the results in order. If you need them in order, then you could do something like this:
function getUserInfo() {
var names = new Array(ids.length);
var pics = new Array(ids.length);
var doneCntr = 0;
ids.forEach(function(id, i) {
T.get('users/lookup', { user_id: id }, function (err, data, response) {
if (err) {
// decide what to display if you get an API error
names[i] = "unknown due to API error";
} else {
names[i] = data[0].screen_name;
pics[i] = data[0].profile_image_url_https;
}
++doneCntr;
if (doneCntr === ids.length) {
res.render('display', {names: names, pics: pics});
}
});
});
}
My preferred solution would to be to use Promise.all() and use a promisified version of T.get().

How to make handle asynchronous functionality of node js to synchronous in node js

In node js I have a aws API call within for loop .
var prodAdvOptions = {
host : "webservices.amazon.in",
region : "IN",
version : "2013-08-01",
path : "/onca/xml"
};
prodAdv = aws.createProdAdvClient(awsAccessKeyId, awsSecretKey, awsAssociateTag, prodAdvOptions);
var n=100//Just for test
for (var i = 0; i <=n; i++) {
prodAdv.call("ItemSearch", {
SearchIndex : "All",
Keywords : "health,fitness,baby care,beauty",
ResponseGroup : 'Images,ItemAttributes,Offers,Reviews',
Availability : 'Available',
ItemPage : 1
}, function(err, result) {
if (err) {
console.log(err);
} else {
console.log(result);
}
});
}
Expected result is, after first result returns value, the second call request should go. But here the request/response were running asynchronously.How to make the next result wait until previous call returns response. Its okay even if it is slow.

You can use async.whilst() as your for loop. Something like this:
var async = require('async');
var prodAdvOptions = {
host : "webservices.amazon.in",
region : "IN",
version : "2013-08-01",
path : "/onca/xml"
};
var prodAdv = aws.createProdAdvClient(awsAccessKeyId, awsSecretKey, awsAssociateTag, prodAdvOptions);
var n=100;//Just for test
var i = 0; // part 1 of for loop (var i = 0)
async.whilst(
function () { return i <= n; }, // part 2 of for loop (i <=n)
function (callback) {
prodAdv.call("ItemSearch", {
SearchIndex : "All",
Keywords : "health,fitness,baby care,beauty",
ResponseGroup : 'Images,ItemAttributes,Offers,Reviews',
Availability : 'Available',
ItemPage : 1
}, function(err, result) {
if (err) {
console.log(err);
} else {
console.log(result);
}
i++; // part 3 of for loop (i++)
callback();
});
},
function (err) {
console.log('done with all items from 0 - 100');
}
);

If you prefer to use promises instead of callbacks you can simply use recursion to achieve synchronization without the need of any external library to define the flow of the code execution.
You can do that with callbacks to but the code will look horrible.

Giving a provided Node JS callback my own custom Callback

First off I thought I'd get this problem solved after this great thread: nodeJs callbacks simple example
However, I am still unsure of how to proceed. Like the title hints at: I need a callback given to a callback who already has node arguments being passed to it
Code:
(function()
var reqs = {
http: require('http'),
path: require('path'),
fs: require('fs')
};
reqs.http.createServer(function (request, response) {
response.writeHead(200, {
'Content-Type': 'text/plain'
});
response.end('Hello HTTP!');
}).listen(8080);
var printCount = function(count) {
console.log(count);
};
var callCount = function(err, list, callback) {
var count = 0;
if(err) throw err;
// console.log(err);
for (var i = 0; i < list.length; i++) {
// console.log(reqs.path.extname(list[i]));
if(reqs.path.extname(list[i]) === ".png" || reqs.path.extname(list[i]) === ".jpg")
{
count++;
console.log(count);
}
}
callback(count);
};
//count images from executing directory
var countImages = function(dirName) {
var imageCount = reqs.fs.readdir(dirName, callCount(null, null, printCount));
};
countImages(__dirname);
})();
I think the key line here is
var imageCount = reqs.fs.readdir(dirName, callCount(null, null, printCount));
I'm passing the printCount function to the same function that is called back after fs.readdir asynchronously executes but it seems that me passing null to its first two arguments is overriding Node functionality that passes the callback err and list automatically. How can I get around this? I simply want to count the images in the executing directory and be able to store that value in my main function.
Pretty new to event style programming. Any extra reading suggestions are welcome. There is tons of content out there but I really want to get this up and running for a meeting this weekend. Thanks guys!

you can't quite do what you are doing, you are doing callCount(null, null, printCount) which executes the function. But you need to pass a function as a callback. What you want is something like the following, which captures the call back you want and returns a function you can pass as a callback to your api call
var callCount = function(callback) {
return function(err, list) {
var count = 0;
if(err) throw err;
// console.log(err);
for (var i = 0; i < list.length; i++) {
// console.log(reqs.path.extname(list[i]));
if(reqs.path.extname(list[i]) === ".png" || reqs.path.extname(list[i]) === ".jpg")
{
count++;
console.log(count);
}
}
callback(count);
}
}
and then
reqs.fs.readdir(dirName, callCount(printCount));