We Keep Coding

DynamoDB.DocumentClient With Dynamic Attributes and Atomic Counter

I'm working a use case where a dynamoDB update should:
Dynamically upsert (update if present, insert if not present) an item, without hardcoding the item's components.
Use the DynamoDB Document Client for simplicity
And in the same atomic operation, update a simple counter
I started with an excellent utility method by Daniel Barrel at https://stackoverflow.com/a/63511693/15369972 that provides a general utility method for the update with dynamic values, but without the atomic counter.
I've attempted to add the atomic counter capability by adding the counter and its incrementor into the parameter objects after the dynamic values are loaded, but am getting a static value in the counter on update instead of a value that increments by one with each call.
Where is this going wrong? I call the modified update function with a table name, a dynamic javascript object, and an array containing the hash and sort key:
await update(tableName, jsonObjectToStore, ['myHashKey', 'mySortKey'])
And the modified update method that's not incrementing as I'd like, is:
async function update (tableName, item, idAttributeNames) {
var params = {
TableName: tableName,
Key: {},
ExpressionAttributeValues: {},
ExpressionAttributeNames: {},
UpdateExpression: "",
ReturnValues: "UPDATED_NEW"
};
for (const attname of idAttributeNames) {
params["Key"][attname] = item[attname];
}
let prefix = "set ";
let attributes = Object.keys(item);
for (let i=0; i<attributes.length; i++) {
let attribute = attributes[i];
if (!idAttributeNames.includes(attribute)) {
params["UpdateExpression"] += prefix + "#" + attribute + " = :" + attribute;
params["ExpressionAttributeValues"][":" + attribute] = item[attribute];
params["ExpressionAttributeNames"]["#" + attribute] = attribute;
prefix = ", ";
}
}
// Add the counter
params["UpdateExpression"] += ", #nImports = :nImports + :incr";
console.log(params["UpdateExpression"])
console.log(params["ExpressionAttributeValues"])
params["ExpressionAttributeValues"][":incr"] = 1;
params["ExpressionAttributeValues"][":nImports"] = 0;
console.log(params["ExpressionAttributeValues"])
console.log(params["ExpressionAttributeNames"])
params["ExpressionAttributeNames"]["#nImports"] = 'nImports'
console.log(params["ExpressionAttributeNames"])
await docClient.update
return await docClient.update(params).promise();
}

Worked with AWS Support to find a reasonable solution. They also were not sure how to do an atomic counter using the ddb document client (as opposed to the low level client which has many documented examples) but suggested the ADD command, which has the side effect of an atomic update on a numeric field.
So, with the example below, we construct our dynamic update from the object to be stored, then append the ADD statement in the update expression (without a comma!), and add what is in effect a numeric incrementor to the ExpressionAttributeValues for nImports. Like this, which should be a complete working lambda example. There's a few console.log statements to show what's happening:
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient();
async function update (tableName, item, idAttributeNames) {
var params = {
TableName: tableName,
Key: {},
ExpressionAttributeValues: {},
ExpressionAttributeNames: {},
UpdateExpression: "",
ReturnValues: "UPDATED_NEW"
};
for (const attname of idAttributeNames) {
params["Key"][attname] = item[attname];
}
let prefix = "set ";
let attributes = Object.keys(item);
for (let i=0; i<attributes.length; i++) {
let attribute = attributes[i];
if (!idAttributeNames.includes(attribute)) {
params["UpdateExpression"] += prefix + "#" + attribute + " = :" + attribute;
params["ExpressionAttributeValues"][":" + attribute] = item[attribute];
params["ExpressionAttributeNames"]["#" + attribute] = attribute;
prefix = ", ";
}
}
console.log('params before adding atomic counter is:', params)
// Add the counter using the ADD syntax
params["UpdateExpression"] += " ADD #nImports :nImports"
params["ExpressionAttributeValues"][":nImports"] = 1;
params["ExpressionAttributeNames"]["#nImports"] = 'nImports'
console.log('params after adding atomic counter is:', params)
try {
const result = await docClient.update(params).promise();
console.log('after await, result is ', result);
return result;
} catch (err) {
console.log('err is ', err)
}
};
exports.handler = async (event) => {
const item = {title: 'sometitle', site_url: "www.amazon.com", key: "G"};
const body = await update('test_table', item, ['title', 'site_url']);
const response = {
statusCode: 200,
body: JSON.stringify(body),
};
return response;
}

The kind folks at AWS did a little more digging, and also pointed out an error in the initial code that, when corrected, should increment as desired using the SET operator.
Basically the original code didn't properly target the variable for increment. So a corrected version where we add the incremented variable should be:
console.log('params before adding atomic counter is:', params)
// Add the counter
params["UpdateExpression"] += ", #nImports = #nImports + :incr";
params["ExpressionAttributeValues"][":incr"] = 1;
//params["ExpressionAttributeValues"][":nImports"] = 0;
params["ExpressionAttributeNames"]["#nImports"] = 'nImports'
console.log('params after adding atomic counter is:', params)```
I'm sticking with the original ADD answer because I like the differentiation it gives over the properties inserted by the SET, but both seem valid and I wanted to include the correction as well

How to call web service endpoints one after other?

I want to implement a function in JavaScript which calls a series of web service endpoint and checks for a value in the response of the API call.
I need to achieve it in a way that the first endpoint page is called first then the there would be a filter method to filter out the specific object from the response. If the object is found, this process should break and the object must be returned. However if the object is not found in the first endpoint, then the second endpoint must be called and the same process is repeated until the object is found.
The Web service endpoint that I am working on is:
https://jsonmock.hackerrank.com/api/countries?page=1
This API returns a list of country data. Here the value of page query varies from 1 to 25. I need to call the endpoint and check for a specific country from 1 to 25 until the country object is found.
I tried achieving this using JavaScript Promise and Fetch API and couldn't think of a way to call the APIs one after the other.
I am really looking forward for your answer. Thank you in advance.

You can use async and await for this:
async function findCountry(country) {
for (let page = 1; page < 26; page++) {
console.log("page = " + page); // for debugging only
let response = await fetch("https://jsonmock.hackerrank.com/api/countries?page=" + page);
let {data} = await response.json();
let obj = data.find(obj => obj.name == country);
if (obj) return obj;
}
}
let country = "Belgium";
findCountry(country).then(obj => {
if (obj) {
console.log("The capital of " + country + " is " + obj.capital);
} else {
console.log("Could not find " + country);
}
});
If you know that the data is sorted by country name, then you could reduce the average number of requests by using a binary search.

Here's a way that you can do it.
const url = 'https://jsonmock.hackerrank.com/api/countries'
const fetchFromApi = async (countryName, page) => {
const res = await fetch(`${url}?page=${page}`)
return await res.json()
}
const getCountryFromResults = (countryName, data) => {
const country = countryName.toLowerCase()
return data.find(({name}) => name.toLowerCase() === country)
}
const findCountry = async (countryName) => {
let page = 1;
let totalPages = 1;
while(page <= totalPages) {
const res = await fetchFromApi(countryName, page);
if(totalPages < res.total_pages) {
totalPages = res.total_pages
}
const country = getCountryFromResults(countryName, res.data)
if(country){
return country
}
page = page + 1
}
}
( async () => {
console.log(await findCountry("Afghanistan"))
console.log(await findCountry("Argentina"))
}
)()

Unable to iterate over an ElementArrayFinder

I'm a beginner writing e2e Javascript tests using Protractor. I have been trying to identify an array of elements that have a text property associated with them. I'm able to target them using
groupedNodeNumbers : ElementArrayFinder;
this.groupedNodeNumbers = this.element.all(by.css(".side-label-background + .side-label"));
This is supposed to give me an array of elements on which I can call getText() to extract the value associated with each of them.
validateCountGraphNodes = async() => {
let count = 0;
console.log('Inside')
this.groupedNodeNumbers.each((groupedNodeElement) => {
groupedNodeElement.getText().then((num) => {
console.log('Inside loop')
count += parseInt(num, 10);
console.log(count);
});
});
}`
I am able to log ('Inside') but not ('Inside Loop') and hence my function fails to retrieve the text associated with each element.
Could someone please point where I'm going wrong here?

Since getText() applied returns string instead of an array of strings (existing issue), you can try the following:
const cssSelector = ".side-label-background + .side-label";
$$(cssSelector).getText().then(textArrNotArr => {
for(let i = 0; i< textArrNotArr.lenght; i++) {
console.log('arr[i] = ', textArrNotArr[i]);
}
});
or
$$(cssSelector).count().then(elFinderArray => {
elFinderArray.forEach(elFinder => {
elFinder.getText().then((txt, index) => {
console.log(index);
});
});
});
note: $$('cssSelector') can be used instead of element.all(by.css('cssSelector'))

Nightmare JS use .evaluate return variable

I'm new to Javascript, Node.js, and Nightmare. I'm trying to add a feature to count the number of titles on a page and then wait until the total number of titles increases by 5.
Here is the snippet:
.evaluate(function () {
var elements = Array.from(document.getElementsByClassName('title'));
var firstCount = elements.length;
var higherCount = firstCount + 5;
var selector = '\'#price' + higherCount + '\'';
return selector;
})
.wait(selector)
When I run I get the error:
.wait(selector)
^ ReferenceError: selector is not defined at Object.
How do I get the variable selector to the .wait command?

According to the docs, the function evaluate passes the return value from the function on to the caller.
http://phantomjs.org/api/webpage/method/evaluate.html
so in your example, the call would be
var selector = page.evaluate(function () {
var elements = Array.from(document.getElementsByClassName('title'));
var firstCount = elements.length;
var higherCount = firstCount + 5;
var selector = '\'#price' + higherCount + '\'';
return selector;
});
If for some reason the contents of the evaluate call are asyncronous, you could return a promise;
page.evaluate(function () {
return new Promise( function(success, error) {
// whatever is the async thing
success(value);
// or if there is an error
error("details of error");
}
}).then( function (results) {
// we have our results
}).catch( function(err) {
// ... oops
});
This might then be compatible with async / await, depending on the context.

You can add something to your Selector html and target it with wait() assuming your Selector is an html element. Something like this.
.evaluate(function () {
var selector = document.getElementById("demo");
selector.className += " waitForThis";
})
.wait(".waitForThis")
I don't think you can use evaluate's return value for wait() though.

Synchronously retrieve a random number of record IDs, change button text and Asynchronously retrieve the corresponding records from the "server"

I need to change the text and style of the "Get next" button to "Loading...",
Synchronously retrieve a random number of record IDs from a "server" and Asynchronously retrieve the corresponding records from the "server", only proceeding when all records have been received.
Sort the records in date order, oldest first and at the end reset the button to its original state
The code is as follows
let loading = true;
const buttonHandler = function () {
loading = !loading;
toggleButton(loading);
getRecords();
};
const btn = document.getElementById('get-records');
btn.addEventListener('click', buttonHandler);
function toggleButton(loaded) {
btn.innerHTML = loaded ? 'Loading...' : 'Get next';
btn.classList.toggle('button-not-loading');
btn.classList.toggle('button-loading');
}
function getRecords() {
// getting the IDs of the records to fetch is a synchronous operation
// you don't need to change this call, it should return the IDs
const ids = Server.getIds();
const allTheRecords = [];
// getting each corresponding record is an async operation
ids.forEach(function (recordId) {
Server.getRecord(recordId, function (error, data) {
// if the fetch is unsuccessful the callback function is invoked with the error only
// if the fetch is successful the callback is invoked with error variable set to null,
// and data variable will hold the response (i.e. the record you wanted to retrieve)
if (error) {
console.log(error);
} else {
error = null;
allTheRecords.push(data);
}
});
// you can get a SINGLE record by calling Server.getRecord(recordId, callbackFunction)
// callbackFunction takes 2 parameters, error and data
// invocation as follows
// you need to make sure the list is not rendered until we have the records...
//but need to allow for any fetch errors or app will hang
// i.e. a record you request might not exist - how would you allow for this?
// when you have the records, call processRecords as follows
processRecords(allTheRecords);
});
}
function processRecords(records) {
toggleButton(true);
const sortedRecords = sortRecords(records);
let html = '';
let tr;
sortedRecords.forEach(function (index, value, array) {
tr = '';
tr +=
'<tr>' +
'<td>' + value.date + '</td>' +
'<td>' + value.name + '</td>' +
'<td>' + value.natInsNumber + '</td>' +
'<td>' + value.hoursWorked + '</td>' +
'<td>' + value.hourlyRate + '</td>' +
'<td>' + (value.hoursWorked * value.hourlyRate) + '</td>' +
'</tr>';
html += tr;
});
document.getElementById('results-body').innerHTML = html;
addTotals(sortedRecords);
}
function sortRecords(records) {
let sorted = records.sort(function (a, b) {
return new Date(a.date) - new Date(b.date);
});
// sort results in date order, most recent last
return sorted;
}
function addTotals(records) {
let hours = 0;
let paid = 0;
records.forEach(function (value, index) {
hours += value.hoursWorked;
paid += (value.hoursWorked * value.hourlyRate);
});
document.getElementById('totals-annot').innerHTML = 'TOTALS';
document.getElementById('totals-hours').innerHTML = hours;
document.getElementById('totals-paid').innerHTML = paid;
}

there is no question there, but ill give a vague pseudo code answer which should be enough to point you in the right direction.
Keyword = Promise.
const loadRecordIds = () => {
return new Promise((resolve, reject) => {
jQuery.get('http://localhost/recordIds').then((data) => {
// do something with the data ... e.g parse/validate
resolve(data);
});
});
};
const loadRecords = (recordIds) => {
return new Promise((resolve, reject) => {
jQuery.get('http://localhost/records?recordIds='+recordIds).then((data) => {
// check the data for errors etc
resolve(data);
});
});
};
const toggleButton = () => {
// toggle your button styles
};
// and you use the functions in sequence using .then() or async keyword(if you have a preprocessor or dont care about old browsers)
loadRecordIds().then((recordIds) => {
// now you have your recordIds loaded
toggleButton();
loadRecords(recordIds).then((records) => {
// now you have your records available for further processing
});
});
// with async await keywords you could do the same like this.
try {
const recordIds = await loadRecordIds();
toggleButton();
const records = await loadRecords(recordIds);
} catch (error) {
// handle errors
}
If you dont know what promises are, google them.
// ok, ill throw in a quick sample of an async code that runs in "sync" using promises.
step1 = () => {
return new Promise((resolve, reject) => {
setTimeout(() => {
// time has run out now, and its time for the second step
// calling "resolve" will call the "then" function and allows the code to continue
// whatever you pass in as the argument for resolve() will be a parameter in the "then()" function callback.
resolve('3000 seconds has passed, time to continue.');
}, 3000);
});
};
step2 = () => {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve('2000 seconds has passed, time to continue.');
}, 2000);
});
};
step1().then((message) => {
console.log(message);
step2().then((message) => {
console.log(message);
setTimeout(() => {
console.log('and now the script is done...all in sequence');
}, 2000);
});
});
/*
this will output
3000 seconds has passed, time to continue.
2000 seconds has passed, time to continue.
and now the script is done...all in sequence
*/