We Keep Coding

Store fetch data in variable to access it later

I'm facing a probably super easy to solve problem regarding fetching.
I'd like to fetch some json datas and store it in a variable to access it later.
The problem is that I always ends up getting undefined in my variable. What's the way to do to deal with that kind of data storing ?
Here's my code.
const fetchCities = () => {
fetch('cities.json')
.then(response => response.json())
.then(data => {
return data;
});
}
let cities = fetchCities();
console.log(cities)
Already looked up for answers but couldn't find a way to do. Thanks !

You could do this very simply with async/await like this:
const fetchCities = async () => {
let cities = await fetch('cities.json');
return cities.json();
};
let cities = await fetchCities();
console.log(cities);

Sending a fetch request takes time, so the console.log works before the data arrives.
The best way to deal with fetch is using async functions and await like so:
const fetchCities = ()=>{
return fetch('cities.json');
}
async function main(){
try {
const res = await fetchCities();
const data = await res.json();
// handle the data here, this will work only after the data arrival
console.log(data);
} catch (err) {
console.log(err);
}
}
main();
Note: await can only be used in async functions, that's the main purpose of the main function.
Or if you want to use .then:
const fetchCities = ()=>{
return fetch('cities.json');
}
function main(){
fetchCities()
.then(res => res.json())
.then(data => {
// handle the data here, all you code should be here
})
.catch (err => console.log(err));
}
main();

Cypress - how to properly wait for result of an imported JS function

I am new to Cypress (and naive to JS). I would like to write a JS library as a wrapper to 3rd party APIs.
I write the API wrapper as an individual file (instead of using Cypress Custom functions) because I believe I can share the library with teams NOT using Cypress E2E tool.
The problem I am facing is "I cannot let my code to be executed sequentially in order"
From the result, I can see:
the data didn't return successfully
it looks like the "getTestPlanIdByName:20974" were executed last, but I expect it should be executed before "line 01b testPlanId:{}"
I need to help to know the correct way to handle the flow sequentially in Cypress/Javascript, thanks.
API Library(api-util.js)
let axios = require('axios');
const proxy = "http://10.8.8.8:8080/";
const apiPatToken = 'OmdrvbvvvvvvvvWZqa2E='
let proxyAgentHttps = require('https-proxy-agent');
let proxyAgentHttp = require('http-proxy-agent');
let agentHttps = new proxyAgentHttps(proxy);
let agentHttp = new proxyAgentHttp(proxy);
let config = {
baseURL: 'https://dev.3rdparty.com/mycompany/myaccount/_apis',
url: 'DUMMY_INJECTED_LATER',
httpsAgent: agentHttps,
httpAgent: agentHttp,
proxy:false,
headers: {
'Authorization': `Basic ${apiPatToken}`
}
}
export async function getTestPlanIdByName(testplan_name){
config.url = '/test/plans?api-version=5.0'
let found = ''
axios.request(config).then( resp => {
found = resp.data.value.find(function(item, index, array){
return item.name === testplan_name
})
})
.then(() => {
console.log("getTestPlanIdByName:"+found.id)
return found.id
})
.catch(err => console.log(err))
}
My Cypress code
import * as UTIL from 'api-util.js'
describe('CI-', () => {
let testPlanId = 'none'
it('01 Get TestPlanID', () => {
//use cy.log() get a Promise for flow control
cy.log()
.then(() => {
new Cypress.Promise((resolve, reject) => {
console.log("01a testPlanId:"+JSON.stringify(testPlanId))
testPlanId = UTIL.getTestPlanIdByName("TESTPLAN-Regression")
console.log("01b testPlanId:"+JSON.stringify(testPlanId))
})
})
.then(() => {
console.log("01c testPlanId:"+JSON.stringify(testPlanId))
})
});
it('02 Get TestSuitesList', () => {
console.log("02 testPlanId:"+testPlanId)
// UTIL.getTestSuitesIdList(testPlanId)
});
});

Thank you all. Cypress flow isn't 100% compatible with standard JS Promise (Wait for an own function (which returns a promise) before tests are executed). After relentless testings, I decided to use a Cypress Custom Command wrapper to wrap my in-house JS library. Though adding an extra layer may seem a little cumbersome. But I am satisfied with the result. Share my code here in case anyone might need it. :)
Cypress Code
before('Prepare TestPlanId', () => {
cy.getTestPlanIdByName(testPlanName)
.then((result) => {
testPlanId = result
console.log("#01_SDET_testplan:Prepare TestPlanId# "+testPlanId)
})
});
Cypress Custom Command
Cypress.Commands.add('getTestPlanIdByName', (wk_testplan_name) => {
return new Cypress.Promise((resolve, reject) => {
TESTPLAN_API.getTestPlanIdByName(wk_testplan_name)
.then(function (data) {
resolve(data);
})
});
})
In-house JS library
export async function getTestPlanIdByName(wk_testplan_name){
return new Promise((resolve, reject) => {
config.method = 'get'
config.url = '/test/plans?api-version=5.0'
let found = ''
axios.request(config).then( resp => {
found = resp.data.value.find(function(item, index, array){
return item.name === wk_testplan_name
})
})
.then(() => {
resolve(found.id)
})
.catch(err => console.log(err))
})
}

How to get a value in async function as soon as possible?

I'm working with Ethereum blockchain, but my problem my is JavaScript (async, await function).
Here my code simplified:
In my html
App.addBlockChain(n.username,n.first,n.last,n.email).then(value => {
**//here I need the hash of my transaction**
}).catch(error => {
alert("Errore: " + error );
});
In my App.js file
addBlockChain: async(u,n,c,e) => {
let hash;
const web3 = new Web3(App.web3Provider);
const signed = await web3.eth.accounts.signTransaction(options, account.privateKey);
const receipt = await web3.eth.sendSignedTransaction(signed.rawTransaction)
.on('transactionHash', function(hash_returned){
//I need this hash hash_returned as soon as possible in my html ***
hash= hash_returned;
})
.on('receipt', function(receipt){... })
.on('confirmation', function(confirmationNumber, receipt){ ... })
.on('error', console.error); // If a out of gas error, the second parameter is the receipt.;
return hash; //it is returned only when on('confirmation') is terminated
Any help with any code of example?
Thanks a lot in advance.

Welcome to the fantastic world of asynchronism... One way to do this would be :
const hash_returned = await App.addBlockChain(n.username, n.first, n.last, n.email);
and in your App class :
addBlockChain: async(u, n, c, e) => {
const web3 = new Web3(App.web3Provider);
const signed = await web3.eth.accounts.signTransaction(options, account.privateKey);
return new Promise(resolve => { // addBlockChain must return a Promise, so it can be "await"ed
web3.eth.sendSignedTransaction(signed.rawTransaction)
.on('transactionHash', function(hash_returned) {
resolve(hash_returned); // now that you have hash_returned, you can return it by resolving the Promise with it
})
// or more simply (equivalent) :
// .on('transactionHash', resolve)
})
}

RXJS Stream - How to wait for my STREAM to end and return DATA and then do my OPERATIONS

I have a Websocket Endpoint I am subscribing. I want to get that Data, and then operate on them.
CODE:
// Simple HTTP POST Request. Works Perfectly. I am Logged In to the API
const authenticationRequest = () => axios.post(authenticationUrl, {
user: username, password
})
.then((response) => response.data)
.catch((error) => console.error(console.error('Error Response', error)));
// WS Request. I need to wait for this to return my Data and then operate on them
const wsRequest = async () => {
// Getting the Auth Token. Working Perfectly.
const reqToken = await authenticationRequest();
// Hitting the ws Endplint. Working Perfectly.
const webSocketRequest = new WebSocket(topicDataUrl);
// Setting the Data for the First Message. Works Perfectly.
const firstMessage = {
token: reqToken,
stats: 2,
sql: "SELECT * FROM cc_payments LIMIT 100",
live: false
};
// Initialising an Empty Array. Works.
let websocketData = [];
// Opening the Endpoint
webSocketRequest.onopen = () => {
// Sending the first Message
webSocketRequest.send(JSON.stringify(firstMessage));
// On Each Message
webSocketRequest.onmessage = (streamEvent) => {
of(streamEvent).pipe(
map(event => JSON.parse(event.data)), // Parse the Data
filter(message => message.type === 'RECORD') // Filter the Data
).subscribe(
message => websocketData.push(message.data.value)// Adding each Value from each message to the Array.
);
};
};
console.log(JSON.stringify(websocketData), 'Websocket DATA'); // Empty Array
return websocketData;
};
Here I am calling it a few lines down, but still with no results. I get an empty Array.
(async function () {
const data = await wsRequest();
console.log(JSON.stringify(data), 'Data'); // Still Empty
}());
So, what am I doing wrong? Can someone, explain to me the problem? I mean I get the asynchronisity of things, but I am awaiting. I even tried setting a timeout but didn't work.
Is my stream correct? Maybe there is a problem there??
So, the RXJS Actions are asynchronous. So, I would need 2 Things.
- Close the Stream when Operationg Completed. (Tried takeUntil, takeWhile, but obviously was doing something wrong)
- Wait in order to return the Actual Data(WebsocketData).
UPDATE:
async function authenticationRequest() {
const AuthenticateWith = await axios.post(authenticationUrl, {
user: username,
password
})
.then(response => response.data)
.catch((error) => console.error('Error:', error));
return AuthenticateWith;
}
const webSocketRequest = new WebSocket(topicDataUrl);
const websocketData = new Array;
const subject = new Subject();
async function requestToWSEndpoint() {
const reqToken = await authenticationRequest();
const firstMessage = {
token: reqToken,
stats: 2,
sql: "SELECT * FROM cc_payments LIMIT 100",
live: false
};
webSocketRequest.onopen = () => {
webSocketRequest.send(JSON.stringify(firstMessage));
webSocketRequest.onmessage = (streamEvent) => {
JSON.parse(streamEvent.data).type === 'RECORD' && websocketData.push(JSON.parse(streamEvent.data).data.value);
subject.next(websocketData);
JSON.parse(streamEvent.data).type === 'END' && subject.complete();
};
};
};
(async function () {
requestToWSEndpoint();
const chartData = subject.subscribe((event) => console.log(event, 'Event')); // Event Adds to the Array and at the End I have all my Items(Filtered). It printed 100 Times.
console.log('ARRAY', chartData); // This returns [Subscriber {closed: false, _parentOrParents: null, _subscriptions: Array(1), syncErrorValue: null, syncErrorThrown: false, …}]. This is what I want. The Array.
}());

My suggestion as outlined in my comment:
const subject = new Subject();
...
}
(async function () {
wsRequest();
subject.pipe(finalize(()=> {console.log('ARRAY', websocketData);})).subscribe();
}());
Actually you dont even need a function for what you do in wsRequest, except for const reqToken = await authenticationRequest();. The rest can easily be globally scoped.
You should take a look at the documentation for rxjs operators.

Can I use crawled from Node.js in javaScript?

I'm new to javaScript and trying to crawl a website with node.js. I could check the data in console log, but want to use the data in another javaScript file. How can I fetch the data?
The problem is I've never used node.js. I do javaScript so I know how to write the code, but I don't know how the back-end or server works.
I've tried to open it in my local host but the node method (e.g., require()) didn't work. I found out it's because node doesn't work in browser.(See? very new to js)
Should I use bundler or something?
The steps I thought were,
somehow send the data as json
somehow fetch the json data and render
Here is the crawling code file.
const axios = require("axios");
const cheerio = require("cheerio");
const log = console.log;
const getHtml = async () => {
try {
return await axios.get(URL);
} catch (error) {
console.error(error);
}
};
getHtml()
.then(html => {
let ulList = [];
const $ = cheerio.load(html.data);
const $bodyList = $("div.info-timetable ul").children("li");
$bodyList.each(function(i, elem) {
ulList[i] = {
screen: $(this).find('a').attr('data-screenname'),
time: $(this).find('a').attr('data-playstarttime')
};
});
const data = ulList.filter(n => n.time);
return data;
})
.then(res => log(res));
Could you please explain what steps should I take?
Also, it would be great if I can get understood WHY the steps are needed.
Thanks alot!

you can try writing your data to a JSON file and proceed, that's one way, then you can use the data as an object in any js file
const appendFile = (file, contents) =>
new Promise((resolve, reject) => {
fs.appendFile(
file,
contents,
'utf8',
err => (err ? reject(err) : resolve()),
);
});
getHtml()
.then(html => {
let ulList = [];
const $ = cheerio.load(html.data);
const $bodyList = $("div.info-timetable ul").children("li");
$bodyList.each(function(i, elem) {
ulList[i] = {
screen: $(this).find('a').attr('data-screenname'),
time: $(this).find('a').attr('data-playstarttime')
};
});
const data = ulList.filter(n => n.time);
return data;
})
.then(res => {
return appendFile('./data.json',res.toString())
}))
.then(done => {log('updated data json')});