how to fetch json from API using vanilla JS through Protractor - javascript

I am trying to download distance between 2 locations from tomtom api.
Protractor will not let me use
*fetch - fetch is not defined - please use import
*import - Cannot use import statement outside of module
*when I add
{
type: module
}
to package.json - protractor stops working, as no entire code is a module of ES
*browser.get - opens http with json data, but I cannot extract it.
Is there any other way? I tried to import json to a different file and export response.data, but the module error stops me from doing that also.

Protractor is for testing angular webpages, but you can have the browser execute arbitrary javascript, but to use fetch, you need to use window
function getTomTomData() {
//replace this with your tomtom api call, and transform the response
return window.fetch(TOM_TOM_URL);
}
browser.executeScript(getTomTomData).then(res=> {
//do something with the response
});

I did not manage to run node-fetch on my script as Protractor kept rejecting the import. I managed to to sort it out with require 'https'
const https = require('https');
let measureDistance = async function(pickup, dropoff) {
let url = `https://api.tomtom.com/routing/1/calculateRoute/${pickup[0]}%2C${pickup[1]}%3A${dropoff[0]}%2C${dropoff[1]}/json?routeType=shortest&avoid=unpavedRoads&key=uwbU08nKLNQTyNrOrrQs5SsRXtdm4CXM`;
await https.get(url, res => {
let body = '';
res.on('data', chunk => {
body += chunk;
});
res.on("end", () => {
try {
let json = JSON.parse(body);
howFar = json.routes[0].summary.lengthInMeters;
} catch (error) {
console.error(error.message);
}
}).on("error", (error) => {
console.error(error.message);
});
});
};
Also I used to put require on top of the file like in Ruby, which seemed to be another issue.

Related

Error [object object] to JSON: Converting circular structure to JSON [duplicate]

I am using request package for node.js
Code :
var formData = ({first_name:firstname,last_name:lastname,user_name:username, email:email,password:password});
request.post({url:'http://localhost:8081/register', JSON: formData}, function(err, connection, body) {
exports.Register = function(req, res) {
res.header("Access-Control-Allow-Origin", "*");
console.log("Request data " +JSON.stringify(req));
Here I am getting this error :
TypeError: Converting circular structure to JSON
Can anybody tell me what is the problem
JSON doesn't accept circular objects - objects which reference themselves. JSON.stringify() will throw an error if it comes across one of these.
The request (req) object is circular by nature - Node does that.
In this case, because you just need to log it to the console, you can use the console's native stringifying and avoid using JSON:
console.log("Request data:");
console.log(req);
I also ran into this issue. It was because I forgot to await for a promise.
Try using this npm package. This helped me decoding the res structure from my node while using passport-azure-ad for integrating login using Microsoft account
https://www.npmjs.com/package/circular-json
You can stringify your circular structure by doing:
const str = CircularJSON.stringify(obj);
then you can convert it onto JSON using JSON parser
JSON.parse(str)
I was able to get the values using this method, found at careerkarma.com
Output looks like this.
I just run this code in the debugger console. Pass your object to this function.
Copy paste the function also.
const replacerFunc = () => {
const visited = new WeakSet();
return (key, value) => {
if (typeof value === "object" && value !== null) {
if (visited.has(value)) {
return;
}
visited.add(value);
}
return value;
};
};
JSON.stringify(circObj, replacerFunc());
I forgotten to use await keyword in async function.
with the given systax
blogRouter.put('/:id', async (request, response) => {
const updatedBlog = Blog.findByIdAndUpdate(
request.params.id,
request.body,
{ new: true }
);
response.status(201).json(updatedBlog);
});
Blog.findByIdAndUpdate should be used with the await keyword.
use this https://www.npmjs.com/package/json-stringify-safe
var stringify = require('json-stringify-safe');
var circularObj = {};
circularObj.circularRef = circularObj;
circularObj.list = [ circularObj, circularObj ];
console.log(stringify(circularObj, null, 2));
stringify(obj, serializer, indent, decycler)
It's because you don't an async response For example:
app.get(`${api}/users`, async (req, res) => {
const users = await User.find()
res.send(users);
})
This is because JavaScript structures that include circular references can't be serialized with a"plain" JSON.stringify.
https://www.npmjs.com/package/circular-json mentioned by #Dinesh is a good solution. But this npm package has been deprecated.
So use https://www.npmjs.com/package/flatted npm package directly from the creator of CircularJSON.
Simple usage. In your case, code as follows
import package
// ESM
import {parse, stringify} from 'flatted';
// CJS
const {parse, stringify} = require('flatted');
and
console.log("Request data " + stringify(req));
If you are sending reponse , Just use await before response
await res.json({data: req.data});
Came across this issue in my Node Api call when I missed to use await keyword in a async method in front of call returning Promise. I solved it by adding await keyword.
I was also getting the same error, in my case it was just because of not using await with Users.findById() which returns promise, so response.status().send()/response.send() was getting called before promise is settled (fulfilled or rejected)
Code Snippet
app.get(`${ROUTES.USERS}/:id`, async (request, response) => {
const _id = request.params.id;
try {
// was getting error when not used await
const user = await User.findById(_id);
if (!user) {
response.status(HTTP_STATUS_CODES.NOT_FOUND).send('no user found');
} else {
response.send(user);
}
} catch (e) {
response
.status(HTTP_STATUS_CODES.INTERNAL_SERVER_ERROR)
.send('Something went wrong, try again after some time.');
}
});
For mongodb
so if you are getting errors while fetching data from MongoDB then the problem is async
previously
app.get('/users',(req,res,next)=>{
const user=chatUser.find({});
if(!user){
res.status(404).send({message:"there are no users"});
}
if(user){
res.json(user);
}
})
After
app.get('/users',async(req,res,next)=>{
const user=await chatUser.find({});
if(!user){
res.status(404).send({message:"there are no users"});
}
if(user){
res.json(user);
}
})
I came across this issue when not using async/await on a asynchronous function (api call). Hence adding them / using the promise handlers properly cleared the error.
I had a similar issue:-
const SampleFunction = async (resp,action) => {
try{
if(resp?.length > 0) {
let tempPolicy = JSON.parse(JSON.stringify(resp[0]));
do something
}
}catch(error){
console.error("consoleLogs.Utilities.XXX.YYY", error);
throw error;
}
.
.
I put await before JSON.parse(JSON.stringify(resp[0])).
This was required in my case as otherwise object was read only.
Both Object.create(resp[0]) and {...resp[0]} didn't suffice my need.
If an object has a different type of property like mentioned in the above image, JSON.stringify() will through an error.
Try this as well
console.log(JSON.parse(JSON.stringify(req.body)));
TypeError: Converting circular structure to JSON in nodejs:
This error can be seen on Arangodb when using it with Node.js, because storage is missing in your database. If the archive is created under your database, check in the Aurangobi web interface.

How to require directories using async / await

I am trying to load data which is in javascript files into an object.
All the files are like this:
module.exports = {
test: ‘qwerty’
}
I’m using require, but I have to load several directories so need to do the loads one at a time.
I tried wrapping it in a promise:
function load(fileOrDirPath) {
return new Promise(function(resolve, reject) {
let data;
debug(`Requiring: [${fileOrDirPath}]`);
try {
data = require(fileOrDirPath);
}
catch(e) {
return reject(e);
}
debug(`Loaded data: [${JSON.stringify(data, 0, 2)}]`);
return resolve(data);
});
}
I use the function:
const dirPath = ‘redacted’
const data = await load(dirPath);
debug(`Loaded: [${JSON.stringify(data, 0, 2)}]`);
And the logs show that the data is loaded inside the function. However outside the data is always null.
Why doesn’t the function await?
I tried looking on npm for a module but couldn’t find any.
How can I load a directory of javascript files recursively into an object?

Handling errors in javascript/react when calling backend APIs

I am building a react-native app, and I am starting to implement a more robust and sophisticated error-handling system, specifically for handling server errors when making http requests. Here is a basic example of how I am currently making http requests in my app.
I have a 'client.js' file which is essentially just a wrapper around axios. I have a 'get' method that looks like this:
const get = async (endpoint, config = {}) => {
try {
const result = await axios.get(domain + endpoint, config);
return result;
} catch (error) {
throw new Error(error.message);
}
};
Then, I have a file for each api endpoint that I need to access. For example, I have a 'posts.js' file, and in that file I have a 'getPosts' method:
const getPosts = async (userID, page, pageSize) => {
try {
const response = await client.get(
`${endpoint}?userID=${userID}&page=${page}&pageSize=${pageSize}`
);
return response.data;
} catch (error) {
throw new Error(error.message);
}
};
And then finally, in the component that is calling getPosts, I have a function that looks something like this:
const loadPosts = async () => {
try {
const response = await getPosts();
// do something with the response from the server
} catch (error) {
// display an error message to the client
}
}
Obviously this is a very simple example of what a request might look like, but this is the basic structure that I use throughout my app. The problem I am having is that it seems very repetitive and messy to have to wrap almost all of my functions in a try/catch block, and then basically raise an error object until I get to the function that is actually going to handle the error. Is there some sort of 'design method' for error handling that simplifies and centralizes this process? Perhaps something similar to an express-middleware when creating a node server? Or is this a standard way to handle errors in javascript?
Thank you to anyone who can help!
As you are using axios as the http library here, so you can take a look at axios interceptor in order to hook the response and do something with that before passing it to the consumer. This will help you to respond to errors raised from once cental place.
axios.interceptors.response.use((response) => {
return response;
}, function(error) {
// do what you want to do with the error.
return Promise.reject(error)
});
Or with ES5 syntax
axios.interceptors.response.use(function (response) {
// Do something with response data
return response;
}, function (error) {
// Not 200 Ok
// Do something with response error
return Promise.reject(error);
});

SyntaxError: Unexpected end of JSON input - why is that?

This is the code I have written using express and node.js
const express = require("express");
const https = require("https");
const app = express();
app.get("/", function(req, res) {
// Url to my api key
const url = "https://api.spoonacular.com/recipes/random?apiKey=...&number=1";
https.get(url, function(response) {
response.on("data", function (data) {
console.log(JSON.parse(data));
// const theRecipe = JSON.parse(data);
console.log(data);
});
});
res.send("The server is up and running");
});
app.listen(3000, function () {
console.log("Server started at port 3000");
});
When I refresh my webpage on localhost, on console I get the following error:
quote
SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at IncomingMessage. (C:\Users\ArunBohra\Desktop\FoodRecipes\app.js:12:33)
quote
Can anyone find what is the problem with my code.
The data event fires when a chunk of data from the response has arrived. You are trying to parse the first chunk as if it were the complete JSON text.
You need to collect the pieces from each data event but wait until the end event fires before joining them together into the complete string of JSON that you can parse.
There is an example of fetching and parsing JSON in the documentation.
You might want to look at modules like axios and node-fetch which take care of that (and the JSON parsing) for you while providing modern Promise based APIs.
If you have a new enough version of Node, you can use the native Fetch API.
If you use a package like node-fetch you can get the whole thing in one go instead of what you have now which is chunking the data
const fetch = require('node-fetch');
const url = "https://api.spoonacular.com/recipes/random?apiKey=...&number=1";
fetch(url)
.then(response => response.json())
.then(data => console.log(data));
In addition to other answers, you can do it without another package.
https.get(url, function (response) {
let result = "";
response.on("data", function (data) {
result += chunk;
});
res.on('end', () => {
// now you have the combined result
console.log(result);
});
});

How to get innertext from body of a URL

I am trying to get innerText of https://www.example.com/ in a nodejs appplication. I tried using request npm module to fetch body of URL as shown below:
function getBodyText() {
request({
url:'https://www.example.com/'
}, (error, response, body) => {
console.log(body.innerText);
});
}
The above code displays body of the current page I am in (https:www.google.com). Am I missing anything?
In your above code, the body value is a just a string. innerText on the other hand assumes body is a DOM Node.
In Node, the DOM is not present like it would be in the browser, so in order to access the DOM Nodes that were returned you'll need to load body using the package Cheerio. You can assign the transform property of the request options to load the body into a DOM using cheerio.load(). Then you can use traditional DOM selectors to traverse body.
In order to use the transform option on your request options object, you'll need to switch from request to request-promise. (npm i --save request request-promise) They function nearly identically except that request-promise will return an A+ promise using Bluebird where request uses a more traditional error first callback.
Since Cheerio uses its own implementation of jQuery you can refer to their docs for the differences when interacting with the DOM returned.
const cheerio = require('cheerio')
const request = require('request-promise')
request({
method: 'GET',
uri: 'https://google.com'
transform: body => cheerio.load(body)
})
.then($ => {
console.log($('p').text)
})
If you didn't want to switch over to request-promise you can still do this and make it use Promises
const cheerio = require('cheerio')
const request = require('request')
const getDOMFromURI = uri => {
return new Promise((resolve, reject) => {
request(uri, (err, res, body) => {
if (err) {
return reject(err)
}
return resolve(cheerio.load(body))
})
})
}
getDOMFromURI('https://google.com').then($ => {
console.log($('p').text)
})
You have to use some other technology combination. It seems that you want to scrap website for data. Please use phantomjs or nightmare or puppeteer or any other headless browser.
A small example for you how to get first result title with puppeteer
const puppeteer = require('puppeteer');
let scrape = async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('https://www.google.com.pk/search?q=puppeteer');
await page.waitFor(2000);
const result = await page.evaluate(() => {
let title = document.querySelector('h3').innerText;
return {
title
}
});
browser.close();
return result;
};
scrape().then((value) => {
console.log(value); // Success!
});
from the docs, you can use a string as first argument if using GET method:
request('https://www.example.com', function (error, response, body) {
console.log('error:', error); // Print the error if one occurred
console.log('statusCode:', response && response.statusCode); // Print the
response status code if a response was received
console.log('body:', body); // Print the HTML for the Google homepage.
const dom = new JSDOM(body);
console.log(dom.window.document.querySelector("p").textContent);
});
see https://www.npmjs.com/package/request
You also might want to try the request-promise module or axios (which is the number 1 lib for making HTTP requests)
Once you've got the body back you may need to use JSDOM or some other lib to convert the body into a document object which you can then traverse using normal JS methods or even jQuery / another DOM traversal lib.

Categories