Node fetch receive a ReadableStream and receive an incomplete response - javascript

I am making a request Node fetch receive a ReadableStream and receive an incomplete response. The problem seen as the ReadableStream is not getting complete in the await.
Request:
static async postData(url = "") {
// Default options are marked with *
const response = await fetch(url, {
method: "POST", // *GET, POST, PUT, DELETE, etc.
mode: "same-origin", // no-cors, *cors, same-origin
cache: "default", // *default, no-cache, reload, force-cache, only-if-cached
credentials: "same-origin", // include, *same-origin, omit
headers: {
"Content-Type": "application/json",
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: "follow", // manual, *follow, error
referrerPolicy: "no-referrer", // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
//body: JSON.stringify(dados), // body data type must match "Content-Type" header
});
const stream = await response.body?.getReader().read();
let jsonBuffer = Buffer.from(stream?.value!);
let jsonString = jsonBuffer.toString("utf8");
console.log(jsonString);
return JSON.parse(jsonString); // parses JSON response into native JavaScript objects
}
Response:
{"retorno":{"status_processamento":"3","status":"OK","pagina":1,"numero_paginas":1,"contatos":[{"contato":{"id":"715461091","codigo":"","nome":"Fabio Moreno","fantasia":"","tipo_pessoa":"F","cpf_cnpj":"","endereco":"","numero":"","complemento":"","bairro":"Vila Medon","cep":"","cidade":"Americana","uf":"SP","email":"linkiez#gmail.com","fone":"","id_lista_preco":0,"id_vendedor":"0","nome_vendedor":"","s`
Error:
[1] SyntaxError: Unexpected end of JSON input
[1] at JSON.parse ()
[1] at TinyERP.postData (file:///home/linkiez/Desktop/Projetos/JCMserver3/dist/services/tinyERP.js:22:21)
[1] at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[1] at async aprovarOrcamento (file:///home/linkiez/Desktop/Projetos/JCMserver3/dist/controllers/orcamentoController.js:259:40)
[1] nodemon --experimental-specifier-resolution=node -q dist/index.js exited with code SIGINT
[0] tsc --watch exited with code SIGINT

You've said you're using Node.js's fetch, which is meant to be compatible with the web platform's fetch.
Your code isn't reading the entire response. Here's the documentation for the read() method on the default reader returned by getReader() with no arguments:
The read() method of the ReadableStreamDefaultReader interface returns a Promise providing access to the next chunk in the stream's internal queue.
(my emphasis) That's not the entire response, that's just the first chunk of the response.
But there's no need for that code to be anywhere near that complicated, just use the built-in json method to read the entire response and parse it from JSON; see the *** comments below:
static async postData(url = "") {
const response = await fetch(url, {
method: "POST",
mode: "same-origin",
cache: "default",
credentials: "same-origin",
headers: {
"Content-Type": "application/json",
},
redirect: "follow",
referrerPolicy: "no-referrer",
// *** It seems odd that there's no `body` here, given it's a POST
// saying that it's *sending* JSON (the `Content-Type` header above
// says what you're *sending*, not what you're expecting back).
});
// *** This was missing, but it's important; `fetch` only rejects on
// *network* errors, not HTTP errors:
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`);
}
// *** Fully read the response body and parse it from JSON:
return await response.json();
}
Here's a post on my anemic old blog about the need for the ok check I added above.

Related

Error in with fetch call "net::ERR_CONNECTION_RESET" and "Uncaught (in promise) TypeError: Failed to fetch"

I'm doing a fetch in react and getting these errors and I cannot figure out how to fix it. I am using TypeScript and a C# rest service. It works fine in postman but having these issues in the client.
I have tried disabling all browser extensions and tried other browsers also. This did not work.
I'm expecting to receive a status "201" back from the REST Call.
on button click
<Button className="w-100 btn btn-lg btn-primary" type='submit'onClick={e => {e.preventDefault() handleForm()}}>Register</Button>
javascript:
` async function handleForm() {
console.log(JSON.stringify({ ...registration }))
const endpoint = 'http://localhost:44309/api/Users/Register';
const data = {
email: registration.email,
userName: registration.username,
password: registration.password,
confirmPassword: registration.passwordConfirmation,
userTypeId: 4
};
// Default options are marked with *
const response = await fetch(endpoint,
{
method: 'POST', // *GET, POST, PUT, DELETE, etc.
mode: 'cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': 'application/json'
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
body: JSON.stringify(data) // body data type must match "Content-Type" header
});
return response.json();
}`
here is the C# rest method:
` [HttpPost("[action]")]
public IActionResult Register([FromBody] ApplicationUser applicationUser)
{
var userExists = _dbContext.AppUsers.FirstOrDefault(u => u.Email == applicationUser.Email);
//todo: add validation code
if (userExists != null)
{
return BadRequest("User with the same email address already exists");
}
applicationUser.Password = HashService.HashPassword(applicationUser.Password);
#if (!DEBUG)
applicationUser.ConfirmPassword = "True";
#endif
_dbContext.AppUsers.Add(applicationUser);
_dbContext.SaveChanges();
return StatusCode(StatusCodes.Status201Created);
}`
There was a few issues with this error that I found. 1) was a cors issue, so I added cors to my c# code. 2) My API was using https and the react client was running under http. I needed to configure the create-react-app to run over https instead of http once I got that configured it worked great. I had a difficult time getting the https to work on windows on the client until I found this post. https://medium.com/#praveenmobdev/localhost-as-https-with-reactjs-app-on-windows-a1270d7fbd1f

API fetch is not responding

I was working on a React APP which fetches data from https://restcountries.com/v2/all and now I have an error.
useEffect(() => {
fetch(`https://restcountries.com/v2/all`)
.then((r) => r.json())
.then((data) => {
if (data !== undefined) {
setCountries(data);
} else {
alert('Can´t Load Data');
}
});
}, []);
**
use this format with header
** ##
// Example POST method implementation:
async function postData(url = '', data = {}) {
// Default options are marked with *
const response = await fetch(url, {`enter code here`
method: 'POST', // *GET, POST, PUT, DELETE, etc.
mode: 'cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': 'application/json'
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
body: JSON.stringify(data) // body data type must match "Content-Type" header
});
return response.json(); // parses JSON response into native JavaScript objects
}
}, []);
You are getting a CORS error which means target domain of api (restcountries) does not allow other domains to fetch data.
The solution to this problem is a server side browser or headless browser. Like selenium and puppeteer
https://www.selenium.dev/
https://github.com/puppeteer
But i have tested and api is giving me data in browser with same fetch code. I cant reproduce the problem. Its an issue with something else
this is happening due to multiple reason like due to authentication or your are not sending token in request header second due to server down or may be your are passing wrong param to request third one my be this endpoint can me access by only specific domains url.

Sending data between Flask and JS using AJAX for chrome extension

I'm trying to use AJAX calls to send data back and forth between my Javascript frontend for my chrome extension and the Flask API where I plan to use my Machine Learning code.
content.js
console.log("Application GO");
function colorChanger() {
let tweets = document.querySelectorAll("article");
tweets.forEach(function (tweet) {
$(document).ready(function () {
$.ajax({
type: "POST",
contentType: "application/json;charset=utf-8",
url: "/_api_call",
traditional: "true",
data: JSON.stringify({tweet}),
dataType: "json"
});
});
tweet.setAttribute("style", "background-color: red;");
});
}
let timer = setInterval(colorChanger, 2000);
flask code
from flask import Flask, flash, request, redirect, url_for
from flask_cors import CORS
app = Flask(__name__)
CORS(app)
#app.route('/_api_call', methods=['GET'])
def fake_news_detector():
data = request.get_json()
with open('temp.txt', 'w') as f:
f.write(data)
return data
Error
Uncaught ReferenceError: $ is not defined
content.js:11 (anonymous function) // which points to line - $(document).ready(function () {
I'm new to both Javascript and Flask. Any help would be really helpful. Thanks a lot !
$(document).ready and $.ajax requires jQuery
fetch and window.addEventListener works in almost all latest browsers
$(document).ready => window.addEventListener('DOMContentLoaded', function(evt) {})
$.ajax => fetch
Note: Calling $(document).ready again and again inside loop for each tweet is not a good option it will run a bunch of code again and again instead setInterval can be called once after document loading completes.
content.js
async function Request(url = '', data = {}, method = 'POST') {
// Default options are marked with *
const response = await fetch(url, {
method: method, // *GET, POST, PUT, DELETE, etc.
mode: 'cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': "application/json;charset=utf-8",
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
body: JSON.stringify(data) // body data type must match "Content-Type" header
});
return response.json(); // parses JSON response into native JavaScript objects
}
console.log("Application GO");
function colorChanger() {
let tweets = document.querySelectorAll("article");
tweets.forEach(function (tweet) {
let response = Request("/_api_call", {tweet});
tweet.setAttribute("style", "background-color: red;");
});
}
window.addEventListener('DOMContentLoaded', (event) => {
console.log('Called once after document load');
let timer = setInterval(colorChanger, 2000);
});
First of all you don't have jQuery installed so you can't access $ and your error
Uncaught ReferenceError: $ is not defined
is saying that. you should include jQuery in your js code in order to use $ and call ajax. just follow #scrappedcola comment and follow the instructions there to add jQuery script.
second you need to define the endpoint as POST.
#app.route('/_api_call', methods=['POST'])

Cannot do http-post in mongodb shell

I am trying to use a shell script to save data to mongodb then send the same to elasticsearch for analysis.
Below is a psudo code in the shell script that illustrate the issue.
mongo $mongo_conn --eval "
// other javascritp code
db.collection.insertOne(insertData);
http_post_to_elasticsearch(url,insertData);
function http_post_to_elasticsearch(url,insertData){
// which http_post() method can work here?
// both fetch and xhr are undefined
// tried installing npm install -g xmlhttprequest but 'require' is also undefined
}
Below are the functions that have tried but seem not to be supported
async function http_post(url = '', data = {}) {
print('starting http_post()')
// Default options are marked with *
const response = await fetch(url, {
method: 'POST', // *GET, POST, PUT, DELETE, etc.
mode: 'cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': 'application/json'
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
body: JSON.stringify(data) // body data type must match "Content-Type" header
});
return response.json(); // parses JSON response into native JavaScript objects
}
async function xhr_post(url = '', data = {}) {
const xhr = require('xmlhttprequest').XMLHttpRequest;
// const xhr = new XMLHttpRequest();
// listen for `load` event
xhr.onload = async () => {
// print JSON response
if (xhr.status >= 200 && xhr.status < 300) {
// parse JSON
const response = JSON.parse(xhr.responseText);
printjson(response);
// return response;
}
};
// create a JSON object
const json = data;
// open request
xhr.open('POST', url);
// set `Content-Type` header
xhr.setRequestHeader('Content-Type', 'application/json');
// send rquest with JSON payload
xhr.send(JSON.stringify(json));
}
mongo shell does not provide a browser nor a nodejs-compatible runtime.
You can look through the server js tests to see if there is anything there that resembles an http client.
See also ReferenceError: require is not defined in MongoDB shell.

Sending GET data via fetch() in body instead of specifying it in the URL

I am trying to convert this jQuery call to native Javascript using fetch() as mentioned in MDN (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch#Supplying_request_options).
$.ajax(
{
method: "GET",
url: CB_ABS_URI + "ajax/get-checkin.php",
dataType: "json",
data: { DwellingUnitID:DwellingUnitID },
})
to
// Example POST method implementation:
async function postData(url = '', data = {}) {
// Default options are marked with *
const response = await fetch(url, {
method: 'GET', // *GET, POST, PUT, DELETE, etc.
mode: 'cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': 'application/json'
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *client
body: JSON.stringify(data) // body data type must match "Content-Type" header
});
return await response.json(); // parses JSON response into native JavaScript objects
}
postData(CB_ABS_URI + "ajax/get-checkin.php", { DwellingUnitID: DwellingUnitID })
.then((data) => {
console.log(data); // JSON data parsed by `response.json()` call
});
But I can't seem to send GET data in the body. Is adding the query to ajax/get-checkin.php the only way ?
But I can't seem to send GET data in the body
fetch makes a clear distinction between the query string in the URL and the data in the request body (unlike jQuery which switches between them depending on the request method).
Is adding the query to ajax/get-checkin.php the only way ?
Yes, see the documentation:
If you want to work with URL query parameters:
var url = new URL("https://geo.example.org/api"),
params = {lat:35.696233, long:139.570431}
Object.keys(params).forEach(key => url.searchParams.append(key, params[key]))
fetch(url).then(/* … */)

Categories