We Keep Coding

POSTing a nested object ruins the encoding in NodeJS Backend

I am having trouble sending JSON data that contains special characters like ` to my Node Express server.
I posted How can I render UTF-8 characters in JSX without using dangerouslySetInnerHTML in 2020 earlier, but I think I am approaching the problem incorrectly.
The Problem:
When I submit my JSON form data using an HTTP Client, my req.body HTMLifies special characters:
// Sending a nested JSON.stringified Object
const response1 = await fetch(
`${baseUrl}/chhands/${granthState.lastChhand.id}/pauris`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
pauri: formattedTuk,
last_pauri_id: granthState.lastPauri?.id,
}),
}
);
Here is a result of the console.logging {pauri: formattedTuk, last_pauri_id: granthState.lastPauri?.id}:
What it looks like on the backend when I submit the nested JSON object:
Here is where the where I am even MORE confused:
Here, instead of sending a nested object, I only send the formattedTuk [Array] object, which looks like:
[
{
line_number: 1,
content_unicode: 'ਆਪ ਅਛਤ ਸਮਰੱਥ ਪ੍ਰਭੁ ਦਈ ਬਡਾਈ ਨਾਮ ।',
content_gs: 'Awp ACq smr`Q pRBu deI bfweI nwm [',
content_transliteration_english:
"aap achhat samara'th prabh dhiee baddaiee naam |",
first_letters: 'ਆਅਸਪਦਬਨ',
thamkis: [],
vishraams: [],
},
];
// Sending the raw array object stringified {}
const response2 = await fetch(
`${baseUrl}/chhands/${granthState.lastChhand.id}/pauris`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(formattedTuk),
}
);
What it looks like on the backend when I submit the ONLY the Array Object:
Other things I've tried:
Used Axios (instead node-fetch) and it gave the same effect
Used POSTMAN and it had the same effect
Double checked my Content-Type and charset but it all looks good.
Here is my Express middleware settings. I was using body-parser before, but it seems that express.json() is giving me the same effect. I've also seen people recommend using qs.stringify() but what I am doing seems fairly simple.
app.use(morgan('tiny'));
app.use(helmet());
app.use(express.json());
app.use(cors());
Updates:
Here is the exact "Copy as Fetch" value:
fetch("http://localhost:1469/api/v1/chhands/53/pauris", {
"headers": {
"accept": "*/*",
"accept-language": "en-US,en;q=0.9",
"content-type": "application/json",
"sec-fetch-dest": "empty",
"sec-fetch-mode": "cors",
"sec-fetch-site": "same-site"
},
"referrer": "http://localhost:3000/",
"referrerPolicy": "strict-origin-when-cross-origin",
"body": "{\"pauri\":[{\"line_number\":1,\"content_unicode\":\"ਆਪ ਅਛਤ ਸਮਰੱਥ ਪ੍ਰਭੁ ਦਈ ਬਡਾਈ ਨਾਮ ।\",\"content_gs\":\"Awp ACq smr`Q pRBu deI bfweI nwm [\",\"content_transliteration_english\":\"aap achhat samara'th prabh dhiee baddaiee naam |\",\"first_letters\":\"ਆਅਸਪਦਬਨ\",\"thamkis\":[],\"vishraams\":[]}],\"last_pauri_id\":60}",
"method": "POST",
"mode": "cors",
"credentials": "omit"
});

Errors while trying to "Put" JSON Data

I am working with the Hubspot API and I am trying to modify the close date of a deal via the "PUT" Method by sending JSON Data. But I am getting errors such as
{ status: 'error', message: 'Invalid input JSON on line 1, column
15: Can not deserialize instance of java.util.ArrayList out of
START_OBJECT token', correlationId:
'b8b47229-184d-40b3-b402-9e3dd684b217', requestId:
'd364fe8dac5e876639928dd0d04045fd' }
This is the code that I have written-
fetch('https://api.hubapi.com/deals/v1/deal/103361780?hapikey=', {
method: 'put',
headers: {
'Accept': 'application/json, text/plain, */*',
'Content-Type': 'application/json'
},
body: JSON.stringify({"properties":{name: "closedate", "value": 1528744207881}})
}).then(res=>res.json())
.then(res => console.log(res));
This is the JSON Data that I am trying to pass
{
"properties":[
{
"name": "closedate",
"value": 1528744207881
}
]
};
And here is the documentation of making a PUT request via the Hubspot API.
I am able to successfully update the value via POSTMAN.
Any help on this matter would be greatly appreciated.

You are missing brackets - [] and on the backend, they are waiting for an array to deserialize it to Arraylist.
Try fetch with this body:
{"properties":[{"name": "closedate", "value": 1528744207881}]}

Difference between fetch and $.getjson

I've been playing around with the Google+ API and trying to get the profile image url of a Google+ user with this url:
https://www.googleapis.com/plus/v1/people/{user_id}?key={API_KEY}
No OAuth is needed and you can also give it a try here (no API key needed):
https://developers.google.com/apis-explorer/#p/plus/v1/plus.people.get?userId=116725099929439898086&_h=1&
At first I used the Fetch API to fetch the data since I also want to use a service worker to do that:
fetch('https://www.googleapis.com/plus/v1/people/116725099929439898086?key=MY_API_KEY')
.then(function(response){
console.log(response);
});
But it only gives me this response:
{
body: ReadableStream
bodyUsed: false
headers: Headers
ok: true
status: 200
statusText: ""
type: "cors"
url: "https://www.googleapis.com/plus/v1/people/115681458968227650592?key=MY_API_KEY"
}
However, if I use jQuery's getJSON method instead:
$.getJSON( "https://www.googleapis.com/plus/v1/people/115681458968227650592?key=MY_API_KEY", function( data ) {
console.log(data);
});
It works like a charm and I can get what I need:
{
"kind": "plus#person",
"etag": "\"FT7X6cYw9BSnPtIywEFNNGVVdio/DgskSQn7XXHCvjcdFBFkiqEbsfo\"",
"gender": "male",
"urls": [ ... ],
"objectType": "person",
"id": "116725099929439898086",
"displayName": "Kevin Lai",
...
}
Can someone explain why they will to such different behaviors? And also, how can I fix the problem while still using the Fetch API so I can still do this asynchronous task in a service worker?

Change the fetch() call to use response.json() (see MDN docs) to extract the JSON from the response body.
fetch('https://www.googleapis.com/plus/v1/people/116725099929439898086?key=MY_API_KEY')
.then(function (response){
return response.json();
})
.then(function (json){
console.log(json);
});

HTTPS request not posting body of REST request

I'm trying to POST to an API endpoint on my server. I know my endpoint works because if I use Advanced REST Client, I can hit it and get a JSON response as expected. The problem seems to be that no data is being sent in the body of my request despite calling request.write(postData) which contains a key, value pair. Without this data being sent in the body, my server returns a 401 error as expected without this information. Printing out the content of the POST server-side is empty but I'm clueless as to why it's empty.
var postData = querystring.stringify({
"access_token" : accessToken,
"id": applianceId
});
var serverError = function (e) {
log("Error", e.message);
context.fail(generateControlError(requestName, "DEPENDENT_SERVICE_UNAVAILABLE", "Unable to connect to server"));
};
var callback = function(response) {
var str = "";
response.on("data", function(chunk) {
str += chunk.toString("utf-8");
});
response.on("end", function() {
result = generateResult(CONTROL, requestName.replace("Request", "Confirmation"), messageId);
context.succeed(result);
});
response.on("error", serverError);
};
var options = {
hostname: REMOTE_CLOUD_HOSTNAME,
port: 443,
path: REMOTE_CLOUD_BASE_PATH + "/" + endpoint,
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded"
}
};
var request = https.request(options, callback);
request.on("error", serverError);
//This doesn't seem to write anything since if I print out the POST
//data server-side it's empty; however, if I print out the value of
//postData here, it looks as expected: 'access_token=xxxxx'
request.write(postData);
request.end();

I have testing you code again httpbin.org/post and it seems that it is working.
I believe that the issue related to, that your should POST application/json and not "application/x-www-form-urlencoded
Please try to change the header
headers: {
"Content-Type": "application/json"
}
Then, try to change the postData to JSON string:
var postData=JSON.stringify({access_token:"xxxxx"})
To be sure that problem you success to send and the problem is not local (maybe there is an issue in your server), change the target to mirror URL:
var options = {
hostname: "httpbin.org",
path:'/post',
port: 443,
method: "POST",
headers: {
"Content-Type": "application/json"
}
};
If there is no problem in your NodeJS version, the is the response you should get: (It is mean that the server got the posted data)
{
"args": {},
"data": "{\"access_token\":\"xxxxx\"}",
"files": {},
"form": {},
"headers": {
"Content-Length": "24",
"Content-Type": "application/json",
"Host": "httpbin.org"
},
"json": {
"access_token": "xxxxx"
},
"origin": "5.29.63.30",
"url": "https://httpbin.org/post"
}
BTW: I really recommend you to move to a library to manage the request for you:
https://github.com/request/request - Very popular
https://github.com/request/request-promise - For popular who like to use the Promise syntax (The next thing in JavaScript)
https://github.com/visionmedia/superagent - For people who like to write same code in Browser & Server

How to send user information through directline botconnector

I'm building a mobile client to talk to a bot built with microsoft botbuilder through botconnector directline. I want to send things like the unique id of the user speaking with the bot, so my bot can operate on this user. Currently I'm just POSTing to directline, but when I add additional things in the body, my bot can't retrieve it. I'm probably doing something really simple wrong. Would love to get your help on this!
POST code from client:
sendToBot: function(setUpObj, message, returnCallback){
var postURL=baseURL+"/"+setUpObj.conversationId+"/messages"
var postOptions ={
method: 'POST',
headers: {
"Authorization": setUpObj.conversationToken,
"content-type": "application/json"
},
body: JSON.stringify({
"text": message,
"from": {
'address': setUpObj.currentUserUid
}
})
}
fetch(postURL, postOptions)
.then(
(response)=>response.text()
)
.then(
(responseText)=>{
this.getResponse(setUpObj, returnCallback)
}
)
}
and I'm accessing the the currentUserUid on the server by
session.message.from.address
Thanks for you patience.

Have you tried setting the channelData in the json? It's described as "data sent unmodified between client and bot" and can look like:
{
"id": "CuvLPID4kDb|000000000000000004",
"conversationId": "CuvLPID4kDb",
"created": "2017-02-22T21:19:51.0357965Z",
"from": "examplebot",
"text": "Hello!",
"channelData": {
"examplefield": "abc123"
}
}
https://docs.botframework.com/en-us/core-concepts/channeldata/
So in your code it could look like:
var postOptions ={
method: 'POST',
headers: {
"Authorization": setUpObj.conversationToken,
"content-type": "application/json"
},
body: JSON.stringify({
"text": message,
"channelData": {
"from": {
"address": setUpObj.currentUserUid
}
}
})
}