We Keep Coding

No body present upon receipt with https POST with body using nodejs client and golang server

I am attempting to send a rest request over https using the following
// Construct the POST auth body and options
let postData = JSON.stringify({
username: this.state.username,
password: this.state.password
});
console.log(postData);
var options = {
host: this.state.peerAddress,
port: this.state.port,
path: "/api/" + this.state.channelName + "/user-authorisation",
method: "POST",
headers: {
"Content-Type": "application/json",
"Content-Length": postData.length
}
};
console.log(options);
var postRequest = https.request(options, function(res) {
console.log("\nstatus code: ", res.statusCode);
res.setEncoding("utf8");
res.on("data", function(data) {
console.log(JSON.parse(data));
});
});
e.preventDefault();
// post the data
postRequest.write(postData);
postRequest.end();
The request is sent to the server but there is no body present. I have tested this by sending the same request over postman and it works fine. I am sending the request on Chrome if that makes any difference? I get the following errors on the console but these happen after the request has been sent I believe so that are not really an issue at this point.
request.js:149 OPTIONS https://<<address>>:<<port>><<path>> net::ERR_CONNECTION_REFUSED
localhost/:1 Uncaught (in promise) TypeError: Failed to fetch
Any help would be greatly appreciated!
Cheers

How to use the Watson Text2Speech REST API in a cordova App?

There are many samples: How to use Watson Services in Node.JS, but if you using the REST API with HTTP calls I run into problems with the authorization.
The documentation of the API tells about the commandline curl interface,
but there is no concrete sample for HTTP calls for web or Hybrid Applications using javascript.
In my situation I want to use the Watson Text2Speech in a cordova Mobile APP, for this I going to build a factory.
The http calls I use do work for other APIs, but what I am doing wrong here?
Any missing format?
Can anyone help?
Which looks like this:
.factory('GetSpeech', ['$http','$cordovaDialogs','$cordovaMedia','Base64', function($http,
$cordovaDialogs,
$cordovaMedia,
Base64){
// http://ngcordova.com/docs/plugins/media/
// https://www.ibm.com/watson/developercloud/doc/speech-to-text/input.shtml
// https://www.npmjs.com/package/cordova-plugin-speech-recognition-feat-siri
var watson_url = "https://stream.watsonplatform.net/text-to-speech/api/v1/synthesize";
var watson_token_url = "https://stream.watsonplatform.net/authorization/api/v1/token?url=https://stream.watsonplatform.net/text-to-speech/api";
var watson_token = "";
var username='YOUR_KEY';
var password='YOUR_PW';
var authdata = 'Basic ' + Base64.encode(username + ':' + password);
console.log(">>> Watson - authdata: ",authdata);
var the_get_header = "{'Authorization':'"+ authdata +"','Content-Type':'application/json'}";
var message = "";
var getSpeech_innner = function (){ $http({method: 'GET',
url: watson_token_url,
headers: the_get_header
}).then( function successCallback(response) {
console.log(">>> GetToken Success:",response);
watson_token=response;
var the_post_header = "{'X-Watson-Authorization-Token':'"+ watson_token +"','Content-Type':'application/json','Accept':'audio/wav'}";
var the_post_text = JSON.stringify({ "text":"This is the first sentence of the paragraph. Here is another sentence. Finally, this is the last sentence."
});
$http({
method: 'POST',
url: watson_url,
headers: the_post_header,
data: the_post_text
}).then(function successCallback(response) {
// this callback will be called asynchronously
// when the response is available
console.log(">>> GetSpeech Success:",response);
message = "Success: " + response;
alert(message);
return true;
}, function errorCallback(response) {
// called asynchronously if an error occurs
// or server returns response with an error status.
console.log(">>> GetSpeech Error:",response);
message = "Error: " + response;
alert(message);
return false;
})
}, function errorCallback(response) {
console.log(">>> GetToken Error:",response);
});
};
return {
getSpeech : getSpeech_innner
};
}])
NOTE: By the way in the postman the HTTP calls working. The GET Token and the POST synthesize.

I've tried the same on a similar way:
$http({
method: 'POST',
url: tts_url,
headers: {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': 'access-control-allow-headers, Authorization',
'content-type': 'application/json',
'Authorization': 'Basic <base64(uid:password)>',
'Accept': 'audio/wav'
},
data: {'\"text\"': '\"hello world\"' },
output: 'hello_world.wav'
}).then(function successCallback(response) {
console.log(">>> Success:",response.status);
}, function errorCallback(response) {
console.log(">>> Error:", response.status);
});
I'll get this error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading
the remote resource at
https://stream.watsonplatform.net/text-to-speech/api/v1/synthesize.
(Reason: missing token 'access-control-allow-headers' in CORS header
'Access-Control-Allow-Headers' from CORS preflight channel).
The behavior is the same, when I remove the "access-control-allow-headers" entry in the "Access-Control-Allow-Headers" header...
Running the same in postman works fine.
How do I allow my cordova app to call a remote resource?

Have you whitelisted the stream.watsonplatform.net origin in your Cordova application? It looks to me like the domain is being blocked. Details on whitelisting here: https://cordova.apache.org/docs/en/latest/guide/appdev/whitelist/

I my situation I wanted to get a token directly from the service broker.
At the current situation it seems for the communication with the text2speech service , I need a extra server app, for example running on Node.JS Server, which provides the token for the cordova app client.
Sample code for the server token app is available in the documentation.
This is documented in the chapter programming models in topic using token.
Watson Documentation Link
This is related to the developing models for the watson services.
The reason is that Postman and CURL calls are allowed and other request by Web or Mobile Applications are not valid Origins.
So you can not use the REST API directly, from the Mobile Application or Web Application. The same situation, is by using the Watson IoT.
This is the reason why I used a node framework in a Angular Web App by using Browserify Sample GitHub Project: browserfied-ibmiotf-webapplication
For a better understanding one picture from the Watson API documentation.
This is documented in the chapter programming models in topic using token.
Watson Documentation Link
#Thanks to Rene and Andrew pointing me in this direction.
Now I have a working cordova app with Text2Speech.
I do not use the token in this sample, the speech is provided directly from the Node.JS server to the Mobile App.
#Thanks to Andrii, providing some code to do this.
Node.JS Server:
app.get('/getText2SpeechOutput', function (req, res) {
console.log(' -> text function called');
console.log(' calling watson to synthesize -> ', req.header('synthesize-text'));
var text_to_speech_l = new Text2speech({
username: req.header('service-username'),
password: req.header('service-password'),
});
var params = {
text: req.header('synthesize-text'),
voice: 'en-US_AllisonVoice',
accept: 'audio/wav'
};
var tempaudio = text_to_speech_l.synthesize(params);
console.log(' response received from Watson');
var reader = new wav.Reader();
reader.on('format', function (format) {
console.log(' file read success');
var writer = new wav.FileWriter('output.wav', {
channels: 1,
sampleRate: 22050,
bitDepth: 16
});
reader.pipe(writer);
console.log(' file write success');
writer.pipe(res)
console.log(' <- response provided');
});
tempaudio.pipe(reader);
})
Cordova App:
var getSpeech_innner = function (thetext, callback){
//const fileTransfer = new FileTransfer();
var headers = {
"Authorization": authdata,
"Accept": "audio/wav",
"synthesize-text": thetext,
"service-username": username,
"service-password": password
};
var options = {
headers: headers,
replace: true
};
$cordovaFileTransfer.download(get_speech_url , cordova.file.dataDirectory + 'speech.wav', options, true).then(
function (FileEntry) {
console.debug('>>> Success', FileEntry);
var filePath = cordova.file.dataDirectory + 'speech.wav';
callback(filePath);
},
function (error) {
console.debug('>>> download failure', error);
callback(error);
});
};

Node.js equivalent of this curl request

I'm trying to use the HTML validator API. The curl examples work fine for me and I can run them find in Node as a child process. Here is the code for that:
var command = ('curl -H "Content-Type:text/html; charset=utf-8" --data-binary #' + file +
' https://validator.w3.org/nu/?out=json');
exec(command, function(err1, out, err2) {
console.log(out);
console.log('done');
});
However, when I tried to use a standard HTTP request, I couldn't get it working. I tried it with the Unirest library for Node. Here is the code I used:
var source = '<html><head><title>a</title></head><body>a</body></html>';
var url = 'http://validator.w3.org/nu/?out=json';
var Request = unirest.post(url);
Request.headers({'Content-Type': 'text/html', 'charset': 'utf-8'});
Request.send(source);
Request.end(res => console.log(res));
The response body is undefined and the response raw_body is empty. I don't know what I'm doing wrong and would appreciate any help.

Seems validator.w3.org won't respond to requests without a user-agent header. Add this header:
Request.headers({'Content-Type': 'text/html; charset=utf-8', 'user-agent': 'Node.js'});
Or use whatever useragent you want.

With super agent:
const request = require('superagent');
const body = '<html><head><title>a</title></head><body>a</body></html>';
request.post('https://validator.w3.org/nu/?out=json')
.set('Content-Type', 'text/html; charset=utf-8')
.send(body)
.end((err, res) => {
console.log('got body: ' + JSON.stringify(res.body));
});

How to correctly format outbound GET requests that contain data in Node.js

I am working on the backend of my iOS game and I want to validate Facebook logins on my server before I send any data back to the client. I have a valid app_access token provided by Facebook and I am able to successfully enter the following link in my browser to debug/validate access tokens:
https://graph.facebook.com/debug_token?input_token=users_access_token&access_token=apps_access_token
Ofcourse the correct access tokens are placed in the placeholders. From that request I receive a response that looks something like this:
{
"data": {
"app_id": app_id,
"is_valid": true,
"application": "My App",
"user_id": user_id,
"expires_at": 1382468400,
"scopes": [
"email",
"publish_actions",
"user_birthday",
"user_location"
]
}
}
From which I am able to determine if the user has a correct access token. However, trying to implement this from the server side has been fruitless. Here is how I am trying now
var http = require('http');
var https = require('https');
var querystring = require('querystring');
var data = querystring.stringify({
'access_token': app_access_token,
'input_token': user_access_token
});
console.log('' + data); //Debug to see if data is correctly formatted
var options = {
host: 'graph.facebook.com',
port: 443,
path: '/debug_token',
method: 'GET',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': Buffer.byteLength(data)
}
};
var req = https.request(options, function(res) {
res.setEncoding('utf8');
res.on('data', function (chunk) {
console.log("body: " + chunk);
});
});
req.write(data);
req.end();
Which always returns me the following response:
body: {"error":{"message":"(#100) You must provide an app access token or a user access token that is an owner or developer of the app","type":"OAuthException","code":100}}
I have noticed that the querystring does tend to format the "|" into "%7C" but I have manually replaced the characters and the data string to no avail. Is there something I am doing wrong? The tokens are correct, I just cant seem to format the data correctly.
Edit
Finally got it working. Michaels solution is right. I tried the same solution earlier and it didnt work because I performed a http.request() instead of an https.request() and I forgot to try it again. Such a silly mistake. Thanks!

With GET parameters are passed as a query string, part of path /path?param1=val1&param2=val2. In your case
path: '/debug_token?'+data,
instead of req.write(data).

Passing CSRF token from Node.js to Django

I'm working on a simple multiplayer word game app in Django (1.5). Following the example here, I'm using a separate Node.js server and Socket.io to manage the client connections.
My question breaks down into two parts:
The tutorial above uses #csrf_exempt for the API view. As the POST does not come from the client, but from the Node.js server over localhost, what exactly am I exposed to by not using CSRF protection for this view?
As I'm not sure of the above, I would like to use the CSRF protection. I have tried to extract the CSRF token from the cookie supplied by Django (as suggested by the docs) and send it along with the POST, but I still get a 403 response.
game_server.js:
io.configure(function () {
io.set('authorization', function (data, accept) {
if (data.headers.cookie) {
data.cookie = cookie_reader.parse(data.headers.cookie);
return accept(null, true);
}
return accept('error', false);
});
io.set('log level', 1);
});
io.sockets.on('connection', function (socket) {
socket.on('check_word', function (data) {
values = querystring.stringify({
word: data,
sessionid: socket.handshake.cookie['sessionid']
});
var options = {
host: 'localhost',
port: 8000,
path: '/node/check_word',
method: 'POST',
headers: {
'X-CSRFToken': socket.handshake.cookie['csrftoken'],
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': values.length
}
};
var req = http.request(options, function (res) {
res.setEncoding('utf8');
res.on('data', function (message) {
if (message) {
console.log(message);
}
});
});
req.write(values);
req.end();
});
});
game.html (script portion only):
(function ($) {
var socket = io.connect('localhost', { port: 4000 });
socket.on('connect', function () {
console.log("connected");
});
word_el = $('#word-input');
word_el.keypress(function (event) {
if (event.keyCode === 13) {
// Enter key pressed
var word = word_el.attr('value');
if (word) {
socket.emit('check_word', word, function (data) {
console.log(data);
});
}
word_el.attr('value', '');
}
});
})(jQuery);
views.py:
#ensure_csrf_cookie
def check_word(request):
return HttpResponse("MATCH:" + request.POST.get('word'))
Any insight would be greatly appreciated!

After a fair bit of research and experimentation, I've solved the problem.
My findings:
In this particular case, CSRF doesn't expose me to any meaningful attacks. In theory it opens a route for cheating in the game, but that's a lot of difficulty (requiring fabrication of session id and targeting a game currently in progress) for zero reward. However, in other applications such as chat, a CSRF vulnerability here allows for someone to impersonate another user, which is a more important concern. And so we dig deeper...
My original attempt to solve the problem via AJAX headers was a mistake. For one, the request is not actually coming over AJAX. (request.is_ajax() returns False within the view.) Secondly, the error page received from Django cites CSRF cookie not set as the reason for failure.
All of which builds up to the solution:
var options = {
// snip...
headers: {
'Cookie': 'csrftoken=' + socket.handshake.cookie['csrftoken'],
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': values.length
}
};
Add the proper 'Cookie' header, and the request succeeds.