Given a full HTTP message string, is there facilities in Node.js to parse this? Basically I'm not going to be using the http.createServer function. I already have the full HTTP message through other means (ZMQ), but I need to parse it, which means headers, query strings, post body.. etc.
Furthermore, the same functionality should be able to reverse the process, that is creating a fully formed HTTP response message string. That I can use and pass through different transport (ZMQ).
I know that PHP has Symfony's HTTP Foundation (which does what I'm asking). Is there something in Node.js that does a similar thing?
I found this: https://github.com/joyent/http-parser but it's in C++
You can use the http-parser from within node, but it's not very well documented. I extracted the following example from the tests:
var HTTPParser = process.binding('http_parser').HTTPParser,
CRLF = '\r\n',
request = new Buffer('POST /it HTTP/1.1' + CRLF +
'Content-Type: application/x-www-form-urlencoded' + CRLF +
'Content-Length: 15' + CRLF +
CRLF +
'foo=42&bar=1337'),
parser = new HTTPParser(HTTPParser.REQUEST),
headers = {},
body = '';
parser.onHeadersComplete = function(info) {
headers = info;
};
parser.onBody = function(b, start, len) {
body = b.slice(start, start + len).toString();
};
parser.onMessageComplete = function() {
console.log('message complete');
console.log('request method: ' + headers.method);
console.log('request body:\n\n ' + body);
};
parser.execute(request, 0, request.length);
Gives the following output:
message complete
request method: POST
request body:
foo=42&bar=1337
Related
As an example, I need the batch request in the following scenario:
After using Gmail.Users.Threads.list(..) I would like to do several Gmail.Users.Threads.get(threadId,..) operations in a batch.
I am talking about something similar to gapi.client.newBatch(); call in javascript gmail api.
First in apps script, one needs to enable the Gmail v1 Api in Advanced Google Services as described here.
Then using the Gmail Api in google apps script looks like this:
The suggestions are:
Users : UsersCollection
newAutoForwarding() : AutoForwarding
newBatchDeleteMessagesRequest() : BatchDeleteMessagesRequest
newBatchModifyMessagesRequest() : BatchModifyMessagesRequest
newDraft() : Draft
newFilter() : Filter
newFilterAction() : FilterAction
newFilterCriteria() : FilterCriteria
newForwardingAddress() : ForwardingAddress
newImapSettings() : ImapSettings
newLabel() : Label
newLabelColor() : LabelColor
newMessage() : Message
newMessagePart() : MessagePart
newMessagePartBody() : MessagePartBody
newMessagePartHeader() : MessagePartHeader
newModifyMessageRequest() : ModifyMessageRequest
newModifyThreadRequest() : ModifyThreadRequest
newPopSettings() : PopSettings
newSendAs() : SendAs
newSmimeInfo() : SmimeInfo
newSmtpMsa() : SmtpMsa
newVacationSettings() : VacationSettings
newWatchRequest() : WatchRequest
There is no newBatch() suggested.
How about this answer? I couldn't find the method of batch request for Gmail.Users.Threads.get(). And at Google Apps Script, there are no methods for requesting the batch request. So it is required to implement the method. The flow of batch request is as follows.
Create the request body for the batch request.
Requst the body to the endpoint of POST https://www.googleapis.com/batch using multipart/mixed.
The access token is required to be used for only this post.
The sample script for this flow is as follows.
Sample script :
Flow :
Retrieve thread list using Gmail.Users.Threads.list().
Create the request body for Gmail.Users.Threads.get().
In this case, Gmail.Users.Threads.get() of Advanced Google Services cannot be used, so it is required to directly use the API.
Post the created body using multipart/mixed.
Parse the response.
Script :
function myFunction() {
var userId = "me"; // Please modify this, if you want to use other userId.
var threadList = Gmail.Users.Threads.list(userId).threads;
var body = threadList.map(function(e){
return {
method: "GET",
endpoint: "https://www.googleapis.com/gmail/v1/users/" + userId + "/threads/" + e.id
}
});
var url = "https://www.googleapis.com/batch";
var boundary = "xxxxxxxxxx";
var contentId = 0;
var data = "--" + boundary + "\r\n";
for (var i in body) {
data += "Content-Type: application/http\r\n";
data += "Content-ID: " + ++contentId + "\r\n\r\n";
data += body[i].method + " " + body[i].endpoint + "\r\n\r\n";
data += "--" + boundary + "\r\n";
}
var payload = Utilities.newBlob(data).getBytes();
var options = {
method: "post",
contentType: "multipart/mixed; boundary=" + boundary,
payload: payload,
headers: {'Authorization': 'Bearer ' + ScriptApp.getOAuthToken()},
muteHttpExceptions: true,
};
var res = UrlFetchApp.fetch(url, options).getContentText();
var dat = res.split("--batch");
var result = dat.slice(1, dat.length - 1).map(function(e){return e.match(/{[\S\s]+}/g)[0]});
Logger.log(result.length)
Logger.log(result)
}
Note :
The parsed response is an array. Each element in the array is corresponding to each element in the request body.
In this sample script, the thread list is retrieved by Gmail.Users.Threads.list("me").threads. If you want to use some threads, please modify the request body.
Reference :
Batching Requests
If I misunderstand your question, I'm sorry.
I am trying to make an API request to fitbit, using the oauth debugger from fitbit (https://dev.fitbit.com/apps/oauthtutorialpage) i am trying to figure out what i am doing wrong. I have added comments to my code below to help you understand what i am trying to achieve. What i am quite sure of is that i am either signing my request wrong, or using the wrong data to sign it. This is echoed by the API response.
I know there are more fitbit api questions here on stackoverflow, however did not find my answer there.
Is there anyone with more experience in Oauth signatures that knows what i could be doing wrong? Or could help me find a different approach to this?
var request = require('request');
var crypto = require('crypto');
var params = {
'oauth_consumer_key' : 'key12345',
'oauth_nonce' : Math.random().toString(36).substring(3), //random string
'oauth_signature_method' : 'HMAC-SHA1',
'oauth_timestamp' : Date.now().toString().substring(0,10), //timestamp with the same length as in the tutorial
'oauth_version' : '1.0'
}
var oauth_consumer_secret = 'secret123';
var post_string = 'POST&https://api.fitbit.com/oauth/request_token';
for(var key in params){
post_string += '&' + key + '=' + params[key];
}
/*At this point we have made a post string that we have to hash with hmac-sha1
the post string looks like this:
POST&https://api.fitbit.com/oauth/request_token&oauth_consumer_key=key12345&oauth_nonce=az6r8cqlzyqfr&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1439147378&oauth_version=1.0
The post_string from the tutorial looks like this:
POST&%2Foauth%2Frequest_token&oauth_consumer_key%3D%26oauth_nonce%3D%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1439145944%26oauth_version%3D1.0
*/
var hmac = crypto.createHmac('sha1', oauth_consumer_secret + "&");
// The tutorial page shows me the signature was 'signed with secret&'. I have tried with and without the & at the end, but without luck.
hmac.setEncoding('base64'); //i'm not sure if this is correct
hmac.write(post_string);
hmac.end();
var hash = hmac.read();
//and finally adding the hash to the parameters.
params.oauth_signature = hash;
//now, making the request with an authorization header.
var header='';
for (var key in params){
if(header.length === 0){
header = ' OAuth ' + key + '="' + params[key] + '"';
}
else{
header += ', ' + key + '="' + params[key] + '"';
}
}
/*
At this point the header parameter looks like this
OAuth oauth_consumer_key="key12345", oauth_nonce="jnr97ppvjs2lnmi", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1439148049", oauth_version="1.0", oauth_signature="random_signature"
The tutorial tells me to use the headers:
OAuth oauth_consumer_key="key12345", oauth_nonce="jnr97ppvjs2lnmi", oauth_signature="different_signature", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1439145944", oauth_version="1.0"
*/
var headers ={
'Authorization' : header
}
var url="https://api.fitbit.com/oauth/request_token";
var requestTimeout = 5000;
var opts = {
url: url,
timeout: requestTimeout,
headers : headers
}
request(opts, function (err, res, body) {
if (err) {
console.dir(err);
return;
}
var statusCode = res.statusCode;
if(res.statusCode === 200){
console.log(body);
}
else{
console.log("http-error-code: " + res.statusCode);
console.log(body);
}
})
/*
The response:
http-error-code: 401
{"errors":[{"errorType":"oauth","fieldName":"oauth_signature","message":"Invalid signature: 9fXI85C7GvZqMyW1AK1EkOSWZCY="}],"success":false}
*/
To get access token and secret use Grant (you can test FitBit in the playground).
Once you have access token and secret use Purest to make subsequent request to the FitBit API.
Here is an example on how to get the user's profile:
var Purest = require('purest')
var fitbit = new Purest({provider:'fitbit',
key:'[CONSUMER_KEY]', secret:'[CONSUMER_SECRET]'})
fitbit.get('user/-/profile', {
oauth:{token:'[ACCESS_TOKEN]', secret:'[ACCESS_SECRET]'}
}, function (err, res, body) {})
Alternatively you can use request for that:
var request = require('request')
request.get('https://api.fitbit.com/1/user/-/profile.json', {
oauth:{
consumer_key:'..',
consumer_secret:'..',
token:'..',
token_secret:'..'
}
}, function (err, res, body) {})
In short - don't try to implement the web server OAuth flow by yourself - use Grant, then use either Purest or request, just keep in mind that you don't have to pass all of the OAuth parameters by yourself, just pass the credentials.
In NetSuite, I'm trying to create a form Portlet POST data to a RESTlet in JSON. I've checked the documentation and internet and all the examples I've been able to find are GET requests or they post to a backend Suitelet instead.
I've come to a point where I can make the request reach the RESTlet but it's not being formatted in JSON, so I get the following error:
Account: xxxxxxxxxxxxx
Environment: Production Date & Time: 6/11/2015 5:09 pm
Execution Time: 0.06s
Script Usage: 0
Script: gw_SS_FormBackend
Type: RESTlet
Function: postWMForm
Error: UNEXPECTED_ERROR
SyntaxError: Empty JSON string (null$lib#3)
I'm using the following code to set the submit button and it's working fine:
var headers = new Array();
headers['User-Agent-x'] = 'SuiteScript-Call';
headers['Authorization'] =
'NLAuth nlauth_account=' + cred.account +
', nlauth_email=' + cred.email +
', nlauth_signature=' + cred.password +
', nlauth_role=' + cred.role;
headers['Content-Type'] = 'application/json';
portlet.setSubmitButton(nlapiRequestURL(getRESTletURL(), null, headers, 'POST'), 'Submit', '_hidden');
My problem is I don't know how to convert the form data to JSON before submitting it.
I'd appreciate any help.
Why would you want to use a RESTlet? If you are in a portlet then you already have a valid NS session so you'd be better off using a Suitelet. A Suitelet you know is set up to handle JSON would be called thus:
nlapiRequestURL(suiteletURL', JSON.stringify{test:'it', when:new Date(), by:'Brett'}), {"content-type":'application/json'}, function(resp){console.log(resp.getBody());}, 'POST');
and your Suitelet code might include something like:
var body = request.getBody();
nlapiLogExecution('DEBUG', 'posted body', body);
var asJSON = JSON.parse(body);
var user = nlapiGetContext().getUser(); // system already knows who this is.
...
var obj = {
success:true,
stuff: asProcessed
};
response.setContentType('JAVASCRIPT');
response.writeLine( JSON.stringify(obj) );
Not quite as clean as a RESTlet but you avoid having to hack the credentials.
you can use JSON.stringify() function.
var headers = new Array();
headers['User-Agent-x'] = 'SuiteScript-Call';
headers['Authorization'] =
'NLAuth nlauth_account=' + cred.account +
', nlauth_email=' + cred.email +
', nlauth_signature=' + cred.password +
', nlauth_role=' + cred.role;
headers['Content-Type'] = 'application/json';
var myJsonHeader = JSON.stringify(headers);
portlet.setSubmitButton(nlapiRequestURL(getRESTletURL(), null, myJsonHeader, 'POST'), 'Submit', '_hidden');
Regards
I am trying to write some code that submits a ticket automatically with information from a page I created in Apps Script. I have tried numerous examples, but I can't seem to get my code to work.
var headers = {
'Content-type': 'application/json',
'Authorization': 'Basic ' + Utilities.base64Encode(API_KEY + ':X')
};
//Puts together the ticket according to the freshdesk api.
//var payload = '{"helpdesk_ticket":{"description":"' + message + '","subject":"' + subject + '","email":"' + arr[0][0] + '","priority":"' + ticketPriority + '","status":2}}';
//var payload = '{"helpdesk_ticket":{"description": message ,"subject": subject,"email": arr[0][0],"priority": ticketPriority,"status":2}}';
var payload = '{"helpdesk_ticket":{"description":"TEST","subject":"TEST","email":"test#test.com","priority":1,"status":2}}';
//Adds the extensions that are needed to post a new ticket to the end of the url
var url = ENDPOINT + '/helpdesk/tickets.json';
var options = {
'method': 'post',
'headers': headers,
'payload': payload,
muteHttpExceptions: true
};
var response = UrlFetchApp.fetch(url, options);
This is what I currently have. I have gotten it to work once, but only when I do not have any variables being assigned to the 'description' or 'subject' header (the line with the payload variables that is uncommented. When I use that line, a ticket is successfully created). I am not sure why my first or second lines with the payload variables would not work. The variable 'message' is just a String with some new line characters '\n' in it. Does anyone know why this might be happening?
Solved by building the message variable with HTML code and using the 'description_html' property instead of 'description'.
I am trying to interface with an external API and I need to POST an XML document over HTTPS.
So I am using the node https interface to try to make the request but if I try to write any data (The XML document) it throws a socket hang up. If I write nothing or an empty string to the request it completes the post just fine.
I've googled and found other people with this error but I haven't been able to fix it following the solutions others have found.
I am using Meteor, which has the HTTP package for making these types of requests. The HTTP package was also throwing this error so I dug down and implemented the post using the node 'https' package thinking it would solve the issue but I get the same error with https as well.
Here is my code:
var http = Npm.require("http");
var oauthSignature = Npm.require('oauth-signature');
var URL = Npm.require('url');
var content = "XML String Here";
var postDestinationUrl = "https://example.com/api/path";
var authObject = {
oauth_consumer_key: "consumerKey",
oauth_signature_method: "HMAC-SHA1",
oauth_timestamp: (Math.round(new Date().getTime() / 1000)).toString(10),
oauth_nonce: Random.id(),
oauth_version: "1.0"
};
authObject.oauth_signature = oauthSignature.generate("POST", postDestinationUrl, authObject, "shared Secret Here");
var authString = objectToQueryString(authObject);
var headers = {
Connection: "keep-alive",
'Content-Length': Buffer.byteLength(content),
Authorization: authString,
'Content-Type': 'application/xml'
};
var parsedUrl = URL.parse(postDestinationUrl);
var requestOptions = {
hostname: parsedUrl.hostname,
path: parsedUrl.pathname,
method: "POST",
headers: headers
};
var request = https.request(requestOptions, function(response){
var body = '';
console.log("statusCode from https.request: ", response.statusCode);
console.log("headers from https.request: ", response.headers);
response.on("data", function(data){
body += data;
}); // end of data
response.on("end", function(){
console.log("body from https.request: ", body);
}); // end of end
response.on("error", function(){
console.log("error in https.request response");
}); // end of error
}); // end of request
request.write(content);
request.end();
request.on("error",function(error){
console.log("Error in https.request: " + error.message);
callback(error, undefined);
}); // end of error
var objectToQueryString = function(queryObject){
var queryString = "";
_.each(queryObject, function(value, key, list){
queryString += key + '="' + value +'",';
});
return queryString.substr(0, queryString.length -1);
};
And the error I am seeing:
"stack":"Error: socket hang up
at Object.Future.wait (/Users/dsyko/.meteor/tools/f3947a4651/lib/node_modules/fibers/future.js:326:15)
at packages/meteor/helpers.js:111
at Meteor.methods.submitAPIPost (packages/api-interface/api_server.js:412)
at packages/check/match.js:77
at _.extend.withValue (packages/meteor/dynamics_nodejs.js:35)
at Object.Match._failIfArgumentsAreNotAllChecked (packages/check/match.js:76)
at maybeAuditArgumentChecks (packages/livedata/livedata_server.js:1403)
at packages/livedata/livedata_server.js:580
at _.extend.withValue (packages/meteor/dynamics_nodejs.js:35)
at packages/livedata/livedata_server.js:579
- - - - -
at createHangUpError (http.js:1472:15)
at CleartextStream.socketCloseListener (http.js:1522:23)
at CleartextStream.EventEmitter.emit (events.js:117:20)
at tls.js:696:10
at process._tickCallback (node.js:415:13)"
I've used postman to make the same API request and it goes through just fine so it doesn't seem to be a bad endpoint. I also switched over to posting to http and used Wireshark to inspect the headers and content of the HTTP POSTs to make sure I'm not mangling something in the request but it all looked ok there. Also when I switched over to http the XML document goes through just fine and I don't see the socket hangup (Although the endpoint responds with a re-direct to the https url so I can't just use http)