node.js + bulk json as response - javascript

I developed an REST API using node.js + express + elastic-search. In that i receive a request and send JSON as response. Everything is working fine.
In a particular situation(say /xxx/yyy/zzz?param1) the response is not sent to the front end. My application simply stays ideal doing nothing. My guess is that for this particular route the JSON response is very bulk i think so.
My code is:
app.get('/xxx/yyy/zzz', function(req, res){
return Ctr.getMaster(req, res);
});
DAO.prototype.master = function(callback) {
var query = {
from: 0,
size: 1000000,
index: 'masterdata',
query:"match_all"
}
client.search(query).then(function (resp) {
var obj = {};
obj.count = resp.hits.total;
obj.master = resp.hits.hits;
callback(null, obj);
}, function(error){
console.log(error);
});
};
How can i sort it out. I want to know whether the JSON is the problem? Or something else?/ How can i crack it. Please share your ideas.

Press F12 on browser and verify that is request received or not and how much size is there. Moreover, print console log on your backend API for json size.
Take look this one too 8937516

Related

How can i send data through an API to other servers only using Nodejs and Express?

I'm new to NodeJS and I am currently working on getting an API working. Currently running is Express for that purpose and i really would like to stick to express to solve it.
My goal is to let other people send me their data through links (Example would be: http://localhost:1000/api/?product=test) so i can just grab them with a simple 'var productname = req.param('product'); That part works just fine.
But i would like to simply call a method to send data from my server, meaning i would like to trigger sending the data with a function and then send the data as a link to another server. (Example would be to http://google.com/search?q=test)
I can't seem to get it to work even if i directly work with the documentation from express: https://nodejs.org/api/http.html#http_http_get_url_options_callback
Could anyone point me in the right direction?
If i try the code snippet below, I'm not even getting a console.log.
My current code attempt is:
// testing purpose to call the method and get a console log
sendServerUpdates('chair');
function sendServerUpdates(product){
url = 'google.com/';
app.get(url + 'search', (res) => {
const {statusCode} = res;
const contentType = res.headers['content-type'];
let error;
if (statusCode !== 200) {
error = new Error('Request Failed.\n' + `Status Code:
${statusCode}`);
} else if (!/^application\/json/.test(contentType)) {
error = new Error('Invalid content-type.\n' + `Expected
application/json but received ${contentType}`);
}
if (error) {
console.error(error.message);
// Consume response data to free up memory
res.resume();
return;
}
// Information for me that the system is sending a message
console.log('sending update');
// sending (if its working) the parameter product
res.status(200).send(product);
})
}
}

Express Node JS POST. How do I add values to my req.body without using url parameters

I'm building a rest api using Express NodeJS with mysql. I'm having no issues at all using GET and using req.params.value and passing values using URL parameters.
Now I'm trying to use POST to insert some data into my database. Using POSTMAN I have no problems doing this because obviously you can set the BODY variables to be used. But it dawned on me in my applications I won't be able to use POSTMAN to do this. My quesiton (which may be silly) is how do I pass these BODY variables to my api? Do I still pass them through the url as parameters? If so, would I use req.body.value or req.params.value? Here is my POST code:
// Add new record
router.post('/editablerecords/add', function (req, res) {
let qb_TxnID = req.body.txnid
let type = req.body.type;
let margin = req.body.margin;
if (!qb_TxnID || !type || !margin ) {
return res.status(400).send({ error:true, message: 'Please provide qb_TxnID, type, or margin' });
}
// res.send(qb_TxnID + ' ' + type + ' ' + margin);
connection.query("INSERT INTO pxeQuoteToClose SET ? ", { 'qb_TxnID': qb_TxnID, 'type': type, 'margin': margin }, function (error, results, fields) {
if(error){
res.send(JSON.stringify({"status": 500, "error": error, "response": null}));
//If there is error, we send the error in the error section with 500 status
} else {
res.send(JSON.stringify({ error: false, data: results, message: 'New record has been created successfully.' }));
//If there is no error, all is good and response is 200OK.
}
});
});
Well it depends on how you will make your request, as an XMLHttpRequest call or via a form, but it will definitely not be using params in that case. Here are three options to send data to your API depending on your requirement, in your case I would advice the last one.
1 - You can use a form and make the action point to to your endpoint. In that case you'll have to add a bodyParser middleware for x-www-form-urlencoded data to your express application. If your using the body-parser library it is as simple as app.use(bodyParser.urlencoded([options])). Data will be available in req.body.
2 - You can send all data in your url but in the query string, not the params. For example: https://yourapi.com/editablerecords/add?qb_TxnID=data_id&type=data_type&margin=data_margin. All data will then be available in the req.query object. No need to add any parser.
3 - Last but not least, I would advise to send your data as a json body using an XMLHttpRequest. To help you out you may use a library like axios or fecth but the principle stay the same: you set your body with your data object and you retrieve it on req.bodyon your api. Seeing your code I do assume that your are using a body parser but if that would not be the case, you should add a middleware using app.use(bodyParser.json()).
I hope I have answer your question and that it will help you out.

How do I parse JSON from a URL for an AWS Lambda

I'm writing an AWS Lambda in node.js 6.10 for a school project with Amazon's Alexa software, and I don't have much experience with Javascript and none with JSON. My school has a transportation API for finding if it is up at: https://prtstatus.wvu.edu/api/[TENDIGITTIMESTAMP]/?format=json
If I go there with the stamp, I get "{"status":"7","message":"The PRT is closed.","timestamp":"1494028926","stations":[],"bussesDispatched":"0","duration":[]}"
What I am trying to get is the message and relay it to something else (I've got that part covered). What I don't know is how to break up the JSON response from the URL or write a request in the first place. Can someone help me figure out what to write to use the "message" string in my project?
So far I have:
'getPRTStatus': function() {
var date = Math.round(new Date().getTime()/1000);
//this is the spot where I need help filling in
//var object = JSON.parse('http://prtstatus.wvu.edu/api/'+date+'/?format=json');
this.attributes.speechOutput = this.t(object.message);
this.attributes.repromptSpeech = this.t(object.message);
this.emit(':ask', this.attributes.speechOutput, this.attributes.repromptSpeech);
},
Thanks for your help!
Is it possible for you to post the JSON response here from the URL because that would help a lot to narrow down the issue.
Update
You need to make an http get request to the API endpoint. You won't get a JSON response with,
var url = "http://prtstatus.wvu.edu/api/"+date+"/?format=json"
You can use a package like https://www.npmjs.com/package/request Check out their documentation on how you can make it work.
Something like this,
var options = {
"method": "get",
"url": "http://prtstatus.wvu.edu/api/1501906657/?format=json",
}
request(options, function(err, response, body) {
if (err) {
console.log(err)
} else {
console.log(body);
}
Another Update
You can try something like,
var request = require('request'); //Import the NPM package
var object; //global variable to be used later on to store the response
Then in your function,
'getPRTStatus': function() {
var date = Math.round(new Date().getTime()/1000);
var options = {
'method' : 'get',
'url' : 'http://prtstatus.wvu.edu/api/' + date + '/?format=json'
};
request(options, function(err, response, body){
if(err) {
console.log(err);
}
else {
object = JSON.parse(body); //You got the response parsed & stored in the global variable named object
}
});
this.attributes.speechOutput = this.t(object.message);
this.attributes.repromptSpeech = this.t(object.message);
this.emit(':ask', this.attributes.speechOutput,
this.attributes.repromptSpeech);
}
Just updated my answer according to your question. Hope that helps. For any future API related issues, you should try Postman in chrome. I'll post a link on how to get started with that. You will also get the direct code of your API call in postman.
Link to postman app: https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdddomop?utm_source=gmail

How to upload a file in to object storage using node js call

Iam trying to create a post call which basically takes a file(eg img,pdf file) and then it need to upload in to object storage on bluemix.I was able to authenticate and get the token and create the authurl.I just need to pass file which we upload along with the url.But Iam out of ideas how I can get the file uploaded from postman to be passed to that url with in the post call..Below is my code
app.post('/uploadfile',function(req,res){
getAuthToken().then(function(token){
if(!token){
console.log("error");
}
else{
var fileName = req.body.file;
console.log("data",file);
console.log(SOFTLAYER_ID_V3_AUTH_URL,"url");
var apiUrl = SOFTLAYER_ID_V3_AUTH_URL + config.projectId + '/' + containerName + fileName ;
url : apiurl,
method :'PUT',
headers :{
'X-Auth-Token': token
},function(error, response, body) {
if(!error && response.statusCode == 201) {
res.send(response.headers);
} else {
console.log(error, body);
res.send(body);
}
}
}
})
});
Can someone help here.
Since you're using Express, you should use something like:
https://www.npmjs.com/package/express-fileupload
https://github.com/mscdex/connect-busboy
https://github.com/expressjs/multer
https://github.com/andrewrk/connect-multiparty
https://github.com/mscdex/reformed
Without a body parser that handles file uploads you will not be able to get the uploaded file in the Express request handler.
Then, you need to pass the uploaded file to the request that you're making.
For that you should use this module:
https://www.npmjs.com/package/bluemix-object-storage
There is no need to reinvent the wheel when there are tested and eay to use solutions available. Especially when you're dealing with sensitive information like API keys and secrets I would not advice you to implement your own solution from scratch, unless you really know what you're doing. And if you really know what you're doing, then you don't need to seek advice for things like that.
Here is the official Object Storage SDK for Node.js:
https://github.com/ibm-bluemix-mobile-services/bluemix-objectstorage-serversdk-nodejs
Connect to Object Storage:
var credentials = {
projectId: 'project-id',
userId: 'user-id',
password: 'password',
region: ObjectStorage.Region.DALLAS
};
var objStorage = new ObjectStorage(credentials);
Create a container:
objstorage.createContainer('container-name')
.then(function(container) {
// container - the ObjectStorageContainer that was created
})
.catch(function(err) {
// AuthTokenError if there was a problem refreshing authentication token
// ServerError if any unexpected status codes were returned from the request
});
}
Create a new object or update an existing one:
container.createObject('object-name', data)
.then(function(object) {
// object - the ObjectStorageObject that was created
})
.catch(function(err) {
// TimeoutError if the request timed out
// AuthTokenError if there was a problem refreshing authentication token
// ServerError if any unexpected status codes were returned from the request
});

Node JS app only display the first JSON object. Why?

I am trying to write a json object in my node application, integrating the Twilio API. When console logging the object all objects are returned properly but when I write it to the document only the first object is written. Why? How should I change the code to see the same written response as in my console log.
var express = require('express');
var app = express();
app.use(express.bodyParser());
app.get('/', function(req, res) {
var accountSid = 'xxx';
var authToken = 'xxx';
var client = require('twilio')(accountSid, authToken);
client.messages.list({
from: "xxx",
to: "xxx"
}, function(err, data) {
data.messages.forEach(function(message) {
console.log(message.body); // THIS WILL DISPLAY ALL OBJECTS
res.json(message.body); // THIS WILL ONLY DISPLAY THE FIRST OBJECT
});
});
});
app.listen(1337);
I am new to Node JS and think this is easy to solve, but I still can’t find the solution.
res.json(...); sends back the response. You are doing that in the first iteration over the array, hence the client only gets the first message.
If you want to extract body from all messages and send all of them back, then do that. Create an array with the data you want and send it back. Example:
res.json(data.messages.map(function(message) {
return message.body;
}));
You can only call res.json once per request. You're calling it multiple times in a loop. The first time you call it, the browser receives the response, and you'll get a headers already sent exceptions (or something like that) for all other res.json calls.
res.json actually does a data conversion to JSON. I'd be willing to bet there is something it is not dealing with, or it's simply screwing it up. If the response from Twilio is already json, you probably don't need to do that. Try res.send, instead, which just returns whatever you got back.

Categories