Say I got:
app.get('/json', function(req, res) {
res.set({
'content-type': 'application/json'
}).send('{"status": "0"}');
});
I'm trying to send the response as UTF-8 with the following with no success:
app.get('/json', function(req, res) {
// From Node.js Official Doc
// http://nodejs.org/api/http.html#http_http_request_options_callback
res.setEncoding('utf8');
res.set({
'content-type': 'application/json'
}).send('{"status": "0"}');
});
What is the correct way to set character encoding in Express?
You will probably want to explicitly add a charset to the end of your content-type string if you find it's not being set already by Express:
res.set({ 'content-type': 'application/json; charset=utf-8' });
The charset is not always set automagically and does need to be set to work correctly everywhere (i.e. with all browsers and all ajax libraries) or you can run into encoding bugs.
In Express 4.x specifically I've found that depending on the object you trying to return, it normally automatically returns with content-type: application/json; charset=utf-8 when you call res.json(someObject), however not always.
When calling res.json() on some objects it can return content-type: application/json (i.e. without the charset encoding!). I'm not actually sure what triggers this, other than it's something about the specific object being returned.
I've only noticed it because of automated tests which explicitly checked the headers and found it was missing the charset declaration on some responses (even though the content-type was still application/json).
Use res.charset: http://expressjs.com/api.html#res.charset
res.charset = 'value';
res.send('some html');
// => Content-Type: text/html; charset=value
However, JSON is UTF-8 by default so you don't need to set anything.
This worked for me
res.writeHead(200, {'Content-Type': 'text/html; charset=utf-8'});
Having similar issues I'm collecting Swedish characters from a database and outputting them as JSON object, node doesn't really care if json must be UTF-8 or not when the chars from the database isn't in UTF-8.. So assuming "you don't need to set anything" is false. Depending on what charsets you are working with.
Before you go to the trouble of manually setting header parameters, check what your server is already sending by default. In my case, I'm using a "serverless" cloud provided Node.js instance. Apparently, these are usually front-ended w/ NGINX which I assume is what sets some of this stuff based on default settings. ...I didn't need to res.set anything at all. Granted, I'm serving back HTML, ...just sayin - before you go fixin, make sure it's broke.
accept-ranges: bytes
accept-ranges: bytes
cache-control: private
content-encoding: gzip
content-type: text/html; charset=utf-8
date: Fri, 21 Dec 2018 21:40:37 GMT
etag: W/"83-xwilN/BBLLLAAAHHH/0NBLAH0U"
function-execution-id: 5thvkjd4wwru
server: nginx
status: 200
vary: accept-encoding, cookie, authorization
via: 1.1 varnish
x-cache: MISS
x-cache-hits: 0
x-cloud-trace-context: 18c611BBBBLLLLAAAHHH9594d9;o=1
x-powered-by: Express
x-served-by: cache-dfw18631-DFW
x-timer: S15BBLLLAAHHH.913934,VS0,VE3404
Related
get headers with angular in second get.
In browser we write this:
http://localhost:4200
En console network apper the headers like this:
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Content-Length: `1428`
Content-Type: text/html; charset=UTF-8
Date: Thu, 05 Mar 2020 00:34:18 GMT
ETag: W/"54e-4HSDfsd4538gfdGFDgdf"
X-Powered-By: Express
With Angular I cant get those params, so, I do a second get with
httpClient.get('http://localhost:4200')
to set headers and then get headers params
But I dont know how put this params in header. So any one have idea?
I think that with a java can do that? maybe with a #Override. I dont know and then return headers with custom paramms
Some guide I will be grateuful
You need to build your headers and then pass it along to the get request:
Example:
const headers = new HttpHeaders({'Content-Type':'application/json; charset=utf-8'});
httpClient.get('<url>', { headers });
https://angular.io/guide/http#http-headers
I am working on node js web application and I am doing a http get request which address a db and takes data via query.
The http get request is working fine on chrome but on IE each get is not updated but returned from some sort of cache. That makes the result of the db query to be not updated (beacuse it is taken from chache).
I can see that it is being taken from chache form F12 developer tool in IE:
My code below. I know that I should add something like:
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
to my request but I think that I maybe put this line in the wrong place beacuse the get reuqest still taken from chache and gives me bad result...
client
$http.get('/users')
.success(function(data) {
$scope.usersNumber = data.length;
})
.error(function(data) {
console.log('Error: ' + data);
});
server
app.get('/users', function(req, res){
get_user(req, res);
});
var get_user= function(req, res){
var query= User_session.find();
query.exec( function(err, docs){
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
res.json(docs);
//mongoose.connection.close();
});
}
Try adding max-age to your header:
Cache-Control:private, no-cache, no-store, must-revalidate, max-age=0
Pragma:no-cache
The response that you captured from the developer tools shows that the response is picked from cache. The cached response had a expires header which specified the duration for which the response can be picked from cache and is not considered stale.
You would want to include a no-cache directive for End-to-end reload
The request includes a "no-cache" cache-control directive or, for
compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field names
MUST NOT be included with the no-cache directive in a request. The
server MUST NOT use a cached copy when responding to such a request.
Reference: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
You can also try this another solution to force client reloads. Add a query string parameter to your url.
Eg: If you have a /users, instead make the url as /users?uniqstamp=<timestamp>
Your browser always thinks that the entire path including the query string makes a URL. so you will always get a 200 ok that is served fresh. Make sure that timestamp that you use is from javascript time in millis so that the timestamp is always unique.
Trying to make a call and retrieve a very simple, one line, JSON file.
$(document).ready(function() {
jQuery.ajax({
type: 'GET',
url: 'http://wncrunners.com/admin/colors.json' ,
dataType: 'jsonp',
success: function(data) {
alert('success');
}
});
});//end document.ready
Here's the RAW Request:
GET http://wncrunners.com/admin/colors.json?callback=jQuery16406345664265099913_1319854793396&_=1319854793399 HTTP/1.1
Host: wncrunners.com
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2
Accept: */*
Referer: http://localhost:8888/jquery/Test.html
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Here's the RAW Response:
HTTP/1.1 200 OK
Date: Sat, 29 Oct 2011 02:21:24 GMT
Server: Apache/1.3.33 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7d SE/0.5.3
Last-Modified: Fri, 28 Oct 2011 17:48:47 GMT
ETag: "166a2402-10-4eaaeaff"
Accept-Ranges: bytes
Content-Length: 16
Content-Type: text/plain
Connection: close
{"red" : "#f00"}
The JSON is coming back in the response (red : #f00), but Chrome reports Uncaught SyntaxError: Unexpected token : colors.json:1
If I navigate directly to url itself, the JSON is returned and is displayed in the browser.
If I paste the contents of colors.json into JSLINT, the json validates.
Any ideas why I can't get this error and I never make it to the success callback?
EDIT - the jQuery.ajax() call above runs perfect at jsfiddle.net, and returns the alert 'success' as expected.
EDIT 2 - this URL works fine 'http://api.wunderground.com/api/8ac447ee36aa2505/geolookup/conditions/q/IA/Cedar_Rapids.json' I noticed that it returned as TYPE: text/javascript and Chrome did not throw the Unexpected Token. I've tested several other url's and the ONLY one that does not throw the Unexptected Token is the wunderground that is returned as TYPE: text/javascript.
Streams returned as text/plain and application/json are not being parsed correctly.
You've told jQuery to expect a JSONP response, which is why jQuery has added the callback=jQuery16406345664265099913_1319854793396&_=1319854793399 part to the URL (you can see this in your dump of the request).
What you're returning is JSON, not JSONP. Your response looks like
{"red" : "#f00"}
and jQuery is expecting something like this:
jQuery16406345664265099913_1319854793396({"red" : "#f00"})
If you actually need to use JSONP to get around the same origin policy, then the server serving colors.json needs to be able to actually return a JSONP response.
If the same origin policy isn't an issue for your application, then you just need to fix the dataType in your jQuery.ajax call to be json instead of jsonp.
I have spent the last few days trying to figure this out myself. Using the old json dataType gives you cross origin problems, while setting the dataType to jsonp makes the data "unreadable" as explained above. So there are apparently two ways out, the first hasn't worked for me but seems like a potential solution and that I might be doing something wrong. This is explained here [ https://learn.jquery.com/ajax/working-with-jsonp/ ].
The one that worked for me is as follows:
1- download the ajax cross origin plug in [ http://www.ajax-cross-origin.com/ ].
2- add a script link to it just below the normal jQuery link.
3- add the line "crossOrigin: true," to your ajax function.
Good to go! here is my working code for this:
$.ajax({
crossOrigin: true,
url : "https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=-33.86,151.195&radius=5000&type=ATM&keyword=ATM&key=MyKey",
type : "GET",
success:function(data){
console.log(data);
}
})
I had the same problem and the solution was to encapsulate the json inside this function
jsonp(
.... your json ...
)
That hex might need to be wrapped in quotes and made into a string. Javascript might not like the # character
Purpose:
I (a javascript beginner) am trying to perform a Rest API call to a web-server (Splunk) get data of content-type json. Call is initiated by angularjs based webApp, the problematic code snippet is posted below. The problem is investigated using firebug and the response & header information is mentioned below.
Angular Code Snippet:
$scope.onClickButton = function (button) {
$http({
method: 'POST',
url: 'https://localhost:8089/servicesNS/admin/search/search/jobs/export',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Splunk ' + $scope.sessionKey
},
data: {search: button.query, output_mode: 'json'}
}).success(function (response) {
alert("Success");
}).error(function (response) {
alert("error");// alerted error
alert(response); // alerts 'undefined'
});
};
Response Object (captured by firebug)
{"preview":false,"offset":0,"result":{"_serial":"0", "_sourcetype":"mylogger", "splunk_server":"mohit-PC"}}
{"preview":false,"offset":1,"result":{"_serial":"1", "_sourcetype":"mylogger", "splunk_server":"mohit-PC"}}
{"preview":false,"offset":2,"result":{"_serial":"2", "_sourcetype":"mylogger", "splunk_server":"mohit-PC"}}
...
Response Header
Access-Control-Allow-Head... Authorization
Access-Control-Allow-Meth... GET,POST,PUT,DELETE,HEAD,OPTIONS
Access-Control-Allow-Orig... *
Cache-Control no-store, no-cache, must-revalidate, max-age=0
Connection Keep-Alive
Content-Type application/json; charset=UTF-8
Date Tue, 02 Dec 2014 13:00:34 GMT
Expires Thu, 26 Oct 1978 00:00:00 GMT
Server Splunkd
Transfer-Encoding chunked
Vary Authorization
access-control-allow-cred... true
x-content-type-options nosniff
x-frame-options SAMEORIGIN
Firebug Error
Error: JSON.parse: unexpected non-whitespace character after JSON data at line 2 column 1 of the JSON data fromJson#angular.js:1054:9
.
Questions
Q1. The status of response is - 200 OK, then why the error is thrown? Though it is apparent that the response text does not represent a json object or an array of json objects. The [] and commas are absent. The server cannot be configured to respond with a proper formatted data. It seems angular does a format verification and throws exception. Can this verification be avoided/bypassed?
Q2. Even though the error occurred, why the response object is 'undefined' in http.error method? If it were not, I could do string manipulations as a work around.
I am parsing in about 4000 URLs with a generic Node.js HTTP request script:
(function (i){
http.get(options, function(res) {
var obj = {};
obj.url = hostNames[i];
obj.statusCode = res.statusCode;
obj.headers = res.headers;
db.scrape.save(obj);
}).on('error',function(e){
console.log("Error: " + hostNames[i] + "\n" + e.stack);
})
})(i);
Around 1300 URLs in, I get this error, which stops the entire script. I don't know what page.ly is, as I do not have that in my list of URLs. I've done a lot of research, but I could not pin-point what's causing this error.
If someone is familiar with HTTP requests on Node.js - could you help me out?
Error: key page.ly must not contain '.'
at Error (unknown source)
at Function.checkKey (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1421:11)
at serializeObject (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:355:14)
at packElement (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:854:23)
at serializeObject (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:359:15)
at Function.serializeWithBufferAndIndex (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:332:10)
at BSON.serializeWithBufferAndIndex (/Users/loop/node_modules/mongojs/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1502:15)
at InsertCommand.toBinary (/Users/loop/node_modules/mongojs/node_modules/mongodb/lib/mongodb/commands/insert_command.js:132:37)
at Connection.write (/Users/loop/node_modules/mongojs/node_modules/mongodb/lib/mongodb/connection/connection.js:198:35)
at __executeInsertCommand (/Users/loop/node_modules/mongojs/node_modules/mongodb/lib/mongodb/db.js:1745:14)
at Db._executeInsertCommand (/Users/loop/node_modules/mongojs/node_modules/mongodb/lib/mongodb/db.js:1801:5)
Loops-MacBook-Air:JS loop$
What could prevent this? It seems my script does not scale very well.
EDIT: From the answers I am getting - there exists a key somewhere that has a ".", which isn't allowed in MongoDB, and I am supposed to escape it. But the question remains - if my keys are only url, statusCode, and headers, what is causing the key with a . in it to show up?
EDIT: Bug is found. Answer below.
This error is caused when you attempt persist an Object in MongoDB and one (or more) of the keys contain the character '.', e.g:
{
"name": "bob",
"url": "http://example.com",
"some.field": "value"
}
would raise the error Error: key some.field must not contain '.'.
Scrub your object keys of '.'s before saving to MongoDB!
The site "divensurf.com" has a header which is called page.ly: v4.0
I have no idea what the is, but that broke my import into MongoDB, since keys cannot symbols. I found the culprit by printing the output onto a .txt file, did a search on the header page.ly, found the site, and deleted it.
I will be sanitizing the headers before importing.
Thanks for the help guys.
HTTP/1.1 304 Not Modified
X-Varnish: 2236761436 2236710300
Vary: Accept-Encoding,Cookie,X-UA-Device
Cache-Control: max-age=7200, must-revalidate
X-Cache: V1HIT 5
Content-Type: text/html; charset=UTF-8
Page.ly: v4.0
Content-Encoding: gzip
X-Pingback: http://divensurf.com/xmlrpc.php
Date: Thu, 21 Mar 2013 19:45:35 GMT
Accept-Ranges: bytes
Via: 1.1 varnish
Connection: keep-alive
Last-Modified: Thu, 21 Mar 2013 19:40:57 GMT
Age: 278