I am working on node js web application and I am doing a http get request which address a db and takes data via query.
The http get request is working fine on chrome but on IE each get is not updated but returned from some sort of cache. That makes the result of the db query to be not updated (beacuse it is taken from chache).
I can see that it is being taken from chache form F12 developer tool in IE:
My code below. I know that I should add something like:
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
to my request but I think that I maybe put this line in the wrong place beacuse the get reuqest still taken from chache and gives me bad result...
client
$http.get('/users')
.success(function(data) {
$scope.usersNumber = data.length;
})
.error(function(data) {
console.log('Error: ' + data);
});
server
app.get('/users', function(req, res){
get_user(req, res);
});
var get_user= function(req, res){
var query= User_session.find();
query.exec( function(err, docs){
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
res.json(docs);
//mongoose.connection.close();
});
}
Try adding max-age to your header:
Cache-Control:private, no-cache, no-store, must-revalidate, max-age=0
Pragma:no-cache
The response that you captured from the developer tools shows that the response is picked from cache. The cached response had a expires header which specified the duration for which the response can be picked from cache and is not considered stale.
You would want to include a no-cache directive for End-to-end reload
The request includes a "no-cache" cache-control directive or, for
compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field names
MUST NOT be included with the no-cache directive in a request. The
server MUST NOT use a cached copy when responding to such a request.
Reference: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
You can also try this another solution to force client reloads. Add a query string parameter to your url.
Eg: If you have a /users, instead make the url as /users?uniqstamp=<timestamp>
Your browser always thinks that the entire path including the query string makes a URL. so you will always get a 200 ok that is served fresh. Make sure that timestamp that you use is from javascript time in millis so that the timestamp is always unique.
Related
I use AJAX to check user updates every 2 seconds, but my javascript does not update the response.
I have one javascript file with XMLHttpRequest object and every 2 seconds it sends a request to another file (.php) where it gets XML with updates. For some reason, it doesn't always get the newest content and seems to have some old cached.
My javascript file contains this code (simplified):
var updates = new XMLHttpRequest();
updates.onreadystatechange = function(){
"use strict";
if(updates.readyState === 4 && updates.status === 200){
console.log(updates.responseXML);
}
};
var timer = 0;
clearInterval(timer);
timer = setInterval(function(){
"use strict";
updates.open('GET','scripts/check_for_notifications.php', true);
updates.send();
},2000);
Then I have the PHP file (check_for_notifications.php), where I have this code:
$response = new SimpleXMLElement('<xml/>');
$update = $response->addChild('update');
$update->addChild('content', 'New message');
$update->addChild('redirect', 'some link');
$update->addChild('date', '1.1.2019 12:00');
header('Content-type: text/xml');
print($response->asXML());
Every two second I receive a log in my console, but when I change the PHP file, while the interval is in progress (e.g. I change the date to '1.1.2019 11:00' and save it) I still receive the '12:00' in the console. For me, it seems that it doesn't update and it still has the repsonseXML cached. Is there any way I could "flush" the output or am I doing it wrong?
It's probably a cache problem. In the network browser console, you should see a response of type 304 Not modified.
To be sure, you can add an element in the url to bypass the cache:
updates.open('GET','scripts/check_for_notifications.php&nocache=' + new Date().getTime(), true);
If this works, you will need to configure the server (apache or nginx) to prevent the file from being cached. It's cleaner than the timestamp solution. the browser does not store cached files unnecessarily.
Apache .htaccess or Apache conf, something like
<Files "check.php">
<IfModule mod_headers.c>
Header set Cache-Control "no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires 0
</IfModule>
</Files>
Nginx conf, something like
location = /check.php {
add_header 'Cache-Control' 'no-cache, no-store, must-revalidate';
expires off;
}
You can also see the fetch api : https://hacks.mozilla.org/2016/03/referrer-and-cache-control-apis-for-fetch/
Be careful with your code, it can launch several requests simultaneously and find yourself with a DOS if there are too many users.
If requests take more than 2 seconds because the server is slow, others requests will be sent in the meantime, which will slow down the server even more....
I'm new to angular and having trouble with a service that always worked, until it didn't. My service has the following call.
this.getForms = function() {
return $http.get("/forms").
then(function(response) {
return response;
}, function(response) {
alert("Error finding forms.");
});
};
When the page is refreshed (safari) getForms is trigged, $http.get is called, my Node.js/Express server forms endpoint returns the forms data properly.
app.get("/forms", function (req, res) {
Form.find({}, function (err, docs) {
if (err) {
server.handleError(res, err.message, "Failed to get forms.");
} else {
res.status(200).json(docs);
}
});
});
But instead of the JSON I get a 304 error, which indicates the data is available in cache. But the 304 response header has an empty data string, so it's not returning any data from the cache.
My questions are
1) Why is it calling my server if the data is available in cache?
2) How can I tell it to not cache this call so the page can update with forms correctly?
Edit: Looks like there could be a potential issue with Safari. See this post as well. NodeJS/express: Cache and 304 status code
You might be misunderstanding what the 304 status code means. 304 is not an error; it just means that the resource you are requesting from the server has not changed. The response is meant to not contain data as it expects the client to have cached it somewhere.
For example, web browsers will try to cache an image so the server doesn't have to send it over the network again if the user reloads the page or returns to it later. However, the browser needs a way to know whether the image is up to date or not. If the server sends a 304 the browser knows it can continue to use the copy in its cache.
In your case you should implement some sort of caching to return the previous response if you recieve a 304.
Alternatively, I believe you could add these headers to the request to force it to return data
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0
I have been trying to get hold of the cookies I am setting in the server using PlayFramework:
response().setHeader(SET_COOKIE, AppConstants.COOKIE_USER_SESSIONID+"="+appSession.getSid());
in the angular app, but am not able to.
If I use the Advanced Rest client, I get the SetCookie header and the cookies get set in the call that follows. However when I call the same api through my angular app, I am not able to get the header in the response and hence no cookies for the app.
Here's what I have already tried:
renaming the environment from localhost to xyz.com since I read on multiple answers that cookies do not work at localhost.
Tried the api form a separate rest client, was able to fetch the header successfully, so the API looks fine to me.
There should be some catch in angular code which I am not able to figure out. Any help on this is highly appreciated. Thanks.
Angular code snippets:
LoginController:
angular.module('wratApp')
.controller('LoginCtrl', function (postService, wratSettings, wratRoutes, $location, $cookies) {
$("#header").hide();
this.user= {};
this.logUserin = function(){
console.log("User: " + this.user.email + " & pwd: " + this.user.pwd);
var payLoad = {};
payLoad.ldap = this.user.email;
payLoad.pwd = this.user.pwd;
postService.postPromise(wratRoutes(wratSettings).POST_USER_LOGIN(),payLoad)
.then(function(){ //login success
console.log("login success");
$location.path(wratRoutes().CLIENT_HOME());
}, function(){ //error in login
console.log("Login failed");
})
;
}
});
postService:
angular.module('wratApp')
.factory('postService', function ($http, $q, $cookies, $location) {
function postService() {
var self = this;
self.postPromise = function (uri,payload){
var deferred = $q.defer();
$http.post(uri,payload)
.success(function(data, status, headers, config){
console.log("Got response cookies: "+$cookies.getAll());
deferred.resolve(data);
})
.error(function(data, status, headers, config){
if(status == 401){
angular.forEach($cookies.getAll(), function (v, k) {
$cookies.remove(k);
});
$location.path('/login');
}else{
deferred.reject(data);
}
});
return deferred.promise;
}
}
return new postService();
});
Now, my login succeeds but without any cookies being set by the server. Also, the PlaySession cookie which is somehow visible in the debugger(the manually set ones are even absent from debugger), is not in the angular $cookies variable(refer image below).
Please suggest how can I resolve this. Thanks.
Update 1:
If I run the Play server not in debug mode, the cookies are appropriately being sent by the server. It's an issue with the angular app where in the following call, it's not transferring the values form the Set-Cookie header to the cookies for the next call. It might be some silly mistake on my end too. Please see if you can help me figure out.
Server response /login:
HTTP/1.1 200 OK
Vary: Origin
Set-Cookie: sid=1e6823e24554598b0521ca5f64d7746b; Path=/
Set-Cookie: PLAY_SESSION=953d9d5730bf1b77cccaadef6b78c209e59d924b-sid=1e6823e24554598b0521ca5f64d7746b; Path=/; HTTPOnly
Content-Type: application/json; charset=utf-8
Access-Control-Allow-Origin: http://wrat.com:9009
Access-Control-Allow-Methods: OPTIONS, GET, POST
Access-Control-Allow-Credentials: true
Date: Thu, 24 Sep 2015 22:27:03 GMT
Content-Length: 429
The following request which should have cookies set, but no cookies there :-( :
GET /products/all HTTP/1.1
Host: wrat.com:9000
Accept: application/json, text/plain, */*
Origin: http://wrat.com:9009
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
Referer: http://wrat.com:9009/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Taking off the Play framework tag since it's not a play issue anymore.
Adding the withCredentials option to the $http calls worked for me and resolved the issue.
$http.post(uri,payload, {withCredentials: true})
The issue is with the ajax calls where if this withCredentials flag is not truthy at both server and client, the cookies set by server are not maintained for cross-origin calls. Hence it was totally ignoring the cookies I was setting from the server.
Text from following question helped resolve the problem:
Why is jquery's .ajax() method not sending my session cookie?
from:
I am operating in cross-domain scenario. During login remote server is
returning Set-Cookie header along with
Allow-Access-Control-Credentials set to true.
The next ajax call to remote server should use this cookie.
CORS's Access-Control-Allow-Credentials is there to allow cross-domain
logging. Check https://developer.mozilla.org/En/HTTP_access_control
for examples.
For me it seems like a bug in JQuery (or at least feature-to-be in
next version).
UPDATE:
Cookies are not set automatically from AJAX response (citation:
http://aleembawany.com/2006/11/14/anatomy-of-a-well-designed-ajax-login-experience/)
Why?
You cannot get value of the cookie from response to set it manually
(http://www.w3.org/TR/XMLHttpRequest/#dom-xmlhttprequest-getresponseheader)
I'm confused..
There should exist a way to ask jquery.ajax() to set
XMLHttpRequest.withCredentials = "true" parameter.
ANSWER: You should use xhrFields param of
http://api.jquery.com/jQuery.ajax/
The example in the documentation is:
$.ajax({ url: a_cross_domain_url, xhrFields: {
withCredentials: true } }); It's important as well that server answers correctly to this request. Copying here great comments
from #Frédéric and #Pebbl:
Important note: when responding to a credentialed request, server must
specify a domain, and cannot use wild carding. The above example would
fail if the header was wildcarded as: Access-Control-Allow-Origin: *
So when the request is:
Origin: http://foo.example Cookie: pageAccess=2 Server should respond
with:
Access-Control-Allow-Origin: http://foo.example
Access-Control-Allow-Credentials: true
[payload] Otherwise payload won't be returned to script. See:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Requests_with_credentials
Say I got:
app.get('/json', function(req, res) {
res.set({
'content-type': 'application/json'
}).send('{"status": "0"}');
});
I'm trying to send the response as UTF-8 with the following with no success:
app.get('/json', function(req, res) {
// From Node.js Official Doc
// http://nodejs.org/api/http.html#http_http_request_options_callback
res.setEncoding('utf8');
res.set({
'content-type': 'application/json'
}).send('{"status": "0"}');
});
What is the correct way to set character encoding in Express?
You will probably want to explicitly add a charset to the end of your content-type string if you find it's not being set already by Express:
res.set({ 'content-type': 'application/json; charset=utf-8' });
The charset is not always set automagically and does need to be set to work correctly everywhere (i.e. with all browsers and all ajax libraries) or you can run into encoding bugs.
In Express 4.x specifically I've found that depending on the object you trying to return, it normally automatically returns with content-type: application/json; charset=utf-8 when you call res.json(someObject), however not always.
When calling res.json() on some objects it can return content-type: application/json (i.e. without the charset encoding!). I'm not actually sure what triggers this, other than it's something about the specific object being returned.
I've only noticed it because of automated tests which explicitly checked the headers and found it was missing the charset declaration on some responses (even though the content-type was still application/json).
Use res.charset: http://expressjs.com/api.html#res.charset
res.charset = 'value';
res.send('some html');
// => Content-Type: text/html; charset=value
However, JSON is UTF-8 by default so you don't need to set anything.
This worked for me
res.writeHead(200, {'Content-Type': 'text/html; charset=utf-8'});
Having similar issues I'm collecting Swedish characters from a database and outputting them as JSON object, node doesn't really care if json must be UTF-8 or not when the chars from the database isn't in UTF-8.. So assuming "you don't need to set anything" is false. Depending on what charsets you are working with.
Before you go to the trouble of manually setting header parameters, check what your server is already sending by default. In my case, I'm using a "serverless" cloud provided Node.js instance. Apparently, these are usually front-ended w/ NGINX which I assume is what sets some of this stuff based on default settings. ...I didn't need to res.set anything at all. Granted, I'm serving back HTML, ...just sayin - before you go fixin, make sure it's broke.
accept-ranges: bytes
accept-ranges: bytes
cache-control: private
content-encoding: gzip
content-type: text/html; charset=utf-8
date: Fri, 21 Dec 2018 21:40:37 GMT
etag: W/"83-xwilN/BBLLLAAAHHH/0NBLAH0U"
function-execution-id: 5thvkjd4wwru
server: nginx
status: 200
vary: accept-encoding, cookie, authorization
via: 1.1 varnish
x-cache: MISS
x-cache-hits: 0
x-cloud-trace-context: 18c611BBBBLLLLAAAHHH9594d9;o=1
x-powered-by: Express
x-served-by: cache-dfw18631-DFW
x-timer: S15BBLLLAAHHH.913934,VS0,VE3404
I need to set my custom http 'User-Agent header when I'm rendering my index.html page in Express.js app.
This doesn't help:
req.headers['user-agent'] = 'myHeader';
Is this possible?
The User-Agent header is sent by an HTTP client(browser) and is meant to be read by a server, for e.g., for Content Negotiation.
You cannot set a request header in a response, it can only be read. Moreover, the req object(IncomingMessage) passed to createServer() callback is a Readable stream.
However, a request can be initiated with a customer header using:
var headers = {'User-Agent': 'Ryan Dahl'};
http.request({hostname: 'nodejs.org', headers: headers}, function(res) {
});