Implementing a simple HTTP server in Qt, with the purpose of streaming real time data to an XMLHttpRequest object (AJAX/JavaScript).
The problem is that the design pattern requires partial transmission of data via the socket connection, changing the readyState in the XHR from '1' (Request) to '2' (Headers received), and then to '3' (Data received) - keeping the request pending. This is also known as "long-polling", or "Comet" and should be possible in most browsers.
However, it stays in request state until the connection is closed, and then readyState '2' and '4' are received. This is normal for HTTP GET, but not desired for this application.
JavaScript:
var request = new XMLHttpRequest();
request.onreadystatechange = function() {
console.log('readyState: ' + this.readyState + ' ' + this.status)
}
request.open("get", "localhost:8080/", true);
request.send();
Qt:
connect(socket, &QTcpSocket::readyRead, [=]()
{
QByteArray data = m_socket->read(1000);
socket->write("HTTP/1.1 200 OK\r\n");
socket->write("Content-Type: text/octet-stream\r\n");
socket->write("Access-Control-Allow-Origin: *\r\n");
socket->write("Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\n");
socket->flush();
}
So the big question is: How can I make the network system underneath the QtTcpSocket flush pending data after writing the headers (and later, the data), without the need to disconnect first?
A side note: I originally implemented this using WebSockets, but the browser I have to use does not support this.
EDIT:
The HTTP header formatting must have an extra set of "\r\n". Now it works:
connect(socket, &QTcpSocket::readyRead, [=]()
{
QByteArray data = m_socket->read(1000);
socket->write("HTTP/1.1 200 OK\r\n");
socket->write("Content-Type: text/octet-stream\r\n");
socket->write("Access-Control-Allow-Origin: *\r\n");
socket->write("Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\n");
socket->write("\r\n");
socket->flush();
}
Got it working now after a full day of trying different HTTP header configurations. It seems like user 'peppe' was on to something, and the only thing I needed was to add "\r\n" after the headers! (See edit).
Related
I'm trying to make a script that will automate something for me and my goal is to make an HTTP request and grab the "set-cookie" that is in the response's headers.
I understand why it's not accessible when done from a normal web page (httpOnly and all) but I am doing it from the background script in a chrome extension.
I tried using both fetch and xhr but they both didn't show me the set-cookie header in the response.
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function(){
if (xhr.readyState === 4) {
console.log("headers", xhr.getAllResponseHeaders()); // doesnt return httpOnly headers
}
};
xhr.open("GET", 'https://' + host, true);
xhr.send();
or
fetch('https://' + host, {method: 'GET'})
Is there any way to get permission to read those headers? Maybe I need to use something other than fetch? Any help would be greatly appreciated.
I use AJAX to check user updates every 2 seconds, but my javascript does not update the response.
I have one javascript file with XMLHttpRequest object and every 2 seconds it sends a request to another file (.php) where it gets XML with updates. For some reason, it doesn't always get the newest content and seems to have some old cached.
My javascript file contains this code (simplified):
var updates = new XMLHttpRequest();
updates.onreadystatechange = function(){
"use strict";
if(updates.readyState === 4 && updates.status === 200){
console.log(updates.responseXML);
}
};
var timer = 0;
clearInterval(timer);
timer = setInterval(function(){
"use strict";
updates.open('GET','scripts/check_for_notifications.php', true);
updates.send();
},2000);
Then I have the PHP file (check_for_notifications.php), where I have this code:
$response = new SimpleXMLElement('<xml/>');
$update = $response->addChild('update');
$update->addChild('content', 'New message');
$update->addChild('redirect', 'some link');
$update->addChild('date', '1.1.2019 12:00');
header('Content-type: text/xml');
print($response->asXML());
Every two second I receive a log in my console, but when I change the PHP file, while the interval is in progress (e.g. I change the date to '1.1.2019 11:00' and save it) I still receive the '12:00' in the console. For me, it seems that it doesn't update and it still has the repsonseXML cached. Is there any way I could "flush" the output or am I doing it wrong?
It's probably a cache problem. In the network browser console, you should see a response of type 304 Not modified.
To be sure, you can add an element in the url to bypass the cache:
updates.open('GET','scripts/check_for_notifications.php&nocache=' + new Date().getTime(), true);
If this works, you will need to configure the server (apache or nginx) to prevent the file from being cached. It's cleaner than the timestamp solution. the browser does not store cached files unnecessarily.
Apache .htaccess or Apache conf, something like
<Files "check.php">
<IfModule mod_headers.c>
Header set Cache-Control "no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires 0
</IfModule>
</Files>
Nginx conf, something like
location = /check.php {
add_header 'Cache-Control' 'no-cache, no-store, must-revalidate';
expires off;
}
You can also see the fetch api : https://hacks.mozilla.org/2016/03/referrer-and-cache-control-apis-for-fetch/
Be careful with your code, it can launch several requests simultaneously and find yourself with a DOS if there are too many users.
If requests take more than 2 seconds because the server is slow, others requests will be sent in the meantime, which will slow down the server even more....
I have a site hosted on https:// in which I want to pull data from the site which shows the properties of the shares. The URL which returns the data is:
http://ir1.euroinvestor.com/asp/ir/xmlirmultiiso2.aspx?companyid=281191
The code which I have developed to get the data is as follows:
function GetSharesUpdate(){
// makeing AJAX calls to the web service for getting the share update
var xhr = new XMLHttpRequest(); // Set up xhr request
xhr.open("GET", "http://ir1.euroinvestor.com/asp/ir/xmlirmultiiso2.aspx?companyid=281191", true); // Open the request
xhr.responseType = ""; // Set the type of response expected
xhr.send();
// Asynchronously wait for the data to return
xhr.onreadystatechange = function () {
if (xhr.readyState == xhr.DONE) {
var tempoutput = xhr.responseXML;
alert(tempoutput);
}
}
// Report errors if they happen
xhr.addEventListener("error", function (e) {
console.error("Error: " + e + " Could not load url.");
}, false);
}
I am getting the error on the statement xhr.send(); which is as below:
Mixed Content: The page at 'https://[SiteUrl]' was loaded over HTTPS,
but requested an insecure XMLHttpRequest endpoint
'http://ir1.euroinvestor.com/asp/ir/xmlirmultiiso2.aspx?companyid=281191'.
This request has been blocked; the content must be served over HTTPS.
If I change the URL to https i.e.
https://ir1.euroinvestor.com/asp/ir/xmlirmultiiso2.aspx?companyid=281191
then xhr.send(); is executed without any error but I am getting xhr.responseXML null.
What should I make changes in my code to make it working?
The first problem is you are doing ajax request to non-https domain but your domain has SSL installed.
The second problem is Cross origin request CORS, you will have this problem even you solve first (ie. even you try ajax request from non-http domain).
Since you are requesting data from another website this is most likely to happen unless the requested server is configured to serve requests for you domain
To solve this you need to call the data from your server(proxy server) or have the requested server configured to allow requests from your domain.
See more - https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
I'm developing a Firefox OS client for ownCloud. When I try to login and send the user credentials to the server, I need to obtain in response the cookie that I will use to authenticate in ownCloud in each request.
My problem is that as I’ve seen in Wireshark, the cookie is sent in a HTTP 302 message, but I cannot read this message in my code because Firefox handles it automatically and I read the final HTTP 200 message without cookie information in the
request.reponseText;
request.getAllResponseHeaders();
So my question is if there is any way to read this HTTP 302 message headers, or if I can obtain the cookie from Firefox OS before I send the next request, or even make Firefox OS to add the cookie automatically. I use the following code to make the POST:
request = new XMLHttpRequest({mozSystem: true});
request.open('post', serverInput, true);
request.withCredentials=true;
request.addEventListener('error', onRequestError);
request.setRequestHeader("Cookie",cookie_value);
request.setRequestHeader("Connection","keep-alive");
request.setRequestHeader("Content-type","application/x-www-form-urlencoded");
request.send(send_string);
if(request.status == 200 || request.status==302){
response = request.responseText;
var headers = request.getAllResponseHeaders();
document.getElementById('results').innerHTML="Server found";
loginSuccessfull();
}else{
alert("Response not found");
document.getElementById('results').innerHTML="Server NOT found";
}
"mozAnon
Boolean: Setting this flag to true will cause the browser not to expose the origin and user credentials when fetching resources. Most important, this means that cookies will not be sent unless explicitly added using setRequestHeader.
mozSystem
Boolean: Setting this flag to true allows making cross-site connections without requiring the server to opt-in using CORS. Requires setting mozAnon: true, i.e. this can't be combined with sending cookies or other user credentials." [0]
I'm not sure if you're an owncloud developer, but if you are and have access to the server, you should try setting CORS headers. [1] Maybe if you can stand up a proxy server and have your app connect to the proxy server that does have CORS enabled?
There's also a withCredentials property [2] you can set on instances of xhr objects. It looks like it will add the header Access-Control-Request-Headers: "cookies" and send an HTTP OPTIONS request, which is the preflight [3]. So this would still require server side support for CORS. [4]
Though it seems like this shouldn't work based on internal comments [5], I was able to run this from a simulator and see the request and response headers:
var x = new XMLHttpRequest({ mozSystem: true });
x.open('get', 'http://stackoverflow.com');
x.onload = function () { console.log(x.getResponseHeader('Set-Cookie')); };
x.setRequestHeader('Cookie', 'hello=world;');
x.send();
You'd probably want to reassign document.cookie in the onload event, rather than logging it, if the response header exists (not every site sets cookies on every request). You'd also want to set the request header to document.cookie itself.
[0] https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#XMLHttpRequest%28%29
[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
[2] https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#Properties
[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Preflighted_requests
[4] http://www.html5rocks.com/en/tutorials/cors/#toc-making-a-cors-request
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=966216#c2
It seems that I am unable to change most request headers from JavaScript when making an AJAX call using XMLHttpRequest. Note that when request.setRequestHeader has to be called after request.open() in Gecko browsers (see http://ajaxpatterns.org/Talk:XMLHttpRequest_Call). When I set the Referer, it doesn't get set (I looked at the request headers sent using Firebug and Tamper Data). When I set User-Agent, it messed up the AJAX call completely. Setting Accept and Content-Type does work, however. Are we prevented from setting Referer and User-Agent in Firefox 3?
var request = new XMLHttpRequest();
var path="http://www.yahoo.com";
request.onreadystatechange=state_change;
request.open("GET", path, true);
request.setRequestHeader("Referer", "http://www.google.com");
//request.setRequestHeader("User-Agent", "Mozilla/5.0");
request.setRequestHeader("Accept","text/plain");
request.setRequestHeader("Content-Type","text/plain");
request.send(null);
function state_change()
{
if (request.readyState==4)
{// 4 = "loaded"
if (request.status==200)
{// 200 = OK
// ...our code here...
alert('ok');
}
else
{
alert("Problem retrieving XML data");
}
}
}
W3C Spec on setrequestheader.
The brief points:
If the request header had
already been set, then the new value
MUST be concatenated to the existing
value using a U+002C COMMA followed by
a U+0020 SPACE for separation.
UAs MAY give the User-Agent header an initial value, but MUST allow authors to append values to it.
However - After searching through the framework XHR in jQuery they don't allow you to change the User-Agent or Referer headers. The closest thing:
// Set header so the called script knows that it's an XMLHttpRequest
xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest");
I'm leaning towards the opinion that what you want to do is being denied by a security policy in FF - if you want to pass some custom Referer type header you could always do:
xhr.setRequestHeader('X-Alt-Referer', 'http://www.google.com');
#gnarf answer is right . wanted to add more information .
Mozilla Bug Reference : https://bugzilla.mozilla.org/show_bug.cgi?id=627942
Terminate these steps if header is a case-insensitive match for one of the following headers:
Accept-Charset
Accept-Encoding
Access-Control-Request-Headers
Access-Control-Request-Method
Connection
Content-Length
Cookie
Cookie2
Date
DNT
Expect
Host
Keep-Alive
Origin
Referer
TE
Trailer
Transfer-Encoding
Upgrade
User-Agent
Via
Source : https://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#dom-xmlhttprequest-setrequestheader
For people looking this up now:
It seems that now setting the User-Agent header is allowed since Firefox 43. See https://developer.mozilla.org/en-US/docs/Glossary/Forbidden_header_name for the current list of forbidden headers.