Hello beloved SO community.
I have a problem that tortures me for months with no solution.
I am trying to make a request at an HTTP/2 endpoint that uses some headers that start with a colon. Example:
:method: "POST"
I have tried with python(hyper, requests), php(guzzle) and js(fetch).
I have managed, presumably, with js to achieve the required result but the CORS policy returns me an "opaque" result.
Any help will be great!
JS result presumbaly correct "opaque"
fetch("https://www.example.com/users/sign_in",
{
"credentials":"include",
"headers":{
"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
"accept-language":"en-US,en;q=0.9,el;q=0.8",
"cache-control":"max-age=0",
"content-type":"application/x-www-form-urlencoded",
"accept-encoding": "gzip, deflate, br",
"content-length": 614,
"origin": "https://www.example.com",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36",
},
"referrer":"https://www.example.com/users/sign_in",
"referrerPolicy":"no-referrer-when-downgrade",
"redirect": "follow",
"body":"..."
"method":"POST",
"mode":"no-cors",
":authority": "www.example.com",
":method": "POST",
":path": "/users/sign_in",
":scheme": "https"
})
Python result 500 or 404
context = tls.init_context()
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE
with HTTP20Connection('www.example.com', port=443, ssl_context=context) as c:
headers = {
":authority": "www.example.com",
":method": "GET",
":path": "/users/sign_in",
":scheme": "https",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.9,el;q=0.8",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
}
c.request('GET', 'www.example.com/users/sign_in', headers=headers)
PHP result 500 or curl error 55
$jar = new CookieJar;
$client = new Client([
// Base URI is used with relative requests
'cookies' => $jar,
'version' => 2.0,
'debug' => fopen('php://stderr', 'w'),
]);
$client->request('GET', 'https://www.example.com/users/sign_in');
$response = $client->request('POST', 'https://www.example.com/users/sign_in', [
'headers' => [
":authority"=> "www.example.com",
":method"=> "POST",
":path"=> "/users/sign_in",
":scheme"=> "https",
"accept"=> "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
"accept-encoding"=> "gzip, deflate, br",
"accept-language"=> "en-US,en;q=0.9,el;q=0.8",
"cache-control"=> "max-age=0",
"content-length"=> "616",
"content-type"=> "application/x-www-form-urlencoded",
"origin"=> "https://www.example.com",
"referer"=> "https://www.example.com/users/sign_in",
"sec-fetch-mode"=> "navigate",
"sec-fetch-site"=> "same-origin",
"sec-fetch-user"=> "?1",
"upgrade-insecure-requests"=> "1",
"user-agent"=> "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
],
'form_params' => [ ... ],
'version' => 2.0,
'allow_redirects' => true
]);
These are HTTP/2 pseudo-headers that apply to requests and responses within an HTTP/2 stream. HTTP/2 creates a single persistent connection from each distinct origin end-point to a server. That connection transmits multiple requests and responses to and from the end-point; these are parsed into "frames" and transmitted as "streams". HTTP/2 can interleave frames from multiple request and response streams simultaneously to get huge performance benefits. intro to http/2
Pseudo-headers apply to streams; a different set of headers apply to the connection itself. Four pseudo-headers are defined for requests: :method, :scheme, :authority, and :path. No others are allowed. These 4 must be included in every request header block and they must precede any other headers:
"All pseudo-header fields MUST appear in the header block before
regular header fields. Any request or response that contains a
pseudo-header field that appears in a header block after a regular
header field MUST be treated as malformed (Section 8.1.2.6)." http2 spec
I'm not familiar with how 'fetch' implements the headers, but in your code above you seem to have the pseudo-headers outside the header block, and they are placed at the end. Might be screwing up the fetch.
Related
I tried several approach to match this specific pattern of the url:
https://app.launchdarkly.com/sdk/goals/123123123
so the 123123123 will be always changing.also for some reason its making a OPTION call beside another GET call every time. Not sure why and that's likely another story...
nock return error like:
Error: Error: Nock: No match for request {
"method": "OPTIONS",
"url": "https://app.launchdarkly.com/sdk/goals/123123123",
"headers": {
"origin": "http://localhost",
"access-control-request-method": "GET",
"access-control-request-headers": "X-LaunchDarkly-User-Agent",
"user-agent": "Mozilla/5.0 (darwin) AppleWebKit/537.36 (KHTML, like Gecko) jsdom/16.5.3",
"host": "app.launchdarkly.com",
"content-length": 0
}
}
nock is not recognizing the pattern if I do (note that I am copying the same pattern as a GET as well)
nock('https://app.launchdarkly.com')
.persist()
.defaultReplyHeaders({
'access-control-allow-origin': '*',
'access-control-allow-headers': '*',
'access-control-allow-credentials': 'true',
})
.options('/sdk/goals.*$/')
.reply(200, mockLDExperiments);
or
nock('https://app.launchdarkly.com')
.persist()
.defaultReplyHeaders({
'access-control-allow-origin': '*',
'access-control-allow-headers': '*',
'access-control-allow-credentials': 'true',
})
.options('/sdk/goals/**/*')
.reply(200, mockLDExperiments);
or
nock('https://app.launchdarkly.com')
.persist()
.defaultReplyHeaders({
'access-control-allow-origin': '*',
'access-control-allow-headers': '*',
'access-control-allow-credentials': 'true',
})
.options('/sdk/goals')
.reply(200, mockLDExperiments);
any idea how to write the correct path matcher so I can allow this segment scenario gets picked up by nock?
Nock supports Regex path matching. It seems you're attempting something similar with globs, however, if a string is provided Nock only does exact matching.
Docs
For your case, something like this should get you going.
nock('https://app.launchdarkly.com')
...
.options(/^\/sdk\/goals\//)
...
I can't understand how the path resolution work in electron when you register a custom protocol as standard.
I have a basic custom protocol for app folder's files:
protocol.registerSchemesAsPrivileged([
{ scheme: 'app', privileges: {
standard: true,
secure: true,
supportFetchAPI: true,
corsEnabled: true,
bypassCSP: true
} }
])
app.on('ready', e=> {
protocol.registerFileProtocol('app',
(i,o)=> console.log(i.url)||
o({path: path.resolve(__dirname, i.url)})
)
let win = new BrowserWindow()
win.loadURL(`app://tabbed-window.html`)
}
<script type="module" src="utils/DOM.js"></script>
<link rel="import"
href="/components/Window.html" />
<link rel="stylesheet" type="text/css"
href="./components/Tab.css" />
<link rel="import"
href="../components/icon.html"
data-="themes/icons/app.svg"
data-fas="themes/icons/fa-solid.svg"
data-far="themes/icons/fa-regular.svg"
data-fab="themes/icons/fa-brands.svg" />
On the logs I have a leading slash after the main .html file so the nested ressources relatives path are wrong, getting the html file as a folder, whatever how each file is request (./ / even ../ don't avoid the .html/ folder !! )
{
method: 'GET',
url: 'app://tabbed-window.html/',
referrer: '',
headers: {
Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'
}
}
{
method: 'GET',
url: 'app://tabbed-window.html/utils/DOM.js',
referrer: '',
headers: {
Origin: 'app://tabbed-window.html',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) termos/0.3.0 Chrome/80.0.3987.163 Electron/8.2.3 Safari/537.36',
'Sec-Fetch-Dest': 'script',
Accept: '*/*'
}
}
{
method: 'GET',
url: 'app://tabbed-window.html/components/Window.html',
referrer: '',
headers: {
Origin: 'app://tabbed-window.html',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) termos/0.3.0 Chrome/80.0.3987.163 Electron/8.2.3 Safari/537.36',
'Sec-Fetch-Dest': 'unknown',
Accept: '*/*'
}
}
{
method: 'GET',
url: 'app://tabbed-window.html/components/Tab.css',
referrer: '',
headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) termos/0.3.0 Chrome/80.0.3987.163 Electron/8.2.3 Safari/537.36',
'Sec-Fetch-Dest': 'style',
Accept: 'text/css,*/*;q=0.1'
}
}
{
method: 'GET',
url: 'app://tabbed-window.html/components/icon.html',
referrer: '',
headers: {
Origin: 'app://tabbed-window.html',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) termos/0.3.0 Chrome/80.0.3987.163 Electron/8.2.3 Safari/537.36',
'Sec-Fetch-Dest': 'unknown',
Accept: '*/*'
}
}
The custom protocols in Electron have the same effect as the file:// protocol, wherein files are served from the local file system rather than the relative app path.
To sum up, when you use relative paths (../../), the URL is altered relative to your local filesystem, like so: file:///absolute/path/to/file. The absolute path uses the base href path (in case of angular).
The requests directed to your assets can be intercepted using intercept methods of the protocol api.
Proper URL Syntax is roughly scheme://host/path. So, you miss the host part. Use something like
win.loadURL(`app://localhost/tabbed-window.html`)
And strip the app://localhost/ prefix in the protocol handler.
I am trying to make an Ajax call to hit a webservice and get the response.But getting uncaught syntaxerror:unexpected token < in google chrome console.
Here's my Ajax Request:
$(document).ready(function(){
$.ajax({
url:"http://10.10.1.5:8089/axis2/services/cmtlpmservice/getAllMonitors",
dataType:"jsonp",
jsonpCallback:"callback",
type:"GET",
success:function(response){
console.log(response);
}
});
});
The Request header is as follows:
Request URL:http://10.10.1.5:8089/axis2/services/cmtlpmservice /getAllMonitors?callback=callback&_=1487912464838
Request Method:GET
Status Code:200 OK
Remote Address:10.10.1.5:8089
Accept:/
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Authorization:Basic YWRtaW46
Cache-Control:max-age=0
Connection:keep-alive
Host:10.10.1.5:8089
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36
The Response Header as follows:
Cache-Control:no-cache
Content-Type:application/xml;charset=UTF-8
Date:Fri, 24 Feb 2017 05:01:05 GMT
Expires:Thu, 01 Jan 1970 10:00:00 EST
Pragma:No-cache
Server:Apache-Coyote/1.1
Transfer-Encoding:chunked
X-Powered-By:Servlet 2.5; JBoss-5.0/JBossWeb-2.1
The Response looks like :
<ns:getAllMonitorsResponse xmlns:ns="ws.core.dorado.com">
<ns:return>
{
"monitors": [{
"id": "com.dorado.broadscope.monitor.Monitor::RGNEXGvol54w2a#_1.3.6.1.4.1.20138.800.20.12.1.9.1",
"maxAttrName": "ifTemperatureValue Max",
"name": "ifTemperatureValue",
"minAttrName": "ifTemperatureValue Min",
"maxAttrId": "com.dorado.broadscope.monitor.Monitor::RGNEXGvol54w2a#_1.3.6.1.4.1.20138.800.20.12.1.9.1Max",
"minAttrId": "com.dorado.broadscope.monitor.Monitor::RGNEXGvol54w2a#_1.3.6.1.4.1.20138.800.20.12.1.9.1Min"
}]
}
</ns:return>
</ns:getAllMonitorsResponse>
Due to some error in getAllMonitors it returns whole html page instade of json deta in response. your controller is culpit insted of your ajax.
I am trying to write a Single Page App (SPA) based on AngularJS. The app should connect to a webserver that provides a RestFul services, but each end point requires username and password along with other parameters. Since I am a beginner in this area, before moving towards the actual development, I tried PostMan/Advanced Rest Client chrome extensions to verify the basic connections. A sample request preview :
POST /servicesNS/admin/search/search/jobs/export HTTP/1.1
Host: localhost:8089
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
search=search+error+|+table+host&output_data=xml&username=admin&password=unity3d
This is actually equivalent to the cURL command:
curl -k -u admin:unity3d --data-urlencode search="search error | table host" -d "output_mode=xml" https://localhost:8089/servicesNS/admin/search/search/jobs/result
After getting successful results in above mentioned ways, I am now looking for equivalent way of doing it in AngularJS.
var app = angular.module('TabsApp', []);
app.controller('TabsCtrl', function ($scope, $http)
{
login = function () {
$scope.userName ="admin";
$scope.password ="unity3d"
$http({
method :"POST",
url:"https://localhost:8089/servicesNS/admin/search/search/jobs/export",
data: { "username" : "admin" , "password": "unity3d", "search" : "search error"},
headers: {'Content-Type': 'application/x-www-form-urlencoded'}
}).success(function (data, status, headers, config) {
console.log('status',status);
console.log('data',status);
console.log('headers',status);
});
}
});
This gives me error 401 Unauthorized, the headers of the response:
> Remote Address:127.0.0.1:8089 Request
> URL:https://localhost:8089/servicesNS/admin/search/search/jobs/export
> Request Method:POST Status Code:401 Unauthorized Request Headersview
> source Accept:application/json, text/plain, */* Accept-Encoding:gzip,
> deflate Accept-Language:en-US,en;q=0.8 Connection:keep-alive
> Content-Length:65 Content-Type:application/x-www-form-urlencoded
> Host:localhost:8089 Origin:http://localhost:63342
> Referer:http://localhost:63342/UI/UI1.html User-Agent:Mozilla/5.0
> (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/39.0.2171.71 Safari/537.36 Form Dataview sourceview URL encoded
> {"username":"admin","password":"unity3d","search":"search error"}:
> Response Headersview source Access-Control-Allow-Credentials:true
> Access-Control-Allow-Headers:Authorization
> Access-Control-Allow-Methods:GET,POST,PUT,DELETE,HEAD,OPTIONS
> Access-Control-Allow-Origin:* Cache-Control:private
> Connection:Keep-Alive Content-Length:130 Content-Type:text/xml;
> charset=UTF-8 Date:Sat, 29 Nov 2014 19:53:59 GMT Server:Splunkd
> Vary:Cookie, Authorization WWW-Authenticate:Basic realm="/splunk"
> X-Content-Type-Options:nosniff X-Frame-Options:SAMEORIGIN
And the output is :
<?xml version="1.0" encoding="UTF-8"?> <response> <messages>
<msg type="ERROR">Unauthorized</msg> </messages> </response>
Any idea what is going wrong?
If you are sending response in the format of 'application/x-www-form-urlencoded' the actual data format should be the same.
What you are sending currently is a JSON object. You would need to use $http tranformer to tranform the request: Something in line of
transformRequest: function (data) {
var postData = [];
for (var prop in data)
postData.push(encodeURIComponent(prop) + "=" + encodeURIComponent(data[prop]));
return postData.join("&");
},
See a working fiddle here http://jsfiddle.net/cmyworld/doLhmgL6/
I have a URL
ex. www.siteA.com/?UGLY_KEY_12312432342SSDFSD
It then redirects you to:
www.siteB.com/?ANOTHER_UGLY_KEY_ASDASDFS2342342
What i need is some way to catch the redirect URL of siteB
I've tried JQuery $.ajax
and I am swamped with
No 'Access-Control-Allow-Origin' header is present on the requested resource.
I am aware that CORS is the typical way to go, but it is impossible in my case.
Shouldn't this be easier security wise as it is a mere, GET?
$.ajax({
type: "GET",
url: "www.siteA.com/?UGLY_KEY_12312432342SSDFSD",
dataType: "json",
success: function (response, status, request) {
// data.redirect contains the string URL to redirect to
alert(response.redirectUrl);
}
}
I need to use a HttpWebRequest on the server side because of CORS Permission issues.
Here it is:
var httpWebRequest = (HttpWebRequest)WebRequest.Create("http://www...");
httpWebRequest.ContentType = "text/json";
httpWebRequest.Method = "GET";
httpWebRequest.AllowAutoRedirect = false;
httpWebRequest.Timeout = 10000;
httpWebRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36";
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
string THEREDIRECTURL = httpResponse.GetResponseHeader("Location");