I'm making an api that communicates with a website to pull player statistics. I've made multiple POST/GET HTTP/1 requests to the server to get a session token and player ID. I then use those values(valid values which I have tested before passing to my function) in my last function to fetch player statistics. The last request is a HTTP/2 GET request. I'm using the got library and vanilla Node. Here is my request:
//THESE ALL HAVE SOME VALUE AFTER I USE SOME OF MY FUNCTIONS; THE FUNCTION I'M
//HAVING TROUBLE WITH IS THE LAST FUNCTION AND IS PASSED VERIFIED NON-NULL VALUES
var session = {
app_id: '3587dcbb-7f81-457c-9781-0e3f29f6f56a',
space_id: '5172a557-50b5-4665-b7db-e3f2e8c5041d',
session_id: null,
ticket: null,
};
var player = {
name: null,
id: null,
platform: 'uplay',
kills: null,
deaths: null,
rank: null,
};
async function get_player_stats(session, player) {
var platform = 'PC';
if (player.platform === 'uplay') {
platform = 'PC';
}
var options = {
':authority': 'r6s-stats.ubisoft.com',
':method': 'GET',
':path': `/v1/current/operators/${player.id}?gameMode=all,ranked,casual,unranked&platform=${platform}&teamRole=attacker,defender&startDate=20200723&endDate=20201120`,
':scheme': 'https',
'authorization': `ubi_v1 t=${session.ticket}`,
'ubi-appid': session.app_id,
'ubi-sessionid': session.session_id,
'content-type': 'application/json',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36',
}
const url = `https://r6s-stats.ubisoft.com/v1/current/operators/${player.id}?gameMode=all,ranked,casual,unranked&platform=${platform}&teamRole=attacker,defender&startDate=20200723&endDate=20201120`;
try {
const response = got(url, {headers: options, http2: true});
console.log(response);
}
catch (err) {
console.log(err);
}
}
//FUNCTION CALL
async function fetch(user) {
var stats_string = await get_player_stats(session, player);
console.log(stats_string);
}
fetch(username);
Chrome's request header from network log:
:authority: r6s-stats.ubisoft.com
:method: GET
:path: /v1/current/operators/e96ae749-8939-43ed-895f-bf1817e849d9?gameMode=all,ranked,casual,unranked&platform=PC&teamRole=attacker,defender&startDate=20200723&endDate=20201120
:scheme: https
accept: */
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
authorization: ubi_v1 t= LONG TOKEN
dnt: 1
expiration: 2020-11-21T09:13:54.804Z
origin: https://www.ubisoft.com
referer: https://www.ubisoft.com/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site
ubi-appid: 3587dcbb-7f81-457c-9781-0e3f29f6f56a
ubi-sessionid: d78f3306-0e5c-4ac8-ad63-5a711b816f76
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36
Chrome's Response header from network tools:
access-control-allow-credentials: true
access-control-allow-origin: https://www.ubisoft.com
content-encoding: gzip
content-length: 16969
content-type: application/json
date: Sat, 21 Nov 2020 06:14:47 GMT
status: 200
vary: Origin
What I've tried:
I've tried just about everything. I've googled what causes 400 errors, which apparently are mostly user error, and I've looked through my code for days and also looked at Chrome's network activity. I've tried matching Chrome's request header with mine to no avail(my header variable is one of many iterations I've tried--pretty sure I've tried every combination of possible headers). However, sometimes I'll get 400 bad error, or an invalid header response from the server. I've tried using the vanilla Node http2Client.request and that gives me an invalid header/400 as well.
Okay, finally figured out why this wasn't working. I missed one tiny line in what I thought I already tried millions of times.
In the request header on the Chrome Network activity there is a field for expiration.
I needed to set the expiration value in the header to get the data.
So the value I needed to add to my header in my code was:
expiration: 2020-11-21T09:13:54.804Z
Future edit: The expiration is the date in ISO format. You can make a date Object and convert to ISO:
var time = new Date();
var expiration = time.toISOString();
function someRequest() {
var options = {
'expiration': expiration,
}
}
Related
I was just trying to make covid vaccine alert using Cowin Setu API (India) in nodejs. But I am facing some strange thing, whenever I hit get request I got 403 response code from cloudfront says 'Request Blocked' but the same is working from postman as well as from browser. Please help me in this
Getting this error:-
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: Q1RZ94qgFp6AjUUKE4e9urMB85VejcqMbaJO6Y8Xq5Qp4kNjDBre9A==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>
Here's my nodejs code:
var express = require("express");
var app = express();
var bodyParser = require("body-parser");
const axios = require("axios");
const { Telegram } = require("telegraf");
const fetch = require("node-fetch");
var cors = require('cors');
var request=require('request');
const tg = new Telegram(process.env.BOT_TOKEN);
const bot = new Telegram(process.env.BOT_TOKEN, {
polling: true
});
//bot.start((ctx) => ctx.reply('Welcom to Covid Vaccine Finder'))
/*bot.hears("about", ctx => {
ctx.reply("Hey, I am CoviBot!");
});
bot.launch();*/
app.use(bodyParser.json());
app.use(cors());
app.use(
bodyParser.urlencoded({
extended: true
})
);
app.get("/", function(req, res) {
res.send("Welcom to Covid Vaccine Finder");
});
app.get("/test", function(req, res, next) {
var d = new Date();
var options = {
year: "numeric",
month: "2-digit",
day: "2-digit"
};
var date = String(d.toLocaleDateString("en", options));
date = date.replace(/\//g, "-");
console.log(date);
const URL =
"https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByPinpincode=110088&date=13-05-2021";
var options = {
url: URL,
method: 'GET',
headers: {
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'en-GB,en;q=0.8,en-US;q=0.6,hu;q=0.4',
'Cache-Control': 'max-age=0',
Connection: 'keep-alive',
Host: 'cdn-api.co-vin.in',
'User-Agent': 'request',
}
};
request(options,function(err,res,body){
let json = body;
console.log(json);
});
const txt = "Finding vaccine centres for you....";
//tg.sendMessage(process.env.GROUP_ID, txt);
res.send(txt);
});
// Finally, start our server
app.listen(process.env.PORT, function() {
console.log("Covid app listening on port 3000!");
});
I hope this problem will solve
Thanks
I added a user-agent header to the request so that the API would recognize that my request is coming from a browser, rather than a script.
headers = {
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
}
url = "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/calendarByDistrict?district_id=303&date="+date
response = requests.get(url, headers=headers)
Use following
var options = {
url: URL,
method: 'GET',
headers: {
Host: 'cdn-api.co-vin.in',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36'
}
};
Try These Headers They worked for me on local server (not production)
let options = {
headers: {
"user-agent":
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36",
},
};
These will not work in production because Cowin APIs are geofenced and can't be accessed from IP address other than Indian. In most free hosting sites like Heroku, Indian IP is not an option. So alternative solution might be to use AWS, GCP, Azure with an Indian server (not tried yet).
Reference - https://github.com/cowinapi/developer.cowin/issues/228
It seems the api is blocked from using outside India. Try to combine some Indian proxy/use in Indian server
You have to use User Agent Identifier API
Please refer this
https://devcenter.heroku.com/articles/useragentidentifier#using-with-python
You have to make your request in the following format, I am attaching sample format for states metadata API:
curl --location --request GET 'https://cdn-api.co-vin.in/api/v2/admin/location/states' --header 'Accept-Language: hi_IN' --header 'Accept: application/json' --header 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36'
Its not about the request user-agent or format. I faced the same issue and further testing proved cloudFront is blocking the IP if multiple requests are coming from same IP back to back. Its also unblocking after couple minutes.
Basically they don't want these alerting this, probably its overloading their server.
Ok if you want to work local you can use
let headers = {
'accept': 'application/json',
'Accept-Language': 'hi_IN',
'X-Requested-With': 'XMLHttpRequest',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36',
}
Now if you want to deploy to Heroku or firebase, then it will return 403, I think it's mostly that they are blocking any IP hit outside from Indian server.
Github link: https://github.com/manojkumar3692/reactjs_nodejs_cowin
I Will keep you posted here
I'm following GitHub Docs try to get issue list and post issues.
I've managed to get the issue list using
GET https://api.github.com/repos/wheatup/wheatup.github.io/issues
But when I try to post an issue to the repo, I got a 404 error with following body:
{
"message": "Not Found",
"documentation_url": "https://docs.github.com/rest/reference/issues#create-an-issue"
}
Here's my post request:
URL
POST https://api.github.com/repos/wheatup/wheatup.github.io/issues
Headers
Accept: application/vnd.github.v3+json
Accept-Encoding: gzip, deflate, br
Accept-Language: en,zh-CN;q=0.9,zh;q=0.8,ja;q=0.7,zh-TW;q=0.6,sr;q=0.5,pl;q=0.4,la;q=0.3
Authorization: token d7fa1e545c*******************31957a97e06
Cache-Control: no-cache
Connection: keep-alive
Content-Length: 79
Content-Type: application/json
DNT: 1
Host: api.github.com
Origin: http://localhost:3000
Pragma: no-cache
Referer: http://localhost:3000/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
Body
{
title: "Test",
body: "test content",
labels: [],
assignees: [],
milestone: 1
}
This is how I post the request:
const result = await axios.post('https://api.github.com/repos/wheatup/wheatup.github.io/issues', {
title: 'Test',
body: 'test content',
labels: [],
assignees: [],
milestone: 1,
}, {
headers: {
'Authorization': 'token d7fa1e545c*******************31957a97e06',
'Content-Type': 'application/json',
'Accept': 'application/vnd.github.v3+json'
}
});
My repo is public, did I miss something? Any help would be greatly appreciated!
It seems you are missing the github token in the request. I get 404 on my local until I add the bearer token. Then I get 401 because I am not using an actual bearer token to hit your repo. So once you add that part, it all should work.
Solution 1:
const result = await axios.post('https://api.github.com/repos/wheatup/wheatup.github.io/issues', {
title: 'Test',
body: 'test content',
// labels: [], --> Since empty, commented out as it is optional param
// assignee: '', --> Since empty, commented out as it is optional param. Also you had a typo and this attributes expects string not array
milestone: 1,
}, {
headers: {
'Authorization': `Bearer ${githubToken}`,
'Content-Type': 'application/json',
'Accept': 'application/vnd.github.v3+json'
}
});
When dealing with github API, I would suggest use their toolkit instead because you only need to provide the token once and then subsequent request can just have the data provided to them
Alternative Solution 2: --> This only applies to not have to deal with passing bearer token on every request so ignore if you rather keep using axios.
const octokit = new Octokit({ auth: githubToken });
const response = await octokit.request('POST /repos/{owner}/{repo}/issues', {
owner: 'wheatup',
repo: 'wheatup.github.io',
title: 'Test',
body: 'test content',
milestone: 1,
});
EDIT
Solution 3: -> Actual solution to the problem provided above
When dealing with OAuth apps there are some steps to take
Users are redirected to request their GitHub identity
Users are redirected back to your site by GitHub
Your app accesses the API with the user's access token
NOTE: When calling the request identity make sure to require the scope necessary for the API calls to make. In this case the 404 was received due to token not having proper permissions to the repo as scope was missing.
More information about oauth API calls can be found here
https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/
When I am trying to receive a message from Socket.IO using the native socket module, instead of receiving the message, I receive this:
GET /socket.io/?EIO=3&transport=polling&t=MIlsTQ_ HTTP/1.1
Host: localhost:5000
Connection: keep-alive
Accept: */*
Origin: http://localhost:8080
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
DNT: 1
Referer: http://localhost:8080/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
How do I fix this, and receive the message rather than the data?
Client code:
socketsSend: function() {
const socket = io.connect('http://localhost:5000');
/*var socket = io.Socket('http://localhost', {
port: 5000
});*/
socket.connect();
socket._connectTimer = setTimeout(function() {
socket.close();
}, 500);
socket.on('connect', function() {
// socket connected successfully, clear the timer
clearTimeout(socket._connectTimer);
});
Server code:
import socket
import json
addr = 'localhost',5000
sock=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(addr)
sock.listen(3)
while True:
connection, client_address = sock.accept()
data = connection.recv(100000)
print(data.decode('utf-8'))
From this how to guide:
In general, they [socket's recv and send functions] return when the associated network buffers have been filled (send) or
emptied (recv). They then tell you how many bytes they handled. It is your
responsibility to call them again until your message has been completely dealt with.
So, from my understanding, connection.recv(100000) won't return any data until its buffer fills up, so to fix this you would need to set buffer to something low, (maybe 2048?).
control:no-cache` field into my request when requesting RSS feed
I cant quit figure what values should i put in
In case of Content-type it works well but it refuses to add correctly Cachce-Control
code :
options = {uri :SUPPORT_FEED_URI,
headers : {
'Content-Type': 'application/x-www-form-urlencoded',
'Cache-Control': 'no-cache'
},
}
request.get(options)
.on('error', (err) => { reject(err); })
.pipe(feedparser)
.on('end', () => { return resolve(items); });
What i get in request headers :
Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:cache-control <-- doesnt seems to be right Want something like Cache-Control : no-cache
Access-Control-Request-Method:GET
Connection:keep-alive
Host: xxxx.yyyy.zz
Origin:http://127.0.0.1:8888
Referer:http://127.0.0.1:8888/webconsole/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
content-type:application/x-www-form-urlencoded
Your capture is a CORS pre-flight (OPTIONS) request as the URL is on a different domain or considered to be different-origin.
Such a request will not include custom headers, they are added to Access-Control-Request-Headers instead to see if the destination server will allow them.
If the destination server responds with an acceptable allow- response the subsequent GET will include your header.
Depends what you are trying to achieve.
If you are trying to force a non-cached response and dont have control over the server, one thing you can do is to add a fake query param like this.
options = {
uri :`${SUPPORT_FEED_URI}?${new Date().getTime()}`,
headers : {
'Content-Type': 'application/x-www-form-urlencoded'
},
}
For more information on the 'Cache-Control' header see the top answer here.
What's the difference between Cache-Control: max-age=0 and no-cache?
response is JSON
Edge: sometimes inserts response property value into DOM briefly then removes it, sometimes logs error "SCRIPT5: access denied" (indicating CORS), response fully accessible from Debugger, request shown in network tab
Chrome: response empty string, request not shown in network tab, no console message
Firefox: console error 'response "malformed JSON"' on breakpoint line using response in JSON.parse(), thus before usage, request not shown in network tab, Firebug and integrated
JS (current browsers only):
var session = "";
var request;
function checkLogin()
{
if(request.readyState > 3)
{
var response = JSON.parse(request.response);
if(verify(response)) // verify inserts argument property "error" in DOM on error via innerHTML on element
{
// do something
}
}
}
function login()
{
request = new XMLHttpRequest();
request.onreadystatechange = checkLogin;
request.open("GET", "authenticateUser.php?user=" + document.getElementById("user").value + "&credential="+md5(document.getElementById("password").value));
request.send();
}
Edge request from network tab:
Anforderungs-URL: http://*MYDOMAIN*/authenticateUser.php?user=df&credential=d41d8cd98f00b204e9800998ecf8427e
Anforderungsmethode: GET
Statuscode: 200 / OK
- Anforderungsheader
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: de-DE, de; q=0.8, en-US; q=0.5, en; q=0.3
Connection: Keep-Alive
Host: *MYDOMAIN*
Referer: http://*MYDOMAIN*/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586
- Antwortheader
Connection: Keep-Alive
Content-Length: 65
Content-Type: application/json
Date: Sat, 21 Nov 2015 07:42:17 GMT
Keep-Alive: timeout=15, max=94
Server: Apache
X-Powered-By: PHP/5.5.29
response:
{"success":false,"error":"authentication failed or unauthorised"}
What is going to make my phps' response be usable in JS code (in Chrome and Firefox) and its properties value be insertable into the DOM?
login() is the handler of onsubmit. I didn't return false; thus the form from which the user and pw came from - having no action - submitted to its own page and made the browser reload the page: cancelling the request in Chrome and Ff. Only Edge let the equal page JS handle the repsonse from the former loads request - until it was to be inserted.
Took me 6 hours and a detour through JSONP and Charles then listing the network traffic from Chrome where the page itself appeared after the request thus letting me have the idea that the page "reloaded" and finally remembering onsubmits handler cancels the submit with return false and else submits leading to a load of the action (or self if none) which is what happened here.