We Keep Coding

SPA: Access Instagram authentication code directly in app

I load user profile informations from instagram by the basic api in a local test app.
So this actually works like written in the documentations.
I used Postman to get along the authentication and token stuff.
The order to access the graph api, to query media fields would be like:
Get access code (from authentication window redirect url)
Get access token (from acces_token endpoint)
Get media data (from graph api by access_token)
I´m using UI5 js framework with UI5 Tooling.
I get a response with data in step 3 from ajax call, but first i have to execute step 1 and step 2 manually.
But i want to do all this authentication-mechanism directly in my app. Including opening this authentication window and acessing the code from the redirect url.
When i do an ajax call in step 1, i get an CORS error of course, because step 1 doesnt respond with the corresponding CORS header (like step 3 does).
Well, anyways i most likely could handle this by a proxy, but whats about the production environment? I dont get how this approach should ever work in a real app. Even when the CORS problems are handled by aproxy in dev environment, it should be still there when a real user uses the app.
So the question is: How can i access or implement this authentication window (on a diffrent origin) in my app? I have seen other apps doing this permission window, but i have no clue how to implement it in a web app.
By clicking "Allow" you get redirected to he redirect_url with the access code
How can i get the access code directly in my app, avoiding CORS in production

I don't have a definite answer, but regarding your CORS issue: It seems like instagram added a CORS policy somewhere last year (see this other question on SO). This means that you would indeed have to build a proxy.
Basically you need something that accepts all the endpoints that the original API accepts, forwards them to instagram, reads the response, and returns the response to the client (aka browser). But your proxy will then not add the CORS headers. Or even better, you add your own CORS headers (assuming your proxy and your app will run on the same server) and no one else will be able to use your proxy from their web app.
Unfortunately I don't know about this authentication window. Maybe this is just a fancy way to hide that they are running a proxy behind the scenes as well? You should be able to see in the network tab where the insta data is coming from after you login. I would guess not directly from their graph API.

Is there any npm library that can avoid getting blocked when web scraping

I'm making and android application that shows some results from Amazon. The app makes a request to a custom Express server that returns amazon results in a JSON via web scrapping (for example, https://www.amazon.com/s?k=nike). My server worked fine, but now it returns empty array. The exact same code works on my laptop, so I supose I'm getting blocked by amazon domain.
I know some things that i could make: change ip, crawl slowler, use proxies, rotate user agents, blablabla...
Is there any npm library that can avoid getting blocked when web scrapping? Maybe selenium?
For your information, i'm using the npm web scrapping module : unofficial-amazon-search

Permitting user agent (chrome) CORS requests programmatically

I need to get some data from external API, from other domain.
I use test app, runned on localhost and I get error:
Origin http://localhost is not allowed by Access-Control-Allow-Origin.
I try to get access to: https://api.coinmarketcap.com/v1/ticker/ by get method
I did a lot of search and:
I know that I can ask that API by using node.js, CURL, and also by running my frontend code in jest tests
By using chrome extension - https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi?hl=en also everything works correctly
But unfortunatelly by simply AJAX request on my page load I can't reach that API and get an error.
And my prediction is that user agent (chrome) somehow block my request.
I can't use chrome extension because I want also access my website from my mobile phone which can't get that desktop extension.
For now my solution is "proxy", node.js server which can access that API and flow looks like:
FE
-- ASK BE -->
BE
-- ASK API -->
-- RESPONSE TO BE -->
BE
-- RESPONSE TO FE -->
FE SHOWS EVERYTHING
FE - Frontend, BE - Backend (node.js proxy), API - external API
But I want to avoid unnecessary code on the BE. Maybe there is some solution?
Like some magical header which will tell chrome to let me pass to external domain without any additional extension?

You are correct you cannot bypass the CORS rules unless you are using jsonp and the server in question supports it. When working with that ticker in the past I had to build a backend exactly as you describe with express.
FE => BE => Ticker => BE => FE

Well you can do that one clean way as described here - https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
Set the header value
Access-Control-Allow-Origin: *
with your response to the first page load from server.

If all you pages uses HTML5,you can try using Window.postMessage.
Another common solution is creating a proxy-server.

Can't make a fetch in IE9 once page is loaded

I have a site that is running in Node and using express for the server. I also have an API on a different server I need to hit. Everything works fine except in IE9. The issue is when ever i try to hit my API once i have the page loaded it breaks.
For example on the home page i have a search form where each step makes an API fetch and once I try to make my first fetch i get this error
"Unhanded promise rejectionError: Access Denied"
It seems like it has to do with a CORS issue since IE9 didnt have CORS and the era of the error message. Also when I make the fetched server side I have no issue, its just when the client tries to perform them.
Has anyone had an issue like this and if so how did you fix it?
Currently I am trying this but it hasnt worked yet:
First I changed my api host address to the current sites address with /api at the end.
apiHost: process.env.API_HOST || 'http://127.0.0.1:3000/api'
Then in my express server is run this:
app.use(modRewrite([`^/api/(.*)$ http://api.mysite.dev:8080/$1 [P]`]));
I was hoping to avoid making a fetch cross address with this but no luck as of yet.
Any help would be much appreciated
thanks!

To solve this problem I changed my host file for my machine to use mysite.dev for the 127.0.0.1 IP. Then went to mysite.dev rather than 127.0.0.1 when testing my site. Then did what I did above to get it to work

Is there a way to bypass Javascript / jQuery's same origin policy for local access?

Trying to use ajax, getJSON, and functions like that to fetch an external URL from a local (non-server) development computer. Is there a way to bypass the same origin policy, so that I can test locally, instead of having to upload to a server?

Here's the simple answer: chrome --disable-web-security
From the source code (chrome_switches.h):
// Don't enforce the same-origin policy. (Used by people testing their sites.)
const char kDisableWebSecurity[] = "disable-web-security";
I wanted to use jquery.js to send AJAX calls to a Google Apps python server running on port 8080. Just for testing, I wanted to run the browser and the server on the same machine.
I don't understand all the security nuances, but for temporary development it seems like a reasonable workaround. So long as I only use chrome for testing with this flag, it shouldn't be a problem.
Here's the whole command for Mac OS X:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --disable-web-security

We had the same need when developing our web app. Here's how we did it:
The browser and the server communicate only through JSON.
All the HTML is rendered in the browser using PURE (our JS template engine).
The browser code is developed locally like this:
We add a host parameter in the url of the app:
http://localhost/app.html?host=test.beebole-apps.com
In production, the JSON are sent to the server with a POST.
But here the function in charge of the ajax call will react to the host parameter and make a JSONP injection(GET) instead.
<script src="http://test.beebole-apps.com/?callback=f2309892&json={...}" />
f2309892 is a temporary function, with a random name, that points to the method that will handle the response
json is the JSON we send to the server
It means you will need some cooperation from the backend to serve you the json wrapped in a callback function like:
f2309892( /*the json here*/ );
Except a size limitation(you can't send a big JSON to the server with a GET) it works like a breeze.
An other advantage is you can call all the different systems(development and test) from the same localhost.

There are different ways to get around this, depending on which browser you're using for development. For example:
In Firefox (Gecko), set security.fileuri.strict_origin_policy to false
In Chrome, start the browser with the option --allow-file-access-from-files
References: Firefox, Chrome

Without touching the server -
The quickest and easiest way to bypass the same origin security policy in Firefox is the install the Force CORS add-on. This works with any service by inserting the proper headers into every response.
https://addons.mozilla.org/en-US/firefox/addon/forcecors/

Since this is a development issue and not a end-user/functionality issue, rather than focusing on getting AJAX to cross domains get your development environment set up as a proxy to fetch the most recent data from the production servers. This is actually really easy to do.
You'd need to set up a web server in your dev environment (if it doesn't have one already), and then configure the server to respond to 404 requests by fetching and then echoing production data. You can set up your server so that only the AJAX data files are picked up (otherwise, it will be confusing to debug other files if production assets start showing up on your development pages). So if http://dev.myserver.com/data/json/mydata.json is missing, your 404 script will get http://prod.myserver.com/data/json/mydata.json and echo it to the client. The nice thing about this set-up is that you can use mock data very easily: if the file is there in your dev environment, your AJAX script will get that; but if you then erase or rename that file, you'll get the production data instead. This feature has been so useful I can't recommend it enough.
If you're working with XML, I'd recommend duplicating the HTTP headers in the 404. If your 404 process responds with a Content-Type of text/html, you won't get any responseXML to parse.

try this (php curl ayax cross domain - by google):
http://www.iacons.net/writing/2007/08/02/ajax-cross-domain-proxy/
http://www.phpfour.com/blog/2008/03/cross-domain-ajax-using-php/
http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html

I had that problem, too, using Chrome and the --allow-file-access-from-files option didn't really help. Back to the script my server needed to return, I added these headers to the response and it worked fine :
'Access-Control-Allow-Origin: http://localhost/'
and another one for allowing a sort of key exchange
'Access-Control-Allow-Headers: X-KEY'

localhost is not allowed to use in CORS http://code.google.com/p/chromium/issues/detail?id=67743 use lvh.me instead