Google Apps Script - How can i get the content of a page? - javascript

When i try to use the XMLHttpRequest() function in a google script, i get an error telling me the element is undefined.
Is there another way to get the content of a page using google script.
Any help would be much appreciated.

This is limited to avoid the ability to redirect people visiting a publicly (and trusted) document which would fetch data accessible on your browser (using your login sessions and etc).
Make sure you set the CORS headers as specified in the W3 specification to allow external services to query your request handler.
See the page here for more limitations: https://developers.google.com/apps-script/guides/html/restrictions#xmlhttprequests_xhr

Related

How can i get DOM of specific page like "google.com"?

for this problem I need to say I use fetch method but some web, like google and ..
does not allow it.
I want to get DOM of a page for tracking it, for example : I wanna check if an element change let me know.
How can I get an specific web page DOM without exception, like: google in using fetch method?
If you have another solutions, may you tell me (With Java script)?
TLDR; You don't
You can insert a page via <iframe>, but it won't (for security reasons) let you access any of its data, thus nor the DOM.
You can't fetch the HTML of google.com either, it is because Google doesn't want to give you the HTML... You can, however, create a proxy server, because a nodeJS server can fetches the HTML of google.com. But then it will be simple HTML yet to be parsed...

Adding html to page

I was wondering if there's a way to add html to a webpage (from another domain) before displaying it in a iframe. Basically my idea is to make a website that allows the user to type in a website and choose or add things to the page, like copy a yahoo anwsers page and still have it function (so javascript and all still works) but the background has changed or a button that does a specific thing to the page has been added in. So far I keep running into cross domain policies and I have no idea how to get round these.
No, you cannot, this will be a massive violation of information security codes.
Imagine if someone could access your code, js and html and just alter it and access your information.
even worse, if you have sensitive information stored in the client code (you shouldn't but lets imagine), everyone across the web will have access to that information.
Displaying the webpage is one thing, you have api's for this sort of things (for example, google, twitter, facebook api's) and you pay for them too.
The reason you are running into cross domain policies is because you are not allowed to make that request. Not by JsonP and not by CORS Requests.
If you want access, look for a proper api.
For beginners, here is the Yahoo Api (Yql)
And here is it's Terms of use
Read them before you continue.
You can't do that because, if you could, an attacker could use this to exploit referred page.
Source: https://developer.mozilla.org/pt-BR/docs/Glossary/Cross-site_scripting

Google API: Authorized JavaScript Origins

I'm implementing a Google+ Sign-In for our web service, and stumbled on "Authorized JavaScript Origins". Our clients have web addresses either as a sub-domain of our main domain, or as a custom domain name. Since the login page is under that sub-domain (or custom domain), and in order to make the Google+ Sing-In button work, that custom domain/sub-domain should be (manually) entered in the "Authorized JavaScript Origins" list (with both http and https).
Does anybody know a way to do that automatically (through some API maybe)?
If not, then how do you do it?
Not sure if there is an API for this. At first glance I don't see one. The alternative (aside from manually adding domains all the time) is to use a hidden iframe on each site - this iframe would come from your domain and would be the only thing that calls google services. The main sites would communicate with the iframe (postMessage) to tell it what to send google. This of course, opens up a security risk (anybody could load your iframe into their page and do bad things on your behalf) so you'll want to make sure that the iframe code refuses to do anything unless it's running within a page on a known-good domain.
You can also have a common URL which all subdomains point to when trying to log in with Google. Then have this URL redirect to your actual Google login path. Beats having to deal with an iframe this way.
Finally I made it to work, however there may be some fixes to apply.
So a server is host for many domain and subdomains (childs) which all of them needs google sign-in and there is a main domain (parent).
I implemented a general login page on parent which childs open this page via window.open() as popup. As client is in a popup, it is very likely that auth2 cannot open another popup, so the parent will do the google auth with {ux_mode: 'redirect'} parameter as gapi.auth2.SignInOptions.
Process will continue to your callback page which you provided as another gapi.auth2.SignInOptions parameter which is redirect_uri and is on parent.
On this page google may have provided you the golden id_token which you must authenticate this token on your server. And this was the main twist which you should use this information to create a token on your server which parent asked server to create, but send it to child on client side (for example via query parameter) to use it for later usage.
I will happily take any advice for security leaks or any comment which may ease the process just a little.

What's the best fallback if Google Ajax API is not available (since Google does not permit to install it locally)?

In my site I have:
...
<script type="text/javascript" src="https://www.google.com/jsapi"></script>
...
The script above is the Google script to load up other resources dynamically.
(eg Google charts API)
This works 99.99% of the time.
However, I just got a client that for some reasons got his company restricting access to google.com.
As a consequence of this my website simply threw a JavaScript error.
Now I know how to handle that, and I can check if window.Google exists.
but my question is
"what's the standard way to deal with this? "
In other words if you embed 3rd party JavaScript how best do you deal with their JS not available?
NOTE: VERY IMPORTANT
You can not host the chart code locally or on an intranet.
SEE FAQ from Google: https://developers.google.com/chart/interactive/faq#localdownload
Can I download and host the chart code locally, or on an intranet?
Sorry; our terms of service do not allow you to download and save or
host the Google.load or Google.visualization code.
There is no real alternative. Due to Google's terms of service you cannot use Google API without access to google.com.
Check the connection to Google and iform user that function is not available
Develop your own or use non-google api. Still you can use Google if available
The solution is that your client's company review their content filtering policies. Google are quite clear in their previous answer concerning offline access:
…your computer must have live access to http://www.google.com/jsapi in order to use charts.
You are using a third-party solution according to their terms and conditions, which naturally imposes limits on how that solution may be used by your clients. You need to stand firm or find a more liberally-licensed solution. (At any rate, you are more likely to succeed at convincing your client's IT department than petitioning Google to change their TOS.)
For the more general case of third party JS APIs that may not load but for which you are allowed to keep a local copy on your server, see this question.
You can try it like this:
Instead of using the direct link to the Google libraries you want to use, use a link which points to your server:
<script type="text/javascript" src="https://www.myserver.com/jsapi"></script>
When your server gets an incoming request to this URL, your server now makes a request to Google to get the API and sends the response to the client.
That means you do not install the API anywhere locally or on a server and always get the most actual version directly from Google. People also do not need access to Google (as in the company you mentioned) and therefore can use your service.
Use Firebug or the Chrome Dev Tools to inspect your HTML source once the charts scripts are loaded. Access the scripts in your browser and save them locally, then serve them from your own server. This isn't recommended, of course, but if you don't have any other choice...
For example, checking the code of one of the pages I use it on, the core script for the Google Charts library is located at:
https://www.google.com/uds/api/visualization/1.0/3d781368978b51b3ca00a01566dccf40/format+en,default,corechart.I.js
Use the javascript window.onload to check whether the api has loaded or not, if no then load it from your server.
You already know how to check whether or not your library has been loaded (checking the object), if it fails, than what you can do within giving constrains:
Keep checking the object with timer and trying to download library, displaying message for a user
In case first one fails, you have two ways again:
Stopping your application and displaying an error: "Application error... try later"
Or downloading different library as a fallback
Are you progressively enhancing or gracefully degrading the page? If so, what do you display to users without JavaScript for this chart? A table? A list? This is what you should leave in the page and only start changing it once google's JS is available. Either that, or find an alternative library like raphaeljs that lets you keep all your code within your project.
IF (BIG IF) you are not worried about the interactivity the Google Charts and want to display them to the user just to see - maybe add your own javascript to it but not depend on the Google Javascript at all, this can turn the google charts into a image that you can display to the user.
Also this requires access to install a command line tool on the server.
http://code.google.com/p/wkhtmltopdf/ is a command line tool that will generate an image from an html page. If you build a simple page that only shows the chart you want and point the wkhtmltoimage tool at the local html file it will load the Google Charts javascript and generate the chart then generate an image out of the results.
YES I understand this is VERY kludgy and is adding a big tool for a small problem but with the browser restriction and the Google Terms of Service this will solve most of the problem.
You can try going straight to google and if it fails (if google is restricted) you can bounce the request off of your server which forwards the request using CURL to google. If that doesn't work then Google is most likely down. This should cover the issue that you described in your question, but there isn't really a fix for if google itself actually goes down. It should, however, give your application access around domain restrictions because the request will be routed to your server rather than straight to google. I use this architecture for all requests so that I don't have ajax requests routed to random servers. It allows me to control what interacts with my front end using my backend. There are other benefits to this, especially if you are using something like AngularJS with NodeJS because you can decouple a lot of your third party libraries. This however, is beyond the scope of your question!
Basically, it works like this (pseudo code):
If(!Browser->Google->Browser){
return Browser->MyServer->Google->MyServer->Browser;
}
An answer has been accepted already, but still I would like to leave an additional aspect elaborating on the comment I made above ....
It has been accepted that the Google Server is the only place from where the API can be loaded. We don't know whether the client's IT manager will re-think their content policy, they might have good reasons for that.
Given a non-100% availability of all the components along the path between a user browser and the Google API, sooner or later a user will end up in an error situation; statistically this is unavoidable.
What is not acceptable (and avoidable) for a user is to receive an "unspecific" JS error making him/her believe there's a bug on the page. So my solution would be to trap the failure loading the Google API and display a message "Third party components temporarily unavailable - Please try later".
This will demonstrate to the user that
we know what's going on
there's nothing we can do about it now
but it's not totally unexpected and still somehow under control

Facebook Javascript SDK Security. How do Facebook verify that the JS SDK is loaded in the right domain that specified in the app settings

There are few questions with same name but none of them intended to ask what I have in my mind. So we initialize FB js sdk with only app id. It's easy to know other web site's app id by looking at their facebook initialization source code. One might think that it's possible that a hacker might try to initialize FB JS SDK with other's app id and try to get their user access tokens. But facebook doesn't allow such stuff. You have to load js sdk from the same domain you specified in the site url property in Facebook Developer Apps page. So the question is how do they know that the jsonp whatever calls coming from the right client? It's not safe to that checking in client side since people can copy and modifty the javascript as they wish. So it has to be server side checking. I can only think of "referer checking" but I feel it cannot be considered a safe way.
Well, I'm not sure so this is only speculation..
First of all, when making an http request the HTTP referer header is added, and so when you load the sdk the url from which you're making the request is added as a referer.
Facebook can check on their servers where the request was originated from and compare that to what they have for the app settings.
It's possible of course to modify this header when making the request, which is why you don't get any error just by loading the sdk for an app if you're in the wrong domain.
The error will only occur when you try to interact with the sdk, for example trying to execute the FB.login method will open the auth dialog pop-up which will show the following error message:
An error occurred. Please try again later.
If you check the url of this auth dialog (which the sdk constructs) you'll notice these two query string parameters:
domain=THE_DOMAIN_OF_THE_PAGE
redirect_uri=FACEBOOK_URL which will contain origin, domain and relation=opener
What (probably) happens is that facebook checks the domain against the app settings, if it's ok it presents the user with the auth dialog, when he finished the process he is redirected to the redirect_uri.
Since the redirect_uri opens in the pop-up it can only communicate with it's opener if they are both in the same domain, a facebook domain which no one can have on his page other than pages served from facebook.
When the sdk loads it adds an iframe into the fb-root container which loads a facebook js which is loaded from the same domain as the redirect_uri, because of that the pop-up window can communicate back with the iframe and inform it with the auth response.
After the iframe got the response, the pop-up closes and the iframe informs the loaded sdk in the main page of the response.
I'm not sure which technique they use for that communication, but you can easy find more info about that by googling "cross domain iframe communication".
That's how I see it, but I can't be sure.
You can check the code for the js sdk # github if you want to really know what's going on.

Categories