We Keep Coding

JavaScript Get Data From Other Site

I'm fairly new to JS and have a question. Would it be possible to pull data off from another website to use in your own? For example, say I have a JS web app that lets a user input their Twitter username and then the script goes to this username and looks for the follower count element and pulls that number off to display back in the web app. I'm sure there are APIs and such to do something like that specific Twitter example, but I'm getting more at the general idea of being able to access data on other sites. How can it be done? Surely there is a way if my browser can access all of that information, right? Would you have to put an invisible iFrame into the app and search through it with JS?

To put it in basic terms for a newbie, this is only possible if the website in question has an API, specifically designed to allow outside access. Sometimes they're pretty easy to understand and set up.
Accessing the content of an outside website in a way it wasn't really designed to support has happened, but it's usually what's known as "hacking". It's sometimes easy to do for very basic types of sites, but most sites that request login information forbid it. (Except through aforementioned APIs). The biggest concern is that if someone is already logged in to Twitter on their browser, an outside site of questionable origin could automatically post bad things to your account while you're not even apparently visiting Twitter.

Yes, it's possible.
The best approach is to use ajax (Asynchronous JavaScript and XML) with jsonp (Javascript Object Notation with Padding) datatype.
To use this method the server you are going to connect should be expecting the request so it can response with jsonp (it's just a json data wrapped by a callback function you defined when you make the request).
The process:
1) Your code make an ajax request to the other server adding a callback function to the url. It's very commom to use "callback" param name, but the server can define the variable name it prefers.
http://other-server-url.com/?callback=myFunction
A good and easy approach here is to use jQuery. If you define dataType:"jsonp" in your ajax call, jQuery handles the process of appending the callback and execute the right function when you get the response. It's a good idea to take a look at jQuery's ajax docs and read a little about jsonp cross domain requests.
$.ajax({
type: "POST",
dataType:"jsonp",
url: "http://other-server.com/",
data: { name: "John", location: "Boston" }
}).done(function( msg ) { // this function will be executed when you get the response
alert( "Data Saved: " + msg );
});
this jQuery method only works if the param name is callback. If it isn't, you should handle by your own, or take a deeper look at documentation to know how to do it with jQuery.
2) The server should be waiting for these "callback" param and response your request with a jsonp data format. It's just json data format wrapped by the function you passed as the callback. Like this:
myFunction({ json-data });
3) When you get the response, the myFunction function will be executed automatically, with the json data as the parameter:
function myFunction(myData) {
console.log(myData); // this will log the data on the browser console
}
I hope I have helped. Good coding.

How to make cross domain ajax call to get plain HTML content

I have a situation where I need to get data from an HTML file existing in another server. It will contain simply plain text content and nothing else. I don't have access to change anything there nor I can get any structured html. What I have is only text content stored in an HTML which is hosted in http://host1.demoserver.com.
Now, from my application which is hosted in http://host2.demoserver.com, I need to get the content through an ajax call. Problem is without implementing CORS. So, upto to my knowledge level I have an option JSONP. But, when I make ajax call with JSONP, I can see the response in Firebug, but it starts throwing javascript exception. There reason that comes into my mind(may be I am not correct) is that, since the content comes as callback parameter it contains undefined variables.
HTML file: copyright-info.html
This is a simple copyright info which needs to be rendered.
And, if we make a call with ajax call:
$.ajax({
url: "http://host1.server.com/static/copyright-info.html?callback=?",
method: "GET",
dataType: "jsonp",
jsonpCallback: "callback",
callback: function(result) {
//code to excute after success
}
});
Then, response would be like:
callback(This is a simple copyright info which needs to be rendered.)
And, as parameter is not in double quotes its not recognizing it and throwing exception.
2nd, this whole thing was working without JSONP, when it was in localhost. And, I guess, http://host1.demoserver.com will be same localhost as it is for http://host2.demoserver.com. But, when it is deployed in server its failing.
Can anyone suggest what's wrong here? Or, any other way for implementing this.
Thanks in advance.

Ajax read HTML after fully processed

I am trying to load the HTML content of a webpage outside of my domain, which I can do just fine using functionality provided by this jQuery plugin: http://www.ajax-cross-origin.com/. However, when I print out the HTML there are pieces missing, which I assume is because the ajax request gets the HTML before the page is fully loaded. When I say "pieces missing," I mean that some tags that should have innerHTML in fact have none. Here's my code:
$.ajax({
crossOrigin: true,
url: "http://siriusxm.com/bpm",
success: function(data) {
console.log(data);
},
timeout: 5000
});
The crossOrigin attribute is from the plugin I mentioned. I get the same behavior with and without the timeout (and strangely, it doesn't seem as though the timeout is doing anything at all--when I check the console, it logs data pretty much immediately).
Is there a way to wait until the page is fully loaded before getting the content? For what it's worth, this is all part of a chrome extension I'm developing, so if there's anything else code-wise you might need just ask.
Thanks!

So according to your comments, the information you're looking for is just the Now Playing artist and Song, which you won't be able to get by loading just the source of the main page.
To find the data you're looking for just open up your Chrome DevTools, go to the network tab, and Refresh to see all requests on the page.
It looks like this is the request you want, you just need to update the timestamp every minute:
http://www.siriusxm.com/metadata/pdt/en-us/json/channels/thebeat/timestamp/08-12-03:48:00
Just parse that json and grab what you need. Of course they can always change the location or format of the file, but for right now that's what it is.

If the console log is showing all of the data you're looking for then the ajax call should be fine.
Any code in the success callback will be ran after the ajax call, so just use JQuery in the success callback function to insert data into the html. All I see there now is the console.log(data) unless you've removed some code.
The timeout just gives the ajax call a set amount of time to complete before it "times out", in other words it tells it to stop waiting after the set amount of time.

JSONP request callback

Ok, after an whole day trying to get this to work I just don't seem to know what's the deal with this, so here it goes:
I have a WebService hosted locally by now (http://localhost:15021/Service1.svc/[whatever_method]) and an HTML Page on a different file.
FYI, both WebService and HTML Page will get hosted on different servers.
I'm trying to get some info off the WebService to the HTML Page by the onload=load() method in the HTML Page.
My JavaScript code is:
function load() {
alert("Loading...");
$.ajax({
url: 'http://localhost:15021/Service1.svc/getAllNoticias',
type:'GET',
dataType: 'jsonp',
jsonp: 'loadNews_callBack'
});
}
function loadNews_callBack(result){
alert(result.data);
}
Additionally, the JSONP is loaded on the Page with an OK (200) status (As you can see here) which should (I guess) call the callback function, but it isn't.
What can I do to get around this? I already change the request parameters 500 times (e.g., added "?callback=? to the url, with or without jsonp attribute, etc...)
Any help would be great,
Thanks in Advance

Check the actual response text to verify that the response is correctly being wrapped by the callback function. It is possible that your service is not setup correctly to handle JSONP so it simply responding with JSON. It will still come back as 200, but fail to execute.
loadNews_callBack([json...])
vs
[json...]
See this question as reference on how to go about updating your service:
ASP.net MVC returning JSONP

Get jQuery post redirect response

I've written some HTML/Javascript that sits on a third-party server for security reasons. This page performs a javascript post to another page on the same site. However, instead of responding with useful data, it instead wants to perform a redirect (if you would post via a normal HTML form to this page, it would redirect your browser). How can I process this process? I basically want to be able to extract the url's query parameters that it is trying to redirect with (and then put this link into a hidden form field).
Here is my basic ajax post...
$.ajax({
url: '/someurl/idontcontrol',
data: serialized_form_data,
async: false,
type: 'POST',
success: function(data, textStatus, x) {
alert(x);
alert(x.getAllResponseHeaders());
return false;
$('#redirect_link').val(WHAT_DO_I_PUT_HERE);
}
});
Note that the URL I am posting to is not one that I control, so I have no power over what it returns.
UPDATE: When I use the above alerts, I receive "[object XMLHttpRequest]" and "null". I'm monitoring the headers with a Firefox plugin and they seem be coming back as expected, but I can't seem to access them via javascript (I've also tried x.getResponseHeader('Location'), but that and all other calls to getResponseHeader return empty).
ALSO: I don't know if it matters, but the status code is 302 (as opposed to 301 or 303).
Thanks!

According to the jQuery Documentation the success method can take a third argument which is the XMLHttpRequest object.
According to Mozilla's XMLHttpRequest page, this object should have a "status" property. If you check the value of this property, and it is a redirect code, such as 301 (permanent redirect) or 303 (temporary redirect) it is known the processing page is trying to perform a redirect. The "statusText" property should be able to be used to determine the location it is trying to redirect you to.
If you know it is trying to redirect, you can then re-post the data through Ajax to the new URL.
The strange thing is though, when researching this, stumbled across this page that indicates the XMLHttpRequest object should follow redirects (see the comments). So it seems like this would be a non-issue.
Unless the processing page is using an HTML meta redirect (like below) to do the redirection. In that case, I'm not sure what could be done - maybe try to parse the returned HTML for meta tags and see if any of them are attempting a redirect.
<meta http-equiv="refresh" content="0;url=http://www.example.com/some-redirected-page">

You can't get the full HTTP headers from an AJAX call in JQUery, so you can't process the redirect in this way.
However with a raw javascript request you do have access to the XMLHttpRequest getAllResponseHeaders() method which will allow you to process the redirect (this function for single headers).

Sorry, not directly an answer to your question, but I'm sure it's possible with jQuery too as it's quite simple with Prototype.
// Warning: this is Prototype, not jQuery ;-)
//...
onComplete: function(response) {
var refresh = response.getResponseHeader("Refresh");
var whatever = response.getResponseHeader("Whatever");
}
//...