I've tried reading up about this and not really sure where to start, so hoping somebody will be able to point me in the right direction.
Basically I'm trying to use jQuery ajax to read an external website and retrieve a list of links from it. No particular reason as such at the moment, just wanted to see if I could challenge myself by doing it.
In doing so I've read up a little about the Same Origin Policy and understand it (sort of) but could do with some pointers.
Is this possible to do? I've been looking at the scrabble points calculator used by Kate Spanos, for example, and her jQuery code contains some ajax which appears to check dictionary websites and work on some of the output.
Could somebody point me in the right direction, or am I barking up the wrong tree and is it basically impossible to do without some other technical knowledge.
Thanks,
Mat
PS I am a 'noob', so please be as gentle as possible. We all have to start somewhere with this stuff so please don't shoot me down...Thanks in advance.
You should look into JSONP, or more likely use some sort of intermediary, like a PHP script (so same origin) that uses cURL or file_get_contents to access the third party site
for instance:
<?php
$file=file_get_contents('http://some_domain_not_yours.com/somefile');
echo $file;
?>
try referring these , hope it help
jsonp with jquery
http://www.ibm.com/developerworks/library/wa-aj-jsonp1/
http://api.jquery.com/jQuery.getJSON/#jsonp
you should do this via PHP i.e. loading via PHP include the external site and than parse it in your PHP.
You cannot do this via jQuery, basically you can't make a client retrive remote content without a server side to filter it. If a client could access freely remote content you won't have any control on data accesses for the SOP you always need a server in between to guarantee the content management and filtering, this can be either your server or a remote server (like an API provider). To assure this, you can only share JSON objects cross domains, JSON objects are objects created via PHP (for example) so you can't get a JSON object without a server script. The other way (your server is between) is you make a server retriving remote content and then givin' it to your client in any format you like.
Related
so on client side running in the browser I have a javascript code that has a variable (namely a url that is 1500 characters long), and which I need to insert it into a online database that lives on the webserver where I have hosted my website. I have these two technologies on my website, mysql DB and PHP.
Please kindly would someone recommend the best way to do this?
showing examples, specifically, how to send this data over to the remote server and how to process return data it may send back to me??
what i was thinking if there's a way to send over this variable string that is 1500 characters long, over to a PHP file living on my website which this PHP file will be able to insert the data into the DB, and then some time afterwards my same script running on the client browser will check and pull data from the remote DB back to itself...... I've tried to follow along some example searches googling but none of them are making sense to me, sorry I am visual learner , and would greatly appreciate any help you may provide me with this task .....
The solution already discussed here is the proper one. You need an API (also called a service).
I don't know who downvoted it but its the right one.
And you need it for several reasons.
Performance issues. Your solution "writting to a file" will be slow. And even "writting to a file" will require a service on top.
Security reasons. To allow in any other kind of way for a user to write in your server directly (FTP or other methods) is a big security risk and your server might end up being attacked.
Scalability and mantainance.
I would recommend reading more at
https://code.tutsplus.com/tutorials/a-beginners-guide-to-http-and-rest--net-16340
And if you are a bigginer an want to start something fast loopback is an amazing option, but you need NodeJS in your server.
In broad terms what you need to do is set up a API on the PHP side of things. Basically you want a structure where your javascript can send the request, and then, using promises, wait till it gets a response to get the data. That way your PHP server can take however long it needs to put the data in the database and process it properly.
Here's a tutorial on how to make a restful api in php
I am trying to search an outside url for content matching "title" and return the results to my HTML page in the background through Javascript. I have been using Javascript and not found any resources that resolve my query, maybe I'm asking wrong?
but I would basically search the document with :
var title = document.getElementsByName("title");
The hard part is connecting to the page and searching through the HTML source code.
TIA!
You can't generally get the content from an outside URL unless server specifically allows you to do so. But, you can do it from server side. You will be able to get the content of any URL from your server. Server must include an header in response with name access-control-allow-origin which contains patterns/name of your domain.
However, you can do it from server side anyway, unless you are blocked specifically by the server.
You will need to develop a solution in which you grab the content for your outside URL from your server. It can be anything like PHP, Node.js, C# etc. After receiving response from the external server, deliver it in response to the browser using AJAX or anything. Then you can play with it anyway you want using JavaScript or JQuery.
Important Note:
Make sure whatever you are trying to access in anyway, you are allowed to do so. If they (your outside URL) wants to share something with public, they must be providing some APIs or other solutions to allow you access to their content.
Research has led to to a solution, implementing a scraper. There are many in existence,scrapy for instance. Just a head's up for those with the same question.
No matter how much I look this up all I get is the w3C File API which focuses on local files for the client.
What I'm trying to do is I have a server. I'm trying to use client-side javascript to grab the server hosted text file, a.txt, and display it to the innerDOM of an html page. My server directory look like this:
index.html
read.js
text files
a.txt
All I want to have happen is for, on the client side, the javascript read.js running in the index.html on onload to display the contents of a.txt. I figure that since a.txt will never be large, leaving it to the client is fine.
But I can't figure out how to do this and the W3C File API isn't offering me answers.
If I had to guess, somehow making sure index.html loads a.txt and then grabbing that via the file API might be the way to go but I'm not sure how to do that.
And I'll admit it, I'm a bit of a noob. If I'm invalidating browser sandbox or doing something impossible, please tell me. I just thought this would be so simple.
Also, I'd appreciate that if you were going to suggest AJAX, either don't, or explain it like I'm a baby because I really don't know.
Thank you all so much for your help.
Why file API is irrelevant:
Web applications should have the ability to manipulate as wide as possible a range of user input, including files that a user may wish to upload to a remote server or manipulate inside a rich web application.
From W3C File API.
So, File API is intended to be used to allow users to upload files from their clients into the server. On the other hand, AJAX is used to allow users to download files and other data from the server into their clients. And this is exactly what you need.
Refer to jQuery's ajax documentation.
I believe this page should help you out with your problem.
http://www.htmlgoodies.com/beyond/javascript/read-text-files-using-the-javascript-filereader.html#fbid=YhNukIHynD3
I would suggest using an Ajax call to the file on the server, since the response of the call will typically be the contents of that file.
Using Jquery this can be done by a simple
$.ajax({ 'url':'a.txt',
'success': function(r){
//display to innerDOM here, using r as the file
});
});
You simply want to display a txt file on the web page?
Do you know about server side includes?
That would be one possibility if you control the server.
If you really want to do it in javascript, then AJAX would be the way to go.
If it were me at that point I would figure out how to include and use jQuery to help with the ajax bits.
You will simply request the text file via its URL (you can get it to load in the browser right?), and then use jQuery to put that text into some DOM element.
I'm trying to build a very simple bit of javascript that reads and displays a couple stock indexes when a webpage is loaded.
I was hoping to find an RSS feed with this data that I could then parse with jQuery.parseXML, but I couldn't track one down. What I did find is this: Yahoo Finance provides a way to download stock data in CSV format, specifying which data you're after via the URL.
So, I'm thinking this might be a way to accmplish what I'm after: when the page is loaded, I could send a request to Yahoo Finance, and then somehow parse the CSV data to get the data I need to populate my stock quote. My question relates to the aforementioned "somehow." Is there a way to do this via javascript? Is it possible to, for example, somehow load the CSV generated by Yahoo Finance as a string?
I'm also very open to any other suggestions of how to accomplish this. If anyone, for example, knows of an RSS feed from which I could get the S&P/TSX Composite index, please let me know!
You'll probably run into some cross site scripting problems as the browser will not let you do that. See the howto on that about avoiding that. You could also do it on the server side and then query that from you client. Depends on the server side technology you are using.
After that parsing the CSV shouldn't be a problem. Use something like string.split on each line.
JavaScript is by default not allowed to cross-domain requests unless you use JSON-P as your format, requesting CSV directly from another domain will not be allowed. Therefor this is a bit problematic. In this case you will probably have to setup a proxy within your own domain that will fetch the data from Yahoo server-side, and send it to your JavaScript from within your own domain.
I want to fetch particular HTML contents from remote websites url.
The website URL is as follow,
http://www.realtor.com/realestateandhomes-detail/10216-Montwood-Drive_El-Paso_TX_79925_M78337-06548
I want to fetch some specific information from above website url.
Here I attached image it highlight the specific area I want to all highlighted portion from there is a title,image, and descriptions.
How can I fetch the contents using JQuery or Javascript or Json call?
Is any other way to get these?
You might be interested in checking out pjscrape (disclaimer: this is my project). It's a command-line tool using PhantomJS to allow scraping using JavaScript and jQuery in a full browser context.
Scrapers can be written in straight Javascript, executed in the context of the site you're scraping, with a very simple, jQuery-friendly syntax.
It can scrape a single page, an array of pages, or you can define a function to look for more URLs to spider on each page.
It supports JSON and CSV output, either to file or to STDOUT
If the site is static and the structure is uniform, it should be very fast to scrape all the content you need into a structured data format.
This will help you out:
http://papermashup.com/use-jquery-and-php-to-scrape-page-content/
When scraping content, it is vital to consider the following:
Is the content static html or will part of it's content be rendered by ajax-calls?
In the first case, simple http-get-routines like the one used in JNDPNT's comment's Link will be sufficient.
In the second case, you may want to look at automating Selenium via it's Webdriver.
In any case it might be better to ask your colleague if he can provide you with an interface to the raw data, e.g. over a webservice.
If I'm getting you right, you want The user's Browser to scrape The content of another Domain on The Fly, right?
That will Not Be Possible without proxying The Request through some Script on The Same Domain (or via a jsonp Request to a Service that returns The HTML to you) due to The Same Origin Policy.
Sorry to disappoint.
Use the Yahoo Pipes (http://pipes.yahoo.com/pipes/ )service.
This can be used to grab and manipulate the page HTML, extracting the bits you want. Data can then be posted server side using the Web Service module or sent directly to the clients browser using an ordinary javascript callback.