Using data from webpages to make a calculation - javascript

I would like to extract specific data from a webpage then open a specific webpage based on what that data is. In the form of an if statement.
Say for example I google the olympics (https://www.google.com/#safe=off&hl=en&q=olympics&btnK=Google+Search)
It says it is in "Rio" right now. Say based on "Rio" I have a link that directs to Wikipedia and if it said something other than "Rio" it directs to www.google.com. I need to find a way to be able to pinpoint the data I need from another webpage then make a decision based off of that. The hard part being assigning that data to a variable.
I would like to do this in jQuery if at all possible. If not what language what I need to do this? Is there a specific name for this type of thing (pulling data off of webpages) and are there any resources so I can learn more about it?
Thank you

I think its what you are looking for. Correct me if i'm wrong
You can use the PHP method on a file called go.php
<?echo get_file_contents($_GET['a']); ?>
and call it from AJAX using:
$.get("go.php?a=https://google.com", function(data){
//Your code....
});
IMPORTANT: DON'T FORGET THE HTTP OR HTTPS, if you forget it, go.php will try to find google.com in your server.
This method doesn't load the images because it tries to load them from your server

Related

dynamically generate content for a page when clicking on product

everyone. I am making a website with t-shirts. I dynamically generate preview cards for products using a JSON file but I also need to generate content for an HTML file when clicking on the card. So, when I click on it, a new HTML page opens like product.html?product_id=id. I do not understand how to check for id or this part ?prodcut_id=id, and based on id it generates content for the page. Can anyone please link some guides or good solutions, I don't understand anything :(.
It sounds like you want the user's browser to ask the server to load a particular page based on the value of a variable called product_id.
The way a browser talks to a server is an HTTP Request, about which you can learn all the basics on javascipt.info and/or MDN.
The ?product_id=id is called the 'query' part of the URL, about which you can learn more on MDN and Wikipedia.
A request that gets a page with this kind of URL from the server is usually a GET request, which is simpler and requires less security than the more common and versatile POST request type.
You may notice some of the resources talking about AJAX requests (which are used to update part of the current page without reloading the whole thing), but you won't need to worry about this since you're just trying to have the browser navigate to a new page.
Your server needs to have some code to handle any such requests, basically saying:
"If anybody sends an HTTP GET request here, look at the value of the product_id variable and compare it to my available HTML files. If there's a match, send a response with the matching file, and if there's no match, send a page that says 'Error 404'."
That's the quick overview anyway. The resources will tell you much more about the details.
There are some solutions, how you can get the parameters from the url:
Get ID from URL with jQuery
It would also makes sense to understand what is a REST Api and how to build a own one, because i think you dont have a backend at the moment.
Here some refs:
https://www.conceptatech.com/blog/difference-front-end-back-end-development
https://www.tutorialspoint.com/nodejs/nodejs_restful_api.htm

Using DOMXpath to extract JSON data

I have used php simple html dom to no success on this issue.
Now I have gone to DOMDocument and DOMXpath and this does seem promising.
Here is my issue:
I am trying to scrape data from a page which is loaded via a web service request after the page initially shows. It is only milliseconds but because of this, normal scraping shows a template value as opposed to the actual data.
I have found the endpoint url using chrome developer network settings. So if I enter that url into the browser address bar the data displays nicely in JSON format. All Good.
My problem arises because any time the site is re-visited or the page refreshed, the suffix of the endpoint url is randomly-generated so I can't hard-code this url into my php file. For example the end of the url is "?=253648592" on first visit but on refresh it could be "?=375482910". The base of the url is static.
Without getting into headless browsers (I tried and MY head hurts!) is there a way to have Xpath find this random url when the page loads?
Sorry for being so long-winded but I wanted to explain as best I could.
It's probably much easier and faster to just use a regex if you only need one item/value from the HTML. I would like to give an example but therefor I would need a more extended snippet of how the HTML looks like that contains the endpoint that you want to fetch.
Is it possible to give a snippet of the HTML that contains the endpoint?

How do I get user specific pages using javascript only?

I am creating a website where each user will have their uniq page. users can visit other user's pages by
http://website/user?user=<username>&session=<session>
Now I want to simplify above URL to
http://website/user/<username> (something like pinterest or facebook)
I thought I can use mod_rewrite. However, mod_rewrite is for server side. I do not want to include any PHP code. What I do to get data for a user :
load the basic HTML template and then based on which user we are talking about, load user's data asynchronously.
Can I achieve above in JS? If yes, how?
-Ajay
Unfortunately, you can't do exactly this.
But possible solution would be to place your HTML hub page to http://website/user/ and form user URLs like this: http://website/user/#username. JS can get the user name simply by var username = location.href.split("#")[1].
By the way, you said that you are not using PHP. How do you parse URL arguments then?

Javascript: get data displayed from another url

I was wondering if I could get some data from another website to get it displayed on mine. The good example can be alexa.com. I need to display Alexa traffic rank and reputation in a div for example on my page, so it will be changed dynamically each time Alexa change its data.
Thank you for your help.
One way is to make an ajax request for the Alexa.com site, once you receive all the html, then you can use jquery or something to scrape it for the div you want.
It feels kinda dirty, but its an easy way to get what you want. Though this is assuming their page content isn't loaded dynamically.
Edit: See this for more info: Request external website data using jQuery ajax
yahoo yql... (instead of a php? proxy serverside script)..
I have a sneaky suspicion you do not own/control the external link site, so getting content from a different site, would fall under cross-domain security restrictions (to a modern browser).
So in order to regain 'power to the user', just use http://query.yahooapis.com/.
jQuery would not be strictly needed.
EXAMPLE 1:
Using the SQL-like command:
select * from html
where url="http://stackoverflow.com"
and xpath='//div/h3/a'
The following link will scrape SO for the newest questions (bypassing cross-domain security bull$#!7):
http://query.yahooapis.com/v1/public/yql?q=select%20title%20from%20html%20where%20url%3D%22http%3A%2F%2Fstackoverflow.com%22%20and%0A%20%20%20%20%20%20xpath%3D%27%2F%2Fdiv%2Fh3%2Fa%27%0A%20%20%20%20&format=json&callback=cbfunc
As you can see this will return a JSON array (one can also choose xml) and calling the callback-function: cbfunc.
Indeed, as a 'bonus' you also save a kitten every time you did not need to regex data out of 'tag-soup'.
Do you hear your little mad scientist inside yourself starting to giggle?
Then see this answer for more info (and don't forget it's comments for more examples).
Good Luck!

With JS, jQuery, how do I save an AJAX response to a (text) file?

It seems like this question is asked periodically and the common response is "You shouldn't do that with AJAX anyway. Just set the window location to the file."
But I'm trying to request a file that doesn't actually exist out on the server anywhere. It's dynamically generated (by a Django view) given the GET/POST context parameters. The file I want to retrieve via AJAX, and then save to the client machine, is a text file (csv).
I can currently get the text to the client machine (and can verify this by seeing it in logging or an alert) but cannot then figure out how to save this text to a file inside of the AJAX success callback fn.
Essentially, is this possible, is it something JS can do? That is, to open file save dialogs for "files" that are actually AJAX response text?
From the browser's point of view, it doesn't matter if the file exists or not, it's just a resource on a server that it's requesting. I think you're going to need to do some version of "Just set the window location to the file". If you set the content type in the header to something that the browser doesn't recognize, I believe it will ask the user if they want to save it.
As others mentioned, you can't do it only with JavaScript.
IMO the best option would be the Flash 10+ FileReference API.
There are some good JavaScript wrapper libraries like Downloadify that provide a JavaScript API to access those methods.
Give a look to this demo.
This isn't something JavaScript (and therefore jQuery or anything other JS framework) is allowed to do, for security reasons. You may be able to do what you want to flash or another route, but not JavaScript. Bear in mind Flash has it's own slew of security restrictions for this as well.
(Yes, IE can do this via an ActiveX object, but I'm not counting that as a "solution" here)
Basically, no. Javascript cant save anything to the local machine due to security restrictions. Your best bet may be to have a signed applet that the user can trust to write the file, or put it in a textarea that they can then easily copy and paste into a new file.
Could you not use the PHP rename() function for this, instead of just Javascript? Call to a PHP file and pass the name of the file you want to copy along with where as parameters?
I have the same problem. You can try this
<button id="Save">Save</button>
<img src="MakeThumbnail.ashx?Image=1.jpg" id="imgCrop">
$("#Save").click(function (e) {
url = $("#imgCrop").attr("src")+"&Action=Save"
e.preventDefault(); //stop the browser from following
window.location.href = url;
});

Categories