Download a file using python from a webpage without opening the webpage - javascript

I was looking for a way to write a script that will download a file from a specific website but without opening the website itself. I want everything to happen in the background.
The website is morningstar and a specific link for the example is this one:
https://financials.morningstar.com/ratios/r.html?t=MSFT
in this page, there is a "Button" (it is not really declared as a button but as a Hyperlink, the <a> tag in HTML)
I added a photo in the bottom so you can see for yourself exactly the way they wrote the code.
Anyway, I saw that when I clicked the button the href attribute actually calls a javascript function which then creates the links from which the file will be downloaded.
I am looking for a way that I can write a script and give it the link I want, for example, the link above, and the script will download this specific CSV file from that page into a folder of my choice.
I was looking at some selenium tutorials but I couldn't find much help for my specific problem.

Here's an example I use:
import requests
url = 'http://via.placeholder.com/350x150'
dashboardFile = requests.get(url, allow_redirects=True)
open('d:/dev/projects/new-wave/dashboard.pdf', 'wb').write(dashboardFile .content)
Oh, and depending on the size of the file you would want to download in chunks. A quick search of: "python download large file in chunks" will help.

Related

How to make file/result downloadable in html

I will request you see this repo: https://github.com/imsikka/ArtGallery
I want to make the result downloadable using a download button, I know how to make a download button but I don't know how it will make the file downloadable, I need to link the button with a specific part code or what?
Well I really don't know how to do it i surfed the net but got nothin really helpful
You can upload the information you want to download on an uploader site and apply the download link to the button using
Maybe this way you can solve your problem
Info: Conclusion. In HTML, a button link to another page can be by using the tag, tag, and the tag. A link on a button is get by href=”” attribute of tag. The “type=button” and “onclick=link” attributes are used to create a link on the button.

Download PDF files Selenium Python

I need some help. I need to download financial statements that are located in PDF files to my MAC using Python. I've tried this using Selenium but unfortunately haven't been able to make progress.
Following is the URL: financials.psx.com.pk
When I load this URL, the page looks like so:
you see the dropdown on the left above "Company Name...", I need to select a company name there. Once I do that, then I see a window like this:
Every link under reports, when clicked downloads a PDF. I require those PDFs. Appreciate any help. Thanks.
I have tried the following code; it selects the company name and shows me the next page for a second but somehow another refresh happens, which leads me back to the original page:
driver = webdriver.Chrome(executable_path="driver-path")
driver.get("https://financials.psx.com.pk/")
elem = driver.find_element_by_id("country")
drp = Select(elem)
drp.select_by_value("SHEL")
Actually, when I select using select_by_value, I can see the second window. I think the code is not stable. But, I am able to select a symbol from the dropdown and see the second page. Now, I need href's from the second page behind links such as 'Quarterly', 'Annual' etc.

Loading .txt content in HTML file? If not, then where?

Very new to code. Searching for 2 days has yielded no results and I've gone nearly mad.
I have this CSS designed bar that I use in my live streaming overlays. I'm trying to populate text from a local .txt, xml, or json file onto the bar. I just cant figure out how to call this text and then style it.
Why? I'd like to eventually write a python script/GUI that can read / write to the .txt file (if its json or xml, ill try to figure it out). Baby steps.
So far, everything is div boxes and a video that is clipped to create the black bar (its looped to look animated).
Using
embed src="txtfile.txt"
loads up the file, but I cant style it T_T. I would really appreciate the help!
Main question. With this file that I am opening locally - how can I (on the backend) select a text file to be read, and then style that text?
I really appreciate the help or even a nudge in the right direction. I am so lost.
You will probably need to use a server language. I recommend using PHP as you are new to code. First you will need to set up a PHP supported development server for developing. You can search how to do this yourself as its out of the scope of your question (WAMP/MAMP/XAMPP).
If you are using PHP as your server language. presuming txtfile.txt is located in the same directory as your CSS designed bar, you can do the following:
<?php
// specify txt file path
include 'txtfile.txt';
// store txt file contents as string
$my_string = file_get_contents('txtfile.txt');
?>
Then wherever you want the string to appear, just echo the variable. Here's an example:
<span class="myCustomClass"><?php echo $my_string; ?></span>
You can then style whatever class you gave it in CSS.
Oh, and remember to change the file extension to .php
You can also use other server languages to achieve this.
Alternatively, you could embed the txt file in an iframe, and place the iframe inside the element you want:
<iframe src='txtfile.txt' scrolling='no' frameborder='0'></iframe>
...and there are other ways, too! Look into using jQuery and AJAX as it may be easier to do without the hassle of using PHP. You can just load the text file via an AJAX call, insert it inside the document and then you can style the content with CSS.

Is it possible to convert a dynamic HTML page with a lot of javascript to a page without javascript?

I have a page with a lots of javascript. However, the page once rendered remains static, there are no moving things or special effects, etc... It should be possible to render the same HTML without any javascript at all using only the plain HTML and CSS. This is exactly what I want - I would like to get a no javascript version of the particular page. Surely, I do not expect any dynamic behavior, so I am OK if buttons are dead, for example. I just want them rendered.
Now, I do not want an image. It needs to be an HTML with CSS, may be embedded with the HTML, which is fine too.
How can I do it?
EDIT
I am sorry, but I must have not been clear. My web site works with javascript and will not work without it. I do not want to check if it works without, I know it will not and I really do not care about it. This is not what I am asking. I am asking about a specific page, which I want to grab as pure HTML + CSS. The fact that its dynamic nature is lost is of no importance.
EDIT2
There is a suggestion to gram the HTML from the DOM inspector. This is what I did the first thing - in Chrome development utils copied as HTML the root html element and saved it to a file. Of course, this does not work, because it continues to reference the CSS files on the web. I guess I should have mentioned that I want it to work from the file system.
Next was to save the page as complete with all the environment using some kind of the Save menu (browser dependent). It saves the page and all the related files forming a closure, which can be open from the file system. But the html has to be manually cleaned up of all the javascript - tedious and error prone.
EDIT3
I seem to keep forgetting things. Images should be preserved, of course.
I have to do a similar task on a semi-regular basis. As yet I haven't found an automated method, but here's my workflow:
Open the page in Google Chrome (I imagine FireFox also has the relevant tools);
"Save Page As" (complete page), rename the html page to something nicer, delete any .js scripts which got downloaded, move everything into a single folder;
On the original page, open the Elements tab (DOM inspector), find and delete any tags which I know cause problems (Facebook "like" buttons for example) (I also try to delete script tags at this stage because it's easier) and copy as HTML (right-click the <html> tag. Paste this into (replace) the downloaded HTML file (remember to keep the DOCTYPE which doesn't get copied;
Search all HTML files for any remaining script sections and delete (also delete any noscript content), and search for on (that's with a space at the start but StackOverflow won't render it) to remove handlers (onload, onclick, etc);
Search for images (src=, url(), find common patterns in image filenames and use regular expressions to replace them globally. So for example src="/images/myimage.png" => |/images/||. This needs to be applied to all HTML and CSS files. Also make sure the CSS files have the correct path (href). While doing this I usually replace all href (links) with #;
Finally open the converted page in a browser (actually I tend to do this early on so that I can see if any change I make causes it to break), use the Console tab to check for 404 errors (images that didn't get downloaded or had a different name) and the Network tab to check if anything is still being loaded from the online version;
For any files which didn't get downloaded I go back to the original page and use the Resources tab to find them and download manually;
(Optional) Cull any content which isn't needed (tracker images/iframes, unused CSS, etc).
It's a big job. I'd love a tool which automated all that, but so far I haven't found one. The pages I download are quite badly made (shops) which have a lot of unusual code, so that's why there are so many steps. You might not need to follow every step.

how can i Grab Source code & save it in text file

I need your help to grab the source code of the page & can save it in a text file.
What actually i want!!
Instead of doing the lot of work like, right click on the page then click on view source code then to copy & paste it in a text file...
I want to make a short, i want to put a link on the page, in which i need to grab a source code, so when i just click on that link let say "Download Code" it grabs the current page source code & save it in a .txt format.
Kindly help me how can i do this?? it would be great if i can achieve this by using a java script, i don't want to use server side programming language.
I tried data URI but dont get the exact what i need
I find some thing on the below link:
http://css-tricks.com/examples/ViewSourceButton/#

Categories