Using Tampermonkey to automate download of csv file on webpage [duplicate] - javascript

This question already has answers here:
Save images to hard disk WITHOUT prompt?
(6 answers)
Closed 3 years ago.
I'm a complete newbie to this, but I'm trying to put together some code that will automatically download a .csv file from a webpage every hour and save it to my computer (preferably a specific place). The data is hourly rainfall data from Scotland at the following website.
http://apps.sepa.org.uk/rainfall
Basically I need to be able to select one of the sites in the map on the left (it'll be the same site each time) and download the hourly rainfall .csv file from the dropdown menu on the upper right hand side of the graph. When doing it manually you hover over the download button, then hover over "Download as..." then select "CSV" when it comes up.
From what I've read on here, the Chrome Tampermonkey extension used in conjunction with the Windows Task Scheduler might be the best way to do this. From the previous question there is the code:
document.getElementById("download_button").click();
I've tried to replace the "download_button" section with something that might describe the ID of the CSV button element on the page, such as using 'CSV'; but I'm really not sure what the element ID of the button is. I've tried inspecting the element in the source code but as I'm new to this I don't really know what I'm looking for.
I also tried to search for the element by class and by path but again I was really unsure as to what value for the class and path to put in to get to the specific download button I required.
document.getElementByClass("export-main").click();
document.getElementByPath("//*[#id=\"chartdiv\"]/div/div[2]/ul/li/ul/li[1]/ul/li[3]/a/span").click();
Any help would be greatly appreciated, I'm not even sure if it's possible or if this is the best way to go about it.
Thank in advance for any help you might be able to give me.

Just as a proof of concept, better approach would be to aim at specific http GET request for the data (location) that you need. That way you get JSON formatted response that you asked for (can be converted to CSV). Google how to open Chrome DevTools (F12), you will find request that you need on Network tab (bunch of requests, some of them holding data that you need in the response).
When you copy request URL (from network tab), follow instructions from this page to execute download from command line.
When prepared, you can execute that command through windows task scheduler.

Related

Acrobat JavaScript: Directory Contents and Hyperlinking Files

I've recently been looking into automating a few things in Acrobat through JavaScript and had a couple questions that came up. I'm wondering if anyone might have thoughts or tips on these, it would be appreciated!
Is it possible to either get the contents of a directory (so the list of documents can be stored as an array), or check for the existence of a file? From what I've gathered, this may not be something that JavaScript in Acrobat is able to accomplish due to lack of access to the file system.
Two tricks that I've seen suggested for checking to see if a file exists are to either import it as an attachment, or attempt to open the file and see what the result is. e.g. if (this.importDataObject("myFile", "/testfile.pdf/")) { This doesn't seem to work well for my situation as the files in question can be quite large, and importing or opening them can be slow.
Is there a best way to use JavaScript to set a hyperlink in a pdf to open a file? I've tried l.setAction("app.openDoc(\"testfile.pdf\")") which adds the quoted JavaScript to the hyperlink and uses it to open the file in a new window, but I'm wondering how this compares with the way Acrobat normally links to a file, or if there's another more optimal way to set this programmatically.
For question #1 - Short answer is "No". As you said, Acrobat JavaScript doesn't have access to the file system. Long answer is "Yes - depending on what you want to do". Acrobat has a way to run "Actions" that can process all the files in a given directory, a list of files, or all the files that are currently open. You can run JavaScript on each file in a directory using an action without that script needing to actually know the file name.
For Question #2. The only option you have to set a link action via JavaScript is to set the JavaScript action. If you create a link, you only get to set the mouse up action. However, you can add a button instead of a link where the background and border are transparent and there's no caption and you get something that visually looks like a link but you can set more actions like mouse up, down, and rollover (enter/exit).

determine final destination/url returned by a asp(?) by script

So, I'm sure this will get down voted and I'll be banned from the site because I'm a total newb that asks elementary questions/wrong questions/vague questions that irritate the community. But...I'm not sure where else to turn, this seems to be one of the best communities.
I use a live chat service. And they have this cut/paste bit of markup that you drop into your page and it'll display a jpeg indicating whether or not the chat service is open.
https://rci.ehosts.net/netagent/client/invites/chatimage.aspx?style=style0&questid=44D115D4-5242-476F-AE56-6AEECE8E9343&portid=28586436-7974-4E4B-BBDE-73E63BC3EAED&imagelanguage=en-us&customopenimage=https%3A//rci.ehosts.net/netagent/custfiles/en_chat_open.gif&customcloseimage=https%3A//rci.ehosts.net/netagent/custfiles/en_chat_closed.gif
If you drop that into a browser, it doesn't seem like anything happens. But if you set that as the src attribute of a image tag in a html page, it displays the appropriate image (chat online/chat unavailable).
My question is...can I determine the outcome of that script without dropping it into the page? Via either javascript or php? What'd like to do is make a script of my own that will determine which image is returned and do things other than just displaying the chat online/chat unavailable images.
Thanks!
EDIT: oh, the above url will redirect to the correct image if you change all the & to just &
I ended up following the redirect to see which image it takes me to. It was the only way I could think of. Ended up basically copying example #2, except that I returned the headers location, from the following link:
php manual

Pull To Refresh Script

I've just downloaded jquery.mobile.pulltorefresh from here
I've uploaded it to my server to test and see if its the right one I want to use... However when you go to it:
quotesin.com
It doesn't work at all. Think I'm having a blonde day, if this is no good does anyone know of any lightweight versions I could use? Rather than Cubiq one?
your error is that you mention 3 files that there are not found or can't access
the files are:
1. jquery.mobile-1.0.js
2. jquery-1.6.4.min.js
3. jquery.mobile-1.0.css
you can test this by open your link on google chrome them right click on the page then click inspect element
after that choose resources tab, then expand the script link in the side tree view you can find that these file not found

VBA Internet Explorer Download

First of all, I want to say that stackoverflow has helped me a GREAT deal with my current project, so thanks! But I have hit a point where I can no longer find help by skimming through previous questions. First, a little overview of what I'm doing.
At my job we have this extremely tedious process that we perform regularly, and I really want to automate it. The process is very basic, we go to a website, log-in, navigate to the appropriate page, copy and paste 6 values from excel into a form on the website, submit the form and download a specific output report. We repeat the process 60 times so this seemed like a prime candidate for Excel-Internet Explorer automation.
I built a pretty basic Excel Macro that can open up a new window with the appropriate URL, log-in, navigate to the correct page, fill in the form and submit the form, but I cannot figure out how to download the report. There is some added complexity to this, and I am just not familiar enough with Javascript, PHP ect. to figure out what is going on.
Problem: when I choose the correct report to generate, a new window pops-up with the generated report. This page is essentially empty when I click "View Source Document." There is a link to a Javascript page in the header, there is a little bit of CSS directly in the page, and like two tiny snippets of HTML. There is a download button on this page, but it doesn't look like it's being created in HTML.
QUESTION 1: Is this a Javascript application? Is there a way for me to simulate pressing the download button?
From what I understand, the complexity of my macro will increase substantially if I have to navigate between two IE windows, (not to mention there isn't an HTML link for me to click to download the spreadsheet) so I started to try and figure out a different way to do this. I looked at the URL of the new window with the generated report, and tried to analyze what was happening with Firefox's Developer tools (specifically Web Console). Here's where I get even more lost...
There are a series of "POSTS". These POSTS show up when the output is being created within the website:
POST https://www.website.com/somethingdb/quickframe/prod/#####/single_frame_results/correct_output.asp?THIS_KEY=370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a32353030303&bEmbedded=1
POST https://www.website.com/somethingdb/javascriptsource/prod/#####/website/forms/datagrid/DataGrid.html
POST https://www.website.com/qc?function=QuickFrameRmi
Then when I click the Download button, One POST and one GET show up:
POST https://www.website.com/qc?function=CorrectReport.generateExportFile
GET
GET https://www.website.com/somethingdb/quickframe/prod/#####/dlf/x6::370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a323530303031356026339.xls/sfn/RIGHTPAGE_scen_1_deal_cf.xls
The referrer for the GET looks like this:
https://www.website.com/somethingdb/quickframe/prod/#####/single_frame_results/correct_output.asp?THIS_KEY=370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a32353030303&sPagename=RIGHTPAGE&nScenarios=1&bShowExtendedFields=1&bShowAllCollats=0&ShowUnderlyingPage=0&sUnderlyingPage=&WebsitewrapDb=websitewrap&iSettleId=&bScrollbars=true&time=1356025839577
Like I said, my knowledge of these things is very limited but I do know that once I click on the download button, the GET from above is an actual file name that is stored on the website’s server. I can go into Excel, click on “Open” and put that file path in and it will download EXACTLY what I need.
So here is what I am thinking. If I could tell the website to simulate the download process, and log what that GET is, then I could just store that away, and when the Macro finishes generating all of the reports I need, it can just navigate back into Excel and set up a simple Loop to download all of them?? I have identified the portions of the GET that change:
x6::370120c59da884dbdc375b1582a2142c
63032353737363a3646314
6026339
This is my first post, so I apologize if I was too long winded in my back-story. Just wanted to be as clear as possible from the get go. I am wondering what is truly happening inside the website? Is what I am suggesting making sense? Is it possible? Is there a better way. Thanks in advance for any help.

Javascript Flowchart designer in style of CKeditor

I've learned here, on stack overflow, that what I need is called "flow-chart". I need to allow user to draw in a browser (using his mouse) a flow-chart that would represent a flow of a process or an algorithm. This flow-chart should contain decisions (yes/no diamonds), tasks (rectangles), arrows, labels etc.
Flow-chart editor should be a part of a Form on a web page, should be written in JavaScript and important is, that it must produce a text output (XML?) that will be storable in DB and when this page is opened again, chart will be rendered based on the previously save text.
No other outputs wil be necessary. Essentially, what I need is just a graphical XML editor/creator.
It would be perfect if it worked as same as CkEditor:
http://ckeditor.com/demo
This CkEditor is a JS Html editor that covers a textarea (in a form) and when you write something in this editor, it is (on the background) converted into HTML and written into the textarea. When you send the whole form using a button, the content of textarea can be processed and saved.
I read following thread, but nothing was suitable for my purposes
stackoverflow.com/questions/363592/javascript-library-for-hierarchical-flowchart
Very close to my idea was this project, but it is a Java program that is run outside of browser and output can not be stored in DB.
www.jgraph.com/jgraph.html
Did anyone use something like I just described?
Update 1: This could be what i was loking for. Does anybody know something similar and/or free?
origramy.com/origramy.html
You will find this thread useful. It talks about generating flowchart from BPMN which is in XML.

Categories