First of all, I want to say that stackoverflow has helped me a GREAT deal with my current project, so thanks! But I have hit a point where I can no longer find help by skimming through previous questions. First, a little overview of what I'm doing.
At my job we have this extremely tedious process that we perform regularly, and I really want to automate it. The process is very basic, we go to a website, log-in, navigate to the appropriate page, copy and paste 6 values from excel into a form on the website, submit the form and download a specific output report. We repeat the process 60 times so this seemed like a prime candidate for Excel-Internet Explorer automation.
I built a pretty basic Excel Macro that can open up a new window with the appropriate URL, log-in, navigate to the correct page, fill in the form and submit the form, but I cannot figure out how to download the report. There is some added complexity to this, and I am just not familiar enough with Javascript, PHP ect. to figure out what is going on.
Problem: when I choose the correct report to generate, a new window pops-up with the generated report. This page is essentially empty when I click "View Source Document." There is a link to a Javascript page in the header, there is a little bit of CSS directly in the page, and like two tiny snippets of HTML. There is a download button on this page, but it doesn't look like it's being created in HTML.
QUESTION 1: Is this a Javascript application? Is there a way for me to simulate pressing the download button?
From what I understand, the complexity of my macro will increase substantially if I have to navigate between two IE windows, (not to mention there isn't an HTML link for me to click to download the spreadsheet) so I started to try and figure out a different way to do this. I looked at the URL of the new window with the generated report, and tried to analyze what was happening with Firefox's Developer tools (specifically Web Console). Here's where I get even more lost...
There are a series of "POSTS". These POSTS show up when the output is being created within the website:
POST https://www.website.com/somethingdb/quickframe/prod/#####/single_frame_results/correct_output.asp?THIS_KEY=370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a32353030303&bEmbedded=1
POST https://www.website.com/somethingdb/javascriptsource/prod/#####/website/forms/datagrid/DataGrid.html
POST https://www.website.com/qc?function=QuickFrameRmi
Then when I click the Download button, One POST and one GET show up:
POST https://www.website.com/qc?function=CorrectReport.generateExportFile
GET
GET https://www.website.com/somethingdb/quickframe/prod/#####/dlf/x6::370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a323530303031356026339.xls/sfn/RIGHTPAGE_scen_1_deal_cf.xls
The referrer for the GET looks like this:
https://www.website.com/somethingdb/quickframe/prod/#####/single_frame_results/correct_output.asp?THIS_KEY=370120c59da884dbdc375b1582a2142c1363533393a313335363032353737363a3646314a313937334a32353030303&sPagename=RIGHTPAGE&nScenarios=1&bShowExtendedFields=1&bShowAllCollats=0&ShowUnderlyingPage=0&sUnderlyingPage=&WebsitewrapDb=websitewrap&iSettleId=&bScrollbars=true&time=1356025839577
Like I said, my knowledge of these things is very limited but I do know that once I click on the download button, the GET from above is an actual file name that is stored on the website’s server. I can go into Excel, click on “Open” and put that file path in and it will download EXACTLY what I need.
So here is what I am thinking. If I could tell the website to simulate the download process, and log what that GET is, then I could just store that away, and when the Macro finishes generating all of the reports I need, it can just navigate back into Excel and set up a simple Loop to download all of them?? I have identified the portions of the GET that change:
x6::370120c59da884dbdc375b1582a2142c
63032353737363a3646314
6026339
This is my first post, so I apologize if I was too long winded in my back-story. Just wanted to be as clear as possible from the get go. I am wondering what is truly happening inside the website? Is what I am suggesting making sense? Is it possible? Is there a better way. Thanks in advance for any help.
Related
This question already has answers here:
Save images to hard disk WITHOUT prompt?
(6 answers)
Closed 3 years ago.
I'm a complete newbie to this, but I'm trying to put together some code that will automatically download a .csv file from a webpage every hour and save it to my computer (preferably a specific place). The data is hourly rainfall data from Scotland at the following website.
http://apps.sepa.org.uk/rainfall
Basically I need to be able to select one of the sites in the map on the left (it'll be the same site each time) and download the hourly rainfall .csv file from the dropdown menu on the upper right hand side of the graph. When doing it manually you hover over the download button, then hover over "Download as..." then select "CSV" when it comes up.
From what I've read on here, the Chrome Tampermonkey extension used in conjunction with the Windows Task Scheduler might be the best way to do this. From the previous question there is the code:
document.getElementById("download_button").click();
I've tried to replace the "download_button" section with something that might describe the ID of the CSV button element on the page, such as using 'CSV'; but I'm really not sure what the element ID of the button is. I've tried inspecting the element in the source code but as I'm new to this I don't really know what I'm looking for.
I also tried to search for the element by class and by path but again I was really unsure as to what value for the class and path to put in to get to the specific download button I required.
document.getElementByClass("export-main").click();
document.getElementByPath("//*[#id=\"chartdiv\"]/div/div[2]/ul/li/ul/li[1]/ul/li[3]/a/span").click();
Any help would be greatly appreciated, I'm not even sure if it's possible or if this is the best way to go about it.
Thank in advance for any help you might be able to give me.
Just as a proof of concept, better approach would be to aim at specific http GET request for the data (location) that you need. That way you get JSON formatted response that you asked for (can be converted to CSV). Google how to open Chrome DevTools (F12), you will find request that you need on Network tab (bunch of requests, some of them holding data that you need in the response).
When you copy request URL (from network tab), follow instructions from this page to execute download from command line.
When prepared, you can execute that command through windows task scheduler.
So, I'm sure this will get down voted and I'll be banned from the site because I'm a total newb that asks elementary questions/wrong questions/vague questions that irritate the community. But...I'm not sure where else to turn, this seems to be one of the best communities.
I use a live chat service. And they have this cut/paste bit of markup that you drop into your page and it'll display a jpeg indicating whether or not the chat service is open.
https://rci.ehosts.net/netagent/client/invites/chatimage.aspx?style=style0&questid=44D115D4-5242-476F-AE56-6AEECE8E9343&portid=28586436-7974-4E4B-BBDE-73E63BC3EAED&imagelanguage=en-us&customopenimage=https%3A//rci.ehosts.net/netagent/custfiles/en_chat_open.gif&customcloseimage=https%3A//rci.ehosts.net/netagent/custfiles/en_chat_closed.gif
If you drop that into a browser, it doesn't seem like anything happens. But if you set that as the src attribute of a image tag in a html page, it displays the appropriate image (chat online/chat unavailable).
My question is...can I determine the outcome of that script without dropping it into the page? Via either javascript or php? What'd like to do is make a script of my own that will determine which image is returned and do things other than just displaying the chat online/chat unavailable images.
Thanks!
EDIT: oh, the above url will redirect to the correct image if you change all the & to just &
I ended up following the redirect to see which image it takes me to. It was the only way I could think of. Ended up basically copying example #2, except that I returned the headers location, from the following link:
php manual
This is a little hard for me to explain as i am not much of a coder. So here we go.
I have script that is mostly encoded with ion cube. Not i have an addon that is also encoded. The addon is loading in a tab number 5. The problem is that every time i click on a function in the tab that complete page reloads, and you have to navigate back to tab 5 to see the results.
So this is what i am thinking. As i can't edit code and the make of the code doesn't know when they are adding tab support to their little piece of code. i figured that there are two options
1 Force the following variable to the end of the url #tab5 so that when people click a function they will not have to navigate back to the tab. How do i do this coding wise, javascript or something else and what would code be or where would i look to code something like that. I have looked everywhere but nothing really applies to my situation
2 I think it would also work if each function which is called just reloads that section of the page in an ajax field so that not the full page is being reloaded.
Who can help find the right solution.
thanks
I am wanting to find a way I can have others edit a html webpage (found locally) and allow them to save the changes they have made.
I know this is an odd request, I have a very specific idea in mind, which I will include below as an optional read for those interested.
I found that this code in the address bar works for the editing: document.body.contentEditable='true'; document.designMode='on'; void 0
Then I found that just adding:
<div id="columns" contentEditable="true">
gets the job done since I can edit the html.
The problem is that I cannot save the changes. I have tried saving the page in several different formats, including webpage/complete, HTML only, TXT file.
Is there anyway to save these edits whether by saving the HTML or using some other kind of simple storage?
Reasoning:
I have a simple local html page I will be using with other co-workers. These co-workers are not very tech-savvy (even less than me). The very mention of trying to explain how to edit the html and make changes to the text created blank stares, and an unwillingness to work on the project. I would very much prefer for them to be able to edit the text of this HTML page from the front end and not mess up any of the coding.
Also, I have installed a Xampp server, and wordpress - this has become a bit overkill for the purpose and again, has confused the others.
Thanks,
Go into JavaScript. Select and right-click on . Click copy as html. Then save into a .txt file somewhere. If you want to restore the website, right-click on and select Edit as html. Copy/paste the code in. Then it's back to normal.
You can use localStorage to save the data in the user's browser.
Ok,
I will not get full answer to this question, but perhaps somebody can give me a few tips, or tell me if it is feasable at all. The current situation is:
A printed out document created from a InDesign file.
No great structure in the data beside the visual markup.
The document is used as a GUIDE/CHECKLIST style.
So kind of "read some stuff" - "if yes page 20" - "if no page 34"
A little bit like these early TXT only adventures.
The client now want to "put this on a tablet" - the obvious approach is to take all the data, format it in some nice XML, write an app to render the data and provide the extra features.
Now, on the other hand, they are already close to their final product, a simple PDF rendering of the document looks nice on a tablet. There are some features missing, however, like "navigate back links" (you click, go to page 23, click there on "back" and are where you started).
I though I could do this with JS in PDF, but this does not work on Android. He just ignores the javascript (calculator example).
So now, before suggesting to implement your own solution, I am looking for something still close to the source. E.g. InDesign PLUS CODE.
I hope you get my drift. Perhaps another example: What you cant do with a simple Word Dokument, you do with VBA inside word.
What I cant do with a simple PDF I do with ??? + ???? on IPad/Android (we would not need both versions, one is enough, its a closed system, we control the device and the source file. Perfect for a standalone app, but overkill for step 1, I think).
Chris
You should consider going with iBooks for this. The format works really well with tools like In Design and you can use HTML5 tools to embed links, navigation, video, etc. You don't go the App Store route, but instead go with the iBookStore which can give just as good of a revenue option. The new iBooks Author will be a great tool, but even older tutorials will still work. Here is a good starting place