Javascript pull data from website (member site) - javascript

I haven't written any code for this yet as I am trying to figure out how to do it first. I have a site at work that I have to log into with a password. Its a site that we use to manage our lanes (I work in transportation currently) the information is displayed as a popup across several taps.
What I need to be able to do is pull the data off of a field on the last tab (its the field that shows total amount for our loads) the page is a .aspx page so is it possible to scrub this? Is there another method without getting the service involved? I am trying to do my own analysis program but really confused on where to start with retrieving the data.
any helpful hints or links are appreciated.

Try the Jquery load function that target and loads the returned HTML into your nominated result field. You also can target that specific field that you want.
$( "#result" ).load( "targetDomain/targetPage.html #targetField" );

Related

modifying HTML page "on the fly"

I looked at "Generating HTML Page on the fly" on this website, but most of it was over my head.
I have a 2 part question that I would like assistance with please.
I want to fill a narrow vertical container, <div id=”counter”> with the numbers 1 .. <xx>.
<xx> is determined by the record count of a database, filtered “on-the-fly”, by the user choosing a category (no problem there – I have an SQL background)
Eg. Category1: 1 .. 200
Category2: 1 .. 6
These numbers could change over time, as I want to allow users to add content to the database (vetted of course).
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
I have created a small website of a similar nature to that, hard-coding all the images and links, but I am looking at 3000+ images (as a starting point here), and they differ per page.
I have created this scenario many times in stand-alone apps and from past experience, I thought perhaps, I could create a javascript routine which would use a loop to
• print the numbers to the <div> using the getelementbyID ( ).
• Fill an array with the record number, a title and an image link.
Question 1: Is this possible or am I beating a “dead horse”?
If it is possible, any suggestions would be gratefully accepted.
Part 2:
My current idea is that, as the user hovers the mouse over any number, a mouseover ( ) event will occur that will read the appropriate array record and display the <title> as a tool-tip-text.
If the user clicks the number, a function (I have yet to write) will read the appropriate array record and attach the image link to an <a> tag, and subsequently display the appropriate image to the screen.
Question 2: repeat of question 1.
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
Why are you so sure about that? You can't see php-code, because it is executed on the server. There is no way to know if it was hardcoded or by php
Answer:
It is possible.
If I understand this correctly, you want to read some data from a database and if the user clicks / hovers something, you want to load more data?
You have to splitt this into two things:
Load data with PHP from the db (Server side)
If you want a live, visual feedback you need JavaScript (and/or CSS3) to do changes. (Client side)
One possible solution is to create a API with php (maybe REST-like) and then call that api with JavaScript.
You could also do everything with PHP but this will require a reload of the website on every click. PHP cannot do changes On-The-Fly.
First of all you should learn the basics about web development.
And most important: If you decide to learn Web-Programming: learn about security, too. For example things like Cross Site Scripting and SQL-Injection. Never trust data coming from a client (e.g. JavaScript)!

How to scrape website data into an Excel worksheet?

I'm a novice programmer trying to compile an Excel list of all the inc5000 companies and their industry, location, revenue, and CEO. Is there any way for me to automate this so that I don't have to manually input all 5000?
Some issues:
-The inc5000 list only displays 50 companies on a page, and scrolling to the next page does not change the URL. I tried converting the URL into HTML, but none of the metadata actually shows up in the HTML code (I used https://try.jsoup.org/~LGB7rk_atM2roavV0d-czMt3J_g).
-All of the information I need is on this one scrolling page (https://www.inc.com/profile/loot-crate), but the URL changes for each company as you progress down the page. Is there any way to grab the data from this site without manually changing 5000 URLs?
I'm really new to programming and I know next to nothing about HTML/JavaScript/Web design-- I only know basic Java. I would really appreciate any help or potential leads into a solution.
Here's the easy way:
Go to the page, hit f12, go to the "Network" tab of debug tools, select XHR (to filter to only the data calls) then scroll to the bottom of the page. The page makes a query for each company, that you can access in the debug tools.
Once you have all the pages, you can highlight all the rows in the file name list to the left, right click, and save it to a .har file.
From there, just write a script to pull out the json and you're set.

Grabbing information from one page and displaying it in another

I have a html page I'm making, which displays a bunch of category links - each of which link to an inventory page with a number of products. The client would like the page with the links to have the number of equipment available next to the title e.g.
Excavators (5)
The issue is, I dont have access to the back end of the inventory, its hosted with another company. At the top of each inventory it has a number of matches field, which is the information I need
Can anyone think of a way i can show that matches number on a different page dynamically (without php). The only thing I can think of at the moment is doing a really hatchet job with an iframe - but that wont be pretty or professional. Any genius thoughts would be appreciated!!
p.s the page im linking to is internal in my site - the inventorys in an iframe
If the data you need is an identifiable container, and you can work out the url to retrieve it from, you could use the jQuery .load() method, e.g.
$( "#result" ).load( "ajax/test.html #container" )
Where #container has the data you want, and #result is where you want to load it to.
http://api.jquery.com/load/
You could store the web page as an html string in localStorage. It's not too complex.
localStorage.setItem('myPage','<div>Your html page in a string friendly format</div>');
to retrieve:
var retrievedObject = localStorage.getItem('myPage');
console.log('retrievedObject: ', retrievedObject);

loading a website page and manipulating its dom to automate form filling

I have to go to https://onlineservices.tin.egov-nsdl.com/etaxnew/tdsnontds.jsp every month and fill the Form 26QB on that page .It is painful to add all details every single time i thought to automate the form filling.
Now i can manipulate the dom using jquery and do it for myself.
But i wanted to come up with a generic solution which helps everyone in this by asking the user to fill the static details only for first time and remembering it for later times.
First idea which came to my mind is to create a website where user adds all static details and get that automated .
But the problem is once i load the Form 26QB page my javascript code goes out of context.
So my questions are :
Is there a way to to create a website which does this.Can you elaborate on steps as my javascript code will disappear as soon as i navigate to the page and hence not possible.
I can use phantom.js or selenium but i am not sure how to run it in a website and whether it will be too cumbersome.
2.How about a chrome extension ?
3.At various places i hear dom manipulation might be termed as "hacking"/DOS attack, although i think since i am not automating any login here and simple filling form on client side it should be legal?
Probably the best solution would be to create a chrome plugin, but the easiest would be to create a jQuery script and then inject it through the chrome console.
So every time that you have to fill the form, just paste something like this:
$( "input[name*='PAN_purchaser']" ).val( "the value you want" );
$( "input[name*='PAN_seller']" ).val( "the value you want" );
$( "input[name*='PAN_seller_confirm']" ).val( "the value you want" );
and so on for each field...

Oracle APEX: Call stored procedure from javascript

I need some help with oracle apex. The thing which I want to do is the following:
I have table with some data about people. So each row describes exactly one human. And I want to show some more information about certain human. For example, list of shops he or she has visited. Such data provided by other tables.
I see it in this way: right click on table with people on certain row -> select option (what kind of info to show) -> execute stored procedure and show new page with data table (e.g. list of shops). But how can I implement it?
I've already found this plugin. Now I can execute some JavaScript function after right-click. But how can I execute stored procedure and show new page?
I'm new in apex, any help would be appreciated.
You're trying to reinvent the wheel. You're new to apex. Have you taken a good look at the documentation?
Start out where everyone else has to start: at the beginning. Report + form. Column links.
There is ample help available to someone new to apex.
The 2-day developer guide, running you through some of the
basics of apex and a good familiarization.
Get a workspace at apex.oracle.com
Each workspace starts with the sample database application, based on
products and customers. You can view and edit this application and
thus you can glean plenty of information from it.
Furthermore, there are the packaged applications, many of which offer
good basic solutions to common situations. Again, you can glean a lot
of information on them, and they are even editable after you unlocked
them.
After you are familiar with the basics, you can look further ahead. What you are asking is simply too much for someone new to the matter. You even want to implement a jQuery plugin straight away. You're talking ajax. It's great if you know those subjects and they'll be of plenty of value to you, but it just seems you don't even know how to present and fetch your data.
A good start would be to make a report and a form. In the form you can then add some classic (or an interactive) report(s) to represent associated data.
It's possible. First of all, assume that each row contains unique identifier ID. You have to add hidden item to the page which would contain additional info about certain row. Let's name it P1_ID. Then add the following JavaScript code to page which contains initial data (in example from question, page with table with information about people):
function TestFunction(action, el, pos) {
var id = $(el).children('td[headers="ID"]').text();
var href = 'f?p={APPLICATION ID}:{PAGE_NUMBER}:&SESSION.::::P2_ID:'+id;
window.location = href
}
Function name should match name in plugin settings. Example: link
Replace APPLICATION_ID and PAGE_NUMBER with actual values according to application. PAGE_NUMBER is the page number which contains additional info about row.
Then you can add some reports to the page with additional info and use ID parameter to select information about certain row.
The only problem is that plugin mentioned in question stop working after table refreshing. For example, if we filter data in table then no menu on right click will be shown. I don't know how to fix it for now. Any ideas?

Categories