Using XmlHttpRequest to fetch webpage - javascript

I'm sorry for the vague title. I'm not quite sure how to word it.
I am fairly new to programming and working on a small chrome extension that checks a particular website for new product listings. I am retrieving the webpage using XmlHttpRequest, however I would like to get the webpage with certain display options already applied.
The webpage I am fetching uses javascript to change its display options. By default, it shows only in-stock items and 20 items. What I want is both in-stock and out-of-stock, and 100 items.
If I go to the website and directly change the options, then fetch the page with XmlHttpRequest, it will return what I want. But I would like to be able to do this all from within the program.
This is the webpage: http://ekizo.mandarake.co.jp/shop/en/category-action-figure.html
Any idea if this is possible and if so, how I could accomplish it? Thanks in advance.

If you look at what happens when you change the number of items to display it goes to this page with the added variables:
http://ekizo.mandarake.co.jp/shop/en/searchOption.do?action=setListCount&keyword=&searchStrategy=keyword&listCount=100
All you need to do is recreate that GET request with the details you want.
(although for the other things such as stock avaliability I have not looked as to where this is)
Firebug is your friend with this kind of thing.

Related

Capture the state of a web page in a URL

I find myself having to interact with a web page that hides state in various places so that one cannot easily share it as a URL, for example this page which allows users to look up information from city zoning applications:
https://aca.cityofberkeley.info/community/Default.aspx
You can interact with the page all you want, but the URL in the location bar will remain the same as the above.
Currently, city staff provide users with instructions like "Load this URL, click on the 'Zoning' tab, enter DRCP2020-0010 under the 'Permit Number' field, click 'Search', then when the records come up, click 'Record Info' and then select 'Attachments' from the dropdown menu, then click on the PDF document that says '2020-10-21_DRCP_APP_PCKT_2801 Adeline.pdf'". I would like to be able to replace these instructions with a URL.
Another example is the website where video from city council meetings is archived:
http://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa
It would be nice to be able to produce a link which brings up one of the meeting videos, and seeks to a certain timestamp like 53:40, so that I can refer to something specific that was said at a meeting.
Looking at the pages that are loaded when I follow the instructions in each case, I can see that there are some POST forms, cookies, hidden input fields, and so on.
Is there some kind of tool that I can use to create "deep links" to pages like these, that were generated using non-URL hidden state, which will allow me to quickly share what I'm looking at with another user?
What I'm seeking is similar to the frmget "bookmarklet", which changes the forms on a page to use GET instead of POST. Sometimes this succeeds in producing a URL which captures form submission query parameters. However, it doesn't work for these applications, for whatever reason.
This question is possibly related to the idea of capturing a web page's DOM state using "browser screenshots" and a script called html2canvas. A possible solution might involve getting and setting cookies in a bookmarklet. Ideally something that produces a normal "https://" URL would be ideal, but if it is impossible to solve the problem except by outputting a "javascript:" URL (bookmarklet) then that is acceptable to me (in spite of the security implications). Thanks.
That seems like not a programming matter. It seems like the site has some security issues as well.
QUESTION A: About Zoning
Here are some links you can use
Direct link to Zoning (I've found it via Advanced search from the site):
https://aca.cityofberkeley.info/CitizenAccess/Cap/CapHome.aspx?module=Planning&TabName=Planning&TabList=Home%7C0%7CBuilding%7C1%7CHousing%7C2%7CPlanning%7C3%7CFire%7C4%7CLicenses%7C5%7CPublicWorks%7C6%7CCurrentTabIndex%7C3
A strange link to the list of files (I've found it via downloading a file, then going to chrome://downloads, then right-clicking the file I've download. The link has been the following):
https://aca.cityofberkeley.info/CitizenAccess/FileUpload/AttachmentsList.aspx?iframeid=ctl00_PlaceHolderMain_attachmentEdit&module=Planning&isInConfirm=False&isdetail=True&isaccountmanager=False&isAdmin=True&isPeopleDocument=&agencyCode=BERKELEY&isForConditionDocument=N
It still doesn't give the direct link to the file, but it it gives the list of attachement of the previously opened Zoning record.
Currently I have no idea what file is triggered by javascipt:__doPostBack('attachmentList$gdvAttachmentList$ctl02$lnkFileName','').
In any case, based on what we have, step one, and then step two seems like minimize the path to download the file. I guess there could be a way to download the file directly, but I currently don't see any easy way. Maybe someone else could figure it out.
QUESTION B: About video
I've used an embed link that shows all the attributes that can be used.
There is a pretty strange but working way to give the exact timestamp. Change starttime from the link below:
https://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa&starttime=0&stoptime=undefined&autostart=1
So replacing 0 for 3600 will rewind the video forward by one hour (3600 seconds):
https://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa&starttime=3600&stoptime=undefined&autostart=1
The problem here is that ... you cannot rewind back manually that particular hour (it just gets kind cropped out). But it works to show the exact episode.
That's a pretty strange site.

modifying HTML page "on the fly"

I looked at "Generating HTML Page on the fly" on this website, but most of it was over my head.
I have a 2 part question that I would like assistance with please.
I want to fill a narrow vertical container, <div id=”counter”> with the numbers 1 .. <xx>.
<xx> is determined by the record count of a database, filtered “on-the-fly”, by the user choosing a category (no problem there – I have an SQL background)
Eg. Category1: 1 .. 200
Category2: 1 .. 6
These numbers could change over time, as I want to allow users to add content to the database (vetted of course).
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
I have created a small website of a similar nature to that, hard-coding all the images and links, but I am looking at 3000+ images (as a starting point here), and they differ per page.
I have created this scenario many times in stand-alone apps and from past experience, I thought perhaps, I could create a javascript routine which would use a loop to
• print the numbers to the <div> using the getelementbyID ( ).
• Fill an array with the record number, a title and an image link.
Question 1: Is this possible or am I beating a “dead horse”?
If it is possible, any suggestions would be gratefully accepted.
Part 2:
My current idea is that, as the user hovers the mouse over any number, a mouseover ( ) event will occur that will read the appropriate array record and display the <title> as a tool-tip-text.
If the user clicks the number, a function (I have yet to write) will read the appropriate array record and attach the image link to an <a> tag, and subsequently display the appropriate image to the screen.
Question 2: repeat of question 1.
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
Why are you so sure about that? You can't see php-code, because it is executed on the server. There is no way to know if it was hardcoded or by php
Answer:
It is possible.
If I understand this correctly, you want to read some data from a database and if the user clicks / hovers something, you want to load more data?
You have to splitt this into two things:
Load data with PHP from the db (Server side)
If you want a live, visual feedback you need JavaScript (and/or CSS3) to do changes. (Client side)
One possible solution is to create a API with php (maybe REST-like) and then call that api with JavaScript.
You could also do everything with PHP but this will require a reload of the website on every click. PHP cannot do changes On-The-Fly.
First of all you should learn the basics about web development.
And most important: If you decide to learn Web-Programming: learn about security, too. For example things like Cross Site Scripting and SQL-Injection. Never trust data coming from a client (e.g. JavaScript)!

Fetching Dynamic Website Data Using Java

I am currently using Java (Eclipse) to build this application which will be able to take a spans value from a specific website and show those values on console. But the issue i am having is, the Website i believe is Dynamic and therefore, i am not able to get those span values. The span (id=spot) keeps refreshing every 1 second and i guess it uses JavaScript. The class keeps auto changing as well when i do a "Inspect element". But if i do "View page source", there is no value inside the span. Could anyone suggest anyway to either render the web page and fetch data or any other method to do this. I have been
on google for the past 2 hours and nothing helped. I am new to this selenuim, JSoup, rhino which did not help at all. So please try to elaborate as much as you can :)
Here is the Web Page: https://www.binary.com/trading?l=EN
span id=spot

Oracle APEX: Call stored procedure from javascript

I need some help with oracle apex. The thing which I want to do is the following:
I have table with some data about people. So each row describes exactly one human. And I want to show some more information about certain human. For example, list of shops he or she has visited. Such data provided by other tables.
I see it in this way: right click on table with people on certain row -> select option (what kind of info to show) -> execute stored procedure and show new page with data table (e.g. list of shops). But how can I implement it?
I've already found this plugin. Now I can execute some JavaScript function after right-click. But how can I execute stored procedure and show new page?
I'm new in apex, any help would be appreciated.
You're trying to reinvent the wheel. You're new to apex. Have you taken a good look at the documentation?
Start out where everyone else has to start: at the beginning. Report + form. Column links.
There is ample help available to someone new to apex.
The 2-day developer guide, running you through some of the
basics of apex and a good familiarization.
Get a workspace at apex.oracle.com
Each workspace starts with the sample database application, based on
products and customers. You can view and edit this application and
thus you can glean plenty of information from it.
Furthermore, there are the packaged applications, many of which offer
good basic solutions to common situations. Again, you can glean a lot
of information on them, and they are even editable after you unlocked
them.
After you are familiar with the basics, you can look further ahead. What you are asking is simply too much for someone new to the matter. You even want to implement a jQuery plugin straight away. You're talking ajax. It's great if you know those subjects and they'll be of plenty of value to you, but it just seems you don't even know how to present and fetch your data.
A good start would be to make a report and a form. In the form you can then add some classic (or an interactive) report(s) to represent associated data.
It's possible. First of all, assume that each row contains unique identifier ID. You have to add hidden item to the page which would contain additional info about certain row. Let's name it P1_ID. Then add the following JavaScript code to page which contains initial data (in example from question, page with table with information about people):
function TestFunction(action, el, pos) {
var id = $(el).children('td[headers="ID"]').text();
var href = 'f?p={APPLICATION ID}:{PAGE_NUMBER}:&SESSION.::::P2_ID:'+id;
window.location = href
}
Function name should match name in plugin settings. Example: link
Replace APPLICATION_ID and PAGE_NUMBER with actual values according to application. PAGE_NUMBER is the page number which contains additional info about row.
Then you can add some reports to the page with additional info and use ID parameter to select information about certain row.
The only problem is that plugin mentioned in question stop working after table refreshing. For example, if we filter data in table then no menu on right click will be shown. I don't know how to fix it for now. Any ideas?

How to prepopulate few fields in a form, in a survey

I have a share point survey. When we responding to the survey, as we know, it will open NewForm.aspx. this page contains a ListFormWebpart in which questions from survey list will be displayed.
Now, i need to add few labels before the questions and these label values should be prepopulated from query string. What i am trying to achieve from this is, i wll created a link
with some values in query string and send to specific users. different users might have different values in query string. Whenever they click on the link, it should open the survey with prepopulated label values along with questions in list.
I am not sure, how to do it. I have tried to add some html control to web part(using share point designer) and through JavaScript i have tried to set query string values. Then i tried to put asp controls and trued. it didn't work. I am trying since last 2 days. No progress. I am using SharePoint 2003, WSS2.0
Can anybody, please help me to implement this solution.
You may find it easier to create a webpart with a custom form that enters data into the survey list.
The survey lists are useful as they are quite flexible, but your solution will likely make it hard to change your list in future. That means that a webpart specific for this survey may be a valid design decision for you.
An issue with your current implementation is that passing values specific to different users through the query string does not give you any guarantee that enterprising users will not change those values.
This may not really be an issue depending on your situation, but a custom control will allow you to query the current user and make decisions that way.
I have written some Java Script for reading query string, parsing and assigning values to controls based on their ControlID. I have used Content Editor Web Part to add this JavaScript to the Existing Survey page. Then this script has done job of prepopulating the fields.

Categories