How to search on any website in Website Scraping - javascript

I am working on a project to scrape data from a website using DOM parsing. It can extract data from a particular website using that page url.
I want to add functionality to pass a php variable to that website's search box. and search that query. When a user manually enters a value in the website search box and presses search, the result comes out and we select that result. I want to do this programmatically.
How I can pass my php variable to a website search box and search on that website?

You will have to check and see how the form works - to which endpoint the data is posted. Then just do the same in your script and process the result (which might be servied in various different formats - JSON, HTML, XML etc.). Sometimes there might be added security, in particular if it's a .NET site that uses viewstate.

A somewhat straight forward suggestion would be to run a script on submission of the form that searches the text in each of the web pages in your working directory to find a match, then display a page with links to the found matches.
I will use PHP for my description of how this is done.
With this in mind, first learn how to read entire pages (i.e. webpages) into a string:
http://php.net/manual/en/function.file-get-contents.php
//YOU WILL HAVE TO LINE THIS UP WITH YOUR WORKING FILE NAMES
$home = file_get_contents('./home.php', FILE_USE_INCLUDE_PATH);
or I suppose you could just search for the actual webpage/URL like so:
$home = file_get_contents('http://www.example.com/');//IMAGINE THIS IS REALLY HOME.PHP
$homePageName = "home.php";//JUST HERE TO SHOW AN EXAMPLE
Example:
///YOUR FORM/INPUT BOX
<form action="search.php" method="post">
<input type="text" name="findMe" placeholder="Search...">
</form>
Now search.php
$search = $_POST['findMe'];
//$search = "example";//THIS WOULD WORK, BUT I WAS SHOWING HOW TO USE FORM
//IF WORD FOUND IN HOME PAGE
if (stripos($home, $search) !== false) {//USING EXAMPLE.COM TO SHOW IT WORKS
echo ''.$homePageName.'';
}
Then if you want to be simplistic and not use an array to store the found pages, take the same code above and use it for every page you want searched (i.e. home, about, products, etc..).
Now a user can search your site (or the pages you want indexed), to find all pages that have matching text. If you want specific keywords to be searched, just add them to the page metadata and the process I have described will still work as it searches everything that makes up the page.
<meta name="keywords" content="keyword1, keyword2, keyword3 " />

Related

Using client side scripting to control an SAPUI5 app

So, we have an existing app, which I think is built on sapui5 if I am not mistaken. It's pretty much a simple search function app. I reviewed the DOM and noted that the search box form has the ID of #__xmlview0--searchField-F, and the search box container including all is under #__xmlview0--searchField and the actual text entry area is under #__xmlview0--searchField-I. Essentially, it looks something like this - highly simplified:
<div id = "__xmlview0--searchField">
<form id = "__xmlview0--searchField-F">
<input type = "search" id = "__xmlview0--searchField-I" />
<div id = "__xmlview0--searchField-search"></div>
</form>
</div>
There are two pages that I am interested in, one is the original start page for searching, and the other is the search page with results displayed, which also contains the same search box so that users can quickly perform another search. Lets call these start page and results page.
Since the app doesn't seem to take any QS for search strings, I was wanting to create a nested page where I can simply pass on the elements of searching into the app generated page, and display search results with one click, instead of having the users need to do them manually and transcribe things from one place to the search function.
I have attempted initially to simply change the value of the search box and then submit the search:
$("#__xmlview0--searchField-I").val("SOMETHING");
$("#__xmlview0--searchField-F").submit();
Such approach did not work. It appears that the submit will execute for the start page, but will not for the results page. But even with the start page where submit executed, the search text of "SOMETHING" has been lost for sure somehow. So I tried typing in text manually and only doing:
$("#__xmlview0--searchField-F").submit();
Which seems to show the submit button working (also only for start page, not for results page), though it appears that "SOMETHING" is still lost somehow even though I manually entered the text when executed in start page. Thinking something may be wrong with clicking the submit mechanism programatically, I tried only doing:
$("#__xmlview0--searchField-I").val("SOMETHING");
and then manually click the search execution icon (also tried pressing "enter"). But to my dismay, despite the DOM display the changes for the input search field properly, the search did not execute though the button blinked so I am sure I properly clicked it at least.
So I thought, maybe I need to trigger some kind of event before I start making changes to the search field to prompt proper event execution behind the scenes so that the input text will be recognized properly when changed:
$("#__xmlview0--searchField-I").focus();
$("#__xmlview0--searchField-I").val("SOMETHING");
$("#__xmlview0--searchField-I").focus();
$("#__xmlview0--searchField-F").submit();
But that did not work either. I am pulling my hairs out trying to figure out what may be going on. Can someone who knows the SAPUI5 stuff give me some insight? Am I doing something that is simply not possible under this environment?

How to take text area value and put it into hyperlink

I am creating a "web page" for employees that will be hosted locally on a drive for people.
We have a ticketing system that runs in a browser that allows us to add tickets, find tickets, etc etc.
I am creating a quick homepage for people to use so they can quickly find their tickets without clicking around on the portal and trying to understand the search functions.
to do this I have a hyperlink but need to change the url to include what is in a text area.
I believe php would work fine for this but it won't be hosted on a web server do not able to run PHP. I also believe there is a with Java/Html.
<a url="ticketsonline.co.uk/<username>/tickets">
I need to be whatever the user types into the text area.
What you can do is to store the base url in a variable then append what ever is the value of the text area to it and then set href value of the link to the resulted value
var baseLink = "ticketsonline.co.uk/";
function changeLink(){
document.getElementById("searchLink").href = baseLink + document.getElementById("searchText").value
}
Search
<br>
<textarea id="searchText"></textarea>
<br>
<button onclick="changeLink()">Update Link</button>

How to make an interactive website and allow the user to edit website content?

To begin with, I'm fairly knowledgeable with both html5 and css3. Right now I'm trying to create a website that allows visitors to edit website content so that all visitors are able to see the edits. I know about contenteditable but this attribute does not last when the page is refreshed and no other visitors to see the edits. For example:
<p> Edit me please </p>
A visitor would be able to edit the element above by visiting the website and simply selecting and typing. Also I have heard that javascript, jquery, sql, and php can all allow visitors to edit website content but I don't know which is the best.
To sum up, if anyone knows how to allow visitors of a website to edit its' content it would be much appreciated to share their knowledge.
Create a database table (lets call it "strings") , and just define 2 columns - "string_identifier" and "string_value"
When you render the page, simply query the database to fetch the strings and their identifiers. Each editable element should have an attribute containing its identifier.
You can use jquery to capture the click event on all editable elements, and replace it with an input with that text.
Then assign an event handler to that input, so when the enter key is pressed, it sends a request to a php script (with the identifier and new string as data) on the server to update the database with the new string.
If you would like live updates to all other clients (no need to refresh), use jquery to continuously request a file on the server (endless loop), which will return a JSON object containing all the strings. Then go through and replace the text of all editable elements on the data with the new strings you have received from the server.

Retrieve javascript string from mysql for subsequent edit

I have constructed an on-line What's On page supported by a mysql database table including a field to contain an event title. An admin page permits entries to be added, edited and deleted. More recently I was asked if web links could be embedded in the title field, and found that editing the field to include a link of the form:
Click <a class=hdg onclick="newWindow('http://www.address');"> HERE </a> to open web link
would do the trick and open the URL in a new window via the newWindow function.
Unfortunately any attempt to use the admin page to edit such a mysql record corrupts the admin page display, since the string is returned from the title field as the value of a text box, and the link text is then interpreted by the browser so that only part of the field displays in the box on the screen. The remainder of the string appears outside the box, which is confusing to a non-technical user.
A quick and dirty fix is to use Ctrl-A to select the whole of the text box contents when editing, and then type or paste the whole of the title content into the box, when it commits correctly to the database. However, if anybody knows of a way to code the javascript so that on the one hand it will function correctly as a web link and on the other hand can be edited via an HTML form, I would be glad to know. Ultimately I guess I'll re-construct the database to hold the actual URL separately and use php to build the javascript link, but meanwhile ?
Use htmlentities() to put the field value into the web page with all the HTML special characters replaced with entities, so they won't be interpreted:
echo '<textarea>' . htmlentities($value) . '</textarea>';

Facebook-like textbox that fetches a URL's contents upon paste

hi all i stuck at a point in which i want to ma a multiple line text box to work just like a FACEBOOK text box . i.e. when a URL is pasted or typed (along with the text) it will it fetches the information from the page specifed by URL
well i found somthing at
This post
i want to know how to get URL in VAR in javascript
coz when i enter a url it fetches complete information how ever when i put another url it treat it as a text
Constantly search the text for a url via regex, possibly on a javascript event like onkeyup. Use ajax to load the page, and scrape out whatever info you want to display.
http://www.devguru.com/technologies/ecmascript/quickref/evhan_onkeyup.html
http://regexlib.com/Search.aspx?k=URL

Categories