I am trying to write a program which can automatically fill in and submit a form in a web in particular time slot.
But i have no idea how and where to start. i searched this in google, but only resulting very general answer like using JavaScript, python. Can anyone tell me which languages should i learn first?
Despite the fact that the generic advice on this thread is quite good, it's pretty broad. I have tackled this problem myself, and despite the fact that I posted a fully functional example, it was deleted by a moderator, despite "theoretically answering the questions".
So, for anyone else looking to solve this problem, you will want to do the following:
Use Selenium and openpyxl, these are two modules that are relatively straight forward and will perform this task perfectly.
You will use selenium to open your web page, and retrieve the relevant html elements that you wish to populate. I suggest finding elements by xPath if you aren't well versed in HTML. The Xpath finder google chrome addon will make this very easy to do.
The driver.get() and driver.find_element_by_xpath() will be the functions that you need.
We will use openpyxl to work with our excel sheet. 'load_workbook()' will load a workbook. We will then use the 'sheet = workbook.active' function to access a sheet from within the workbook.
We now have the functionality to open our website and select an excel sheet.
Now we need to assign cell values to variables, so that we can then populate the HTML form. We assign a variable to each COLUMN in the workbook. So if column A contained first_names we could assign that to by a variable by writing 'FNAME = sheet['A']. Now that we have a way of referring to cells within columns we can begin feeding the data into our HTML form.
We populate our form by using the .send_keys() function in Selenium.
first_name.send_keys(FNAME.value)
the .value makes sure that the correct value is displayed, as sometimes selenium will print the cell function rather than value, we do not want this to happen.
Now that we can print values to our HTML forms from our excel sheet we will need to iterate through each row. We do this with a simply while loop:
i = 1
x = 1
while x <= 50:
first_name.send_keys(FNAME[i].value)
i+=1
x+=1
driver.quit
Once the loop occurs 50 times, the driver will quit, closing the browser and stopping the script.
Some other misc stuff you may find useful when trying to automate this:
driver.back()
time.sleep()
If you would like to see an actual working example feel free to PM me, as apparently posting it here doesn't contribute to the discussion.
The answers you found, as general as they are, are correct. I'll try to explain it to you.
As you are trying to automatize some activity, you have to use a script language to basically
Get stuff references (like getting indexes, forms/fields IDs, etc)
Insert/change stuff (like filling a field, following the field references)
Save stuff (prepare a file, with field references and it's content, that can be injected to the specific system or website)
One example of script language is Python.
So you already have your script. Now you need to inject it on the page. The programming language that can do it for you is Javascript, or JS. It allows you to talk with the webpage, GETting data/references or POSTing data.
I suggest you to write a Python script using a library called Selenium Webdriver.
It will be easy to implement what you have in mind.
Related
I have a small web application written in vanilla PHP with MySQL database, on which registered users are able to create custom profile pages.
Id like to add a textarea form field in the control panel, for users to add their custom tracking code (namely Facebook Pixel or Google Analytics) for their tracking purposes.
My question is, what is the correct way to add such functionality? I'm afraid letting my users to "inject" custom code would lead to security issues for my website. As far as i know the aforementioned tracking codes use regular JS/HTML for their tracking. If that is the case how to allow JS while restricting server side code, like PHP, from being executed?
Any help would be greatly appreciated.
The tracking code is always the same, apart from one or two elements. For instance, in a Google Tracking code the only variable element is a tracking ID in the format: UA-XXXXX-Y.
You could let the user select which type of tracking they want and let them only enter the tracking ID, and perhaps one more variable. You can easily check whether these have been entered in the correct format. From that you can generate the Javascript code yourself.
Alternatively, now that you know the code is always the same you could also use that to check what has been entered. Strip out any non-linguistical elements like space, and check it against your template: Only the ID's should be variable and you know their format.
However, I think this last method is less user-friendly, because it relies on the users to provide you with a rather exact copy of the code. That might be too difficult, and that will be your problem.
I made a website for crowd-funding. I know that we should have used a platform for this. (other issues determined us not to)
The page that I have created has no database behind.
What I am trying to do is create some kind of hidden form that updates the sum that was raised so far.
I am not a very technical person but I do know that modifications made through javascript / jQuery ar usually temporary.
But, since scripts like website visit counters do exist I am wondering and appealing to the collective wisdom of this community:
Is there a way to update an attribute of a html element through some kind of hidden form without a database behind?
Perhaps writing to a .json file and updating the attribute from the data?
(I need to do this today as I will not be at the office during the campaign and it is very hard for a person that has no technical skills to do it... not that hard, but still, not user friendly.)
In order to display variable data, you need to get these data somewhere.
Do you have write access to your server file system?
What service level do you expect during data manipulation? Does it suffice if you just go and upload modified file every time manually?
What about embed in your Web page an IMG and then upload it with always the same name and different content?
There is a database even behind "dummy" hit counters, no magic.
I'd like to replace a parts requisition process at my workplace with a simple (and cheap to implement) electronic process, initiated using a Google Form. The problem is that I would like users to be able to enter multiple parts (along with associated info, e.g. quantities required, booking references etc.), but I want to do so without having to have multiple repeated questions.
I have researched this extensively and cannot find anything which fits the bill - my thoughts are to use Google Apps Script to create a table in the form which a user can fill-in. The closest I have found is something like this: Creating Form Elements Dynamically
The original paper form looks like the below - I would like the table to request the information as shown below in a similar format:
Thanks in advance!
EDIT! To make it clear, I'm happy to consider other solutions to run this process through an online interface - I have gone for Google Sheets/Forms in the first instance as they are already well integrated within my company and I have experience of them (setting-up triggers etc is pretty simple)
I understand the OP has probably long moved on from this problem. I however have done something along these lines in the past and thought I'd share my approach with the community.
I'll start with the premise Google forms are just good ol' plain HTML forms that users programmatically generate with their form builder. Therefore it's possible to traverse the as-built form and extract both submit location and all field names:
document.querySelectorAll('form').forEach((x) => {console.log(x.action)})```
document.querySelectorAll('[name^="entry."]').forEach((x) => {console.log(x.name + '=' + x.closest('[role="listitem"]').querySelector('[role="heading"]').innerText)})
The above snippet will give you an idea of what the components are.
All that's left after is to build a front end to your requirements with the framework of your choice (I used AngularJs at the peak of its popularity) and incorporate as much or as little UI and validations into it as you desire.
Here you've got the flexibility to either submit the whole thing as one JSON, or to parse it into individual fields and submit entries one by one, for purposes of this demo I opted for the easiest way but this thing surely's got the potential.
In my application, there is a comment box. If someone enters a comment like
<script>alert("hello")</script>
then an alert appears when I load that page.
Is there anyway to prevent this?
There are several ways to address this, but since you haven't mentioned which back-end technology you are using, it is hard to give anything but rough answers.
Also, you haven't mentioned if you want to allow, or deny, the ability to enter regular HTML in the box.
Method 1:
Sanitize inputs on the way in. When you accept something at the server, look for the script tags and remove them.
This is actually far more difficult to get right then might be expected.
Method 2:
Escape the data on the way back down to the server. In PHP there is a function called
htmlentities which will turn all HTML into which renders as literally what was typed.
The words <script>alert("hello")</script> would appear on your page.
Method 3
White-list
This is far beyond the answer of a single post and really required knowing your back-end system, but it is possible to allow some HTML characters with disallowing others.
This is insanely difficult to get right and you really are best using a library package that has been very well tested.
You should treat user input as plain text rather than HTML. By correctly escaping HTML entities, you can render what looks like valid HTML text without having the browser try to execute it. This is good practice in general, for your client-side code as well as any user provided values passed to your back-end. Issues arising from this are broadly referred to as script injection or cross-site scripting.
Practically on the client-side this is pretty easy since you're using jQuery. When updating the DOM based on user input, rely on the text method in place of the html method. You can see a simple example of the difference in this jsFiddle.
The best way is replace <script> with other string.For example in C#use:
str.replace("<script>","O_o");
Other options has a lot of disadvantage.
1.Block javascript: It cause some validation disabled too.those validation that done in frontend.Also after retrive from database it works again.I mean attacker can inject script as input in forms and it saved in database.after you return records from database in another page it render as script!!!!
2.render as text. In some technologies it needs third-party packages that it is risk in itself.Maybe these packages has backdoor!!!
convert value into string ,it solved in my case
example
var anything
I am planning to create a website which will let you iteratively construct an SQL query.
The idea is the following:
while(user wants more where clauses)
{
show selection (html select) for table columns
let user choose one column
upon selection, show distinct values of that column
let user choose one/multiple value(s)
}
I know how to handle the SQL part, but I am not sure how to tackle the iterative building of the page.
So my questions are:
What is the best method to build the page iteratively with the idea sketched above?
What do I do, if the user changes one of the previous selections?
The website will be build with Perl and I am thinking of utilizing Ajax for the dynamic part.
Any help is much appreciated.
If I were to do this, I'd use SQL::Abstract.
UPDATE:
If you don't want to redraw the whole page, you're going to be using AJAX. So find yourself a JavaScript library that you feel comfortable with that includes ajax calls. Jquery has this, a bunch of others do too. People have differing opinions on various libraries.
Anyway, your workflow looks like this:
user submits form
javascript performs client-side validation
javascript submits AJAX-style to the server
Server performs server-side validation, data manipulation, etc.
Server responds with data paylod
client updated the screen based on the contents of the payload.
So let's concentrate on 5 and 6.
Data Payload: The X in AJAX means XML, but many apps send back JSON, or HTML.
Update the Screen:
You can apply HTML directly to the existing page by setting a tag's innerHTML or outerHTML attribute. But that doesn't update the DOM. If you don't dig around the DOM in your clcinet code, then this can suffice. If you dig around, then you need to build nodes and add them to you page's DOM, so you might want to consider sending back JSON or XML.
So let's say that you have a div with id='generatedSQL' when your AJAX call retruns, it will fire a callback method (let's call it updateSQL()) and inside the callback you'll have code that looks approximately like:
$(#generatedSQL).innerHTML = theVariableHoldingTheHtml;
Your other option is to parse the JSONXML/etc. and using createNode(),etc, build new DOM bits and plug them into your page. I don't have the jquery chops to write this for you. I look it up every time I need to do something like it.
If the query text is only ever display-only, and you never try to dig around in it on the client side, just use the innerHTML method, whether you pass HTML or pass JSON and generate HTML from it. If the query text is important to the inner workings of the client, then you'll need to write bunch of DOM manipulation code.
No sure what you would be using but, if uses clicks on something then use the Onload event of the element with ajax call to script which brings back the data/content and on readystate just update the innerHTML of the container DIV.
Hope following link will help you understand the concept will give you a code to start with as well.
http://www.w3schools.com/php/php_ajax_database.asp