I am trying to write a Python script that logs into website for me and downloads my messages. I'm using the the "requests" library and succesfully able to log in with a session
The website uses js/Ajax to download the actual messages client side after the messages page has been loaded, and uses pagination to achive an infinite list of messages effect.
The messages page url is in the form of https://website.com/messages. and that page contains an Ajax call that retrives the messsages like this:
$.ajax({
url: '/messages?mailbox=' + mailbox + optional params
}).done(function(data) {
urlLazyLoading = data.next_page_url;
$.each(data, function(i, v) {
//from this point on, data for each message can be
//accessed using the v object as such:
//v.sender, v.date, v.last_message, etc
//and the <ul> element is populated with '<li>'s
But when I try to access the same url in python (after I've loggedin) as such:
session.get('https://website.com/messages?mailbox=inbox')
instead of getting a json response with the messages data, I get the same/original messages page.
the only parameter passed to ajax in the code above is url, so by defult it should be just making a simple GET request, which is what I'm doing.
Is it possible that maybe somewhere else in the code ajax is being set up to default to POST or have some other non-default settings?
or is the server somehow detecting that one request is coming from ajax and the other from sessions? maybe using user-agent?
Got it!
Ajax sends a header value by default:
{'X-Requested-With': 'XMLHttpRequest'}
I added that in and viola
headers = {'X-Requested-With': 'XMLHttpRequest'}
r = s.get("https://website.com/messages?mailbox=inbox",headers=headers)
Related
This issue comes down to the fact that I haven't yet figured out how to pass data from javascript to Flask/Python via AJAX in the right way.
I have built a Flask application that uses JsPsych as a javascript/front-end framework for an online experiment. jsPsych allows you to create a handler function, on_finish(), that executes once the experiment is finished.
I wanted my experiment's data to be stored on the backend, and that was supposed to be the end o the experiment. I created a function called data_to_csv() that was mounted to the url /temp_storage, that did this. Then I created another url endpoint for /experiment_end, which displays a simple html page stating that the experiment is over and data was saved.
jsPsych.init({
...
on_finish: function() {
var subj_id = jsPsych.data.get().select('user_id').values[0];
var test_trials_collection = jsPsych.data.get();
var test_trial_json = test_trials_collection.json();
$.ajax({
type: "POST",
url: "/temp_storage",
data: {experiment_data:test_trial_json, webapp_id:subj_id},
success: function(response) {
window.location.href="/experiment_end";
},
error: function(xhr, textStatus, thrownError) {
alert('An error occurred while trying to save data');
console.log(thrownError);
}
});
},
});
The Flask side/the code for /temp_storage looked like...
#app.route('/temp_storage', methods=['GET', 'POST'])
def data_csv_func():
test_trial_json_obj = request.form['experiment_data']
subj_id = request.form['webapp_id']
test_trial_json_obj = json.loads(test_trial_json_obj)
df_data = pd.DataFrame(test_trial_json_obj)
csv_name = "app_dir/static/textdata/" + "appid_" + str(subj_id) + ".csv"
df_data.to_csv(csv_name)
return "Experiment is over!"
#app.route('/experiment_end', methods=['GET', 'POST'])
def show_qualtrics_after_experiment():
return render_template("end.html")
But now I don't want the AJAX call to simply end my experiment. I want my AJAX call to lead to a new webpage within the Flask application that contains a survey, take_qualtrics.html. I also need take_qualtrics.html to have, at the very least, the webapp_id string passed into it (but it would be ideal if I could also pass in the json object for jspsych's data, i.e. test_trials_json_obj). This is in order to link survey responses and web app experiment data
I initially just tried replacing the return statement in data_csv_func() to return render_template("take_qualtrics.html", data={"webapp_id":subj_id, "experiment_data":test_trial_json_obj}), and I deleted/commented out the windows.href("/experiment_end") call. I kept the AJAX call as is (the url was still /temp_storage), I found that this would execute data_to_csv() but the render_template() call wouldn't actually redirect my web page to another url.
So it seems like an AJAX call to a Flask function can't actually trigger redirects on the user's side. And I'm not sure how to work around this currently. I want to have some way of loading take_qualtrics.html and passing it the webapp id (created in javascript) as an argument.
I made a website in python using Django. My site allows you to control lights while indicating if the light is on or not.
I'm looking for a solution that could make a simple request with data to the server and send data back to the client without updating the entire page but only a part of it.
My ideal would be for the client to make a request to the server with identification data. Then, the server returns the updated data that the user is allowed to have.
Is that possible to make a JavaScript to do that ? And the server, how it can return data to the client ?
You can Use jquery AJAX for send request and get a response and Update an element or part of the page you can read more about it in :
W3schools
or :
jquery.com
AJAX function I use for PHP project:(Just for example)
$.ajax({
type: "POST",
url: Url,
data: InfoData, // data if you want to send for example InfoData={dataName: variable} or you can set this = '' if you don't want to send data
datatype: "html",
success: function(result) {
console.log(result);
}
});
I've got a few similar POST request on a website. In one case, the request is working (data is stored on the server), but then, there's an unexpected redirect. I send/receive data using jQuery. On the backend, I use the PHP framework Laravel.
Let's say I'm on myapp.dev/clients/123. Then I click the store-data-button and data is sent to/received from the server (Let's assume $('#resubmission_note').val() === 'abc'):
// AJAX request
$('#store-data-button').on('click', function() {
let ajaxRequest = $.ajax({
url: '/resubmissions',
type: 'POST',
data: {
// An integer, some text, and a date
'client_id': $('#id').html(),
'resubmission_note': $('#resubmission_note').val(),
'resubmission_due_date': $('#resubmission_due_date').val()
}
});
ajaxRequest.done(function(returned_id) {
// Removed all code here for testing. Browser still redirecting.
});
});
But then the browser is redirected to myapp.dev/clients/123?resubmission_note=abc.
The network tab in Chrome devtools says
Name:123?resubmission_note=abc
Status: 200
Type: document
initiator: Other
There's data appended in the URL, that should only be the case with GET request, AFAIK. I checked whether some other JavaScript code might interfere, but couldn't find store-data-button or resubmission_note in any unexpected files.
Any advice on how to fix the problem or how to debug it?
I'm sending a big amount of data with POST using ajax in Laravel.
When I send request rapidly and redirect the page to another page then data is getting lost when ajax call is canceled or aborted.
I've mentioned the process below:
Send ajax request rapidly
Then before responding to latest request of ajax call by browser I redirect to another page
Then I check the data which is storing when ajax call happen. I found the data is lost.
Below is the code
<script>
var cssData = $('.css-textarea').val();
$.ajax({
url: '/save-data',
type: 'POST',
data: {cssData: cssData},
success:function(response){
//do something after success
}
});
</script>
PHP/Laravel Code
public function saveCssData(){
$data = Input::get();
$cssData = $data['cssData'];
//Do something to save data
$db->cssData = $cssData;
$db->save();
}
Please do not assume the exact code because I've given an example.
This code works fine but when I send rapid calls by ajax and before getting a response, I redirect to another page then the data is lost. I've set enough post_max_size and other configurations but still, it's not fixed
Example of Input which I am sending by ajax request
{
"testCss":
" #class{background:#fff}../*big amount data */...
#testDiv{color:#000}
"
}
Example of The Output which I am receiving in database which is lost
{
"testCss":
" #class{background:#fff}../*big amount data */...
#testDiv{col
You can see the data is lost in output. This issue is appearing at sometimes but I noticed that when I request rapidly with ajax and abort the requests by redirecting to another page then it's appearing but don't know how to fix this.
I am developing an app with the node, express, and mongoose. In my login module, res.render() the function sends the code back to the client side ajax call as string format data. Whereas I wanted it to render a particular view. ajax call is post type call. I can see the entire HTML in string format in success field of ajax.
I did search for similar problem and solutions, but I couldn't find any. Let me know what I am doing wrong.
Client Side :
$.ajax({
type: 'POST',
url: '/login',
data: userDetail,
success: function(data){
console.log(data);
},
error: function(err){
$("p").text("Invalid User Details")
}
});
Server Side :
app.post('/login', urlencodedParser ,function(req,res){
console.log(req.body);
User.find({name : req.body.name , password : req.body.pass},function(err,data){
if (data.length != 0){
Todo.find({},function(err,todo){
if (err) throw err;
var token = jwt.sign(req.body,config.secret,{
expiresIn : '1h'
});
res.render('todo',{
todos : todo,
token : token
});
});
}
else
res.status(401).json({ msg : "Invalid User" });
});
});
Ajax calls do not, by themselves, change what is displayed in the browser. They just fetch data and your Javascript code them decides what to do with that data. This is true no matter what type of Ajax call it is, GET, POST, etc...
If you want to change the current page to show the content you fetched with Ajax, then you have to insert that content into the current page yourself.
Or, in the case of a POST, perhaps you want to submit an HTML form and then the browser will render the content that comes back from that form post for you, but an ajax post will not change what the browser displays at all.
Submitting an HTML form can be done either via Javascript or via native user actions (without any Javascript). But, for the browser to process the result for you, it has to be the browser submitting the form, not an ajax call sending the form. If an Ajax call sends the form (as in the code you show), then the result just comes back to your Javascript and it's up to your Javascript to decide what to do with that result (insert it in the page, etc...).