I have a query form that can generate a large json object from data on the server. Ideally, on submit the user should be redirected to the results page (+progress bar) that gets updated from the AJAX request until the results have been generated and are ready to display.
Currently, when the user submits the form, they're left hanging on the queryForm page whilst the results are generated. What's the correct django-way to implement a callback for dataFromQuery once it's completed?
Here is a stripped down version of my class-based view:
class QueryForm(generic.View):
form_class = ReturnQuery
template_name = 'myapp/form/queryForm.html'
def get(self, request, *args, **kwargs):
#render form
return render(request, self.template_name, {'form': form})
def post(self, request, *args, **kwargs):
form = self.form_class(request.POST)
if form.is_valid():
# <process form cleaned data>
print(request.POST)
print(form.cleaned_data)
# continues to build results object (json) which can take
# any length of time
return render(request, 'myapp/form/queryResults.html', {
'dataFromQuery': dataFromQuery,
})
I tried is_ajax() within the POST method:
if request.is_ajax():
results = { "progress" : {'progress':1} }
try:
dataFromQuery
except NameError:
dataFromQuery_exists = False
else:
dataFromQuery_exists = True
results['data']=dataFromQuery
return JsonResponse(results)
but dataFromQuery isn't being passed to the results object once it's complete.
JS:
var refreshIntervalId = setInterval(function(){
$.ajax({type: "POST",url: '/website/queryForm/', data: {csrfmiddlewaretoken : csrftoken}, dataType:'json', success: function(results){ //do something with results }})
});
I've looked at template responses but I'm not sure how they can help here as it looks like the callback is executed once the page render is complete. Any help would be appreciated.
EDIT: I may have been a little unclear. The results.progress is retrieved on successful AJAX, but this line: results['data']=dataFromQuery isn't updating once the results have been generated.
Consider using some task queue/manager, like celery. Just create task that will produce your result, monitor it somehow and display to user current progress of creating response.
HTTP works with request -> response. Django can't 'push' messages (like progress) to the browser without it asking.
What you could do is use polling, which is sending an ajax request every X seconds to get the status.
This solution is not that simple to create, as you'll need a seperate function that returns the progress, and do the actualling progressing in a seperate thread (asynchronous).
depending on the situation, a much easier solution is let your user know that data is being processed, by using a spinner for example.
I am currently developing a very similar case. What you are trying to do is not so trivial. Unfortunately Django is totally synchronous by nature, therefor your AJAX calls will be blocking on the Python side.
As already suggested use some kind of task management tool. I use celery which is easy to use and fully integrated on Django.
Your code will look like this
class QueryForm(generic.View):
...
def post(self, request, *args, **kwargs):
form = self.form_class(request.POST)
token = generate_request_token()
if form.is_valid():
celery_task.apply_async(<token>, <parameters>) <-- this returns immediately
return render(request, 'myapp/form/queryResults.html', {
'token': <token>,
})
Your celery_task is now calculating the results in the background and will store the results once it has finished.
Where you want to store the results is your choice.
You still need to create another Django endpoint to get the result. Map it to something like www.yoursite.com/queryForm/data'. Something like this
def get(self, request, *args, **kwargs):
#return data in JSON format
return JsonResponse(get_data_from_storage(request.<token>))
get_data_from_storage will look into the result storage and check if the results are ready or partially ready. (Another approach is to check the status of the celery_task. When it is done, the results must be ready)
Then on the JS side, as soon as the user submit the QueryForm, he is redirected to www.yoursite.com/queryResults.html/TOKEN (result page), or find a way to pass the token to the result page.
What you can do now is to use TOKEN to query the backend for data belonging to that token. Once inside the result page you do something like.
var refreshIntervalId = setInterval(function(){
$.ajax({type: "GET", url: '/website/queryForm/data', data: {token: <token>}, dataType:'json', success: function(results){ if (results) {// do something} }})
});
The JS approach above is called continuous polling and it is not optimal. (See Socket.io for alternative, but you will need an additional component like Node.js, I went this way). The rendering is finally done on the frontend side. Each time you get new results, you update the page with fresh data.
There are other approaches, but currently this seemed the most straightforward to me.
Alternatives to Celery are:
Threads (if u know what you are doing)
RabbitMQ/ZeroMQ
Tornado (but might not be easy to integrate with Django)
Gevent
Related
This issue comes down to the fact that I haven't yet figured out how to pass data from javascript to Flask/Python via AJAX in the right way.
I have built a Flask application that uses JsPsych as a javascript/front-end framework for an online experiment. jsPsych allows you to create a handler function, on_finish(), that executes once the experiment is finished.
I wanted my experiment's data to be stored on the backend, and that was supposed to be the end o the experiment. I created a function called data_to_csv() that was mounted to the url /temp_storage, that did this. Then I created another url endpoint for /experiment_end, which displays a simple html page stating that the experiment is over and data was saved.
jsPsych.init({
...
on_finish: function() {
var subj_id = jsPsych.data.get().select('user_id').values[0];
var test_trials_collection = jsPsych.data.get();
var test_trial_json = test_trials_collection.json();
$.ajax({
type: "POST",
url: "/temp_storage",
data: {experiment_data:test_trial_json, webapp_id:subj_id},
success: function(response) {
window.location.href="/experiment_end";
},
error: function(xhr, textStatus, thrownError) {
alert('An error occurred while trying to save data');
console.log(thrownError);
}
});
},
});
The Flask side/the code for /temp_storage looked like...
#app.route('/temp_storage', methods=['GET', 'POST'])
def data_csv_func():
test_trial_json_obj = request.form['experiment_data']
subj_id = request.form['webapp_id']
test_trial_json_obj = json.loads(test_trial_json_obj)
df_data = pd.DataFrame(test_trial_json_obj)
csv_name = "app_dir/static/textdata/" + "appid_" + str(subj_id) + ".csv"
df_data.to_csv(csv_name)
return "Experiment is over!"
#app.route('/experiment_end', methods=['GET', 'POST'])
def show_qualtrics_after_experiment():
return render_template("end.html")
But now I don't want the AJAX call to simply end my experiment. I want my AJAX call to lead to a new webpage within the Flask application that contains a survey, take_qualtrics.html. I also need take_qualtrics.html to have, at the very least, the webapp_id string passed into it (but it would be ideal if I could also pass in the json object for jspsych's data, i.e. test_trials_json_obj). This is in order to link survey responses and web app experiment data
I initially just tried replacing the return statement in data_csv_func() to return render_template("take_qualtrics.html", data={"webapp_id":subj_id, "experiment_data":test_trial_json_obj}), and I deleted/commented out the windows.href("/experiment_end") call. I kept the AJAX call as is (the url was still /temp_storage), I found that this would execute data_to_csv() but the render_template() call wouldn't actually redirect my web page to another url.
So it seems like an AJAX call to a Flask function can't actually trigger redirects on the user's side. And I'm not sure how to work around this currently. I want to have some way of loading take_qualtrics.html and passing it the webapp id (created in javascript) as an argument.
I am working on an application where a chatbot asks users questions, takes their answers, runs their answers through a python sklearn model, and outputs the result.
I have an html/javascript/jquery file that has input fields with id='0', id='1',...,id='16'. I am trying to use these values as input to a python sklearn model and return the result back without refreshing the page. I have it setup so when the last chatbot answer is submitted the data from the page is all gotten by the answer id's. I want to post this data as input to a model and return the result of the model back and have the chatbot respond with this result.
Python app.py:
app.route('/')
def index():
return render_template('index.html')
app.route('/getdata', methods=['GET','POST'])
def getdata():
if request.method == 'POST':
print('getting data')
return json['data']
else:
return 'no data'
Javascript
function endMessage() {
var e0 = document.getElementById('1')
var e1 = document.getElementById('2')
var data = [e0,e1]
$.ajax({
type:"POST",
url:"/getdata",
data: JSON.stringify(data),
contentType: 'application/json;charset=UTF-8',
success: function(response) {
console.log(response);
}
});
}
When the last question is submitted, the function endMessage is triggered. This is where I am trying to have ajax make the POST request.
First of all, I cannot get the data to post. My flask app never prints 'getting data' so it never seems that ajax posted the data. Secondly, I am confused how I can take this data pass it to my python model function and send it back to the html file. In my javascript should I use ajax for a "GET" and have a flask function that POSTS to a different route the model results?
This may sound simple, but how do I send the data from a Javascript array in my index.html template to my views.py?
When the user clicks a "Recommend" button, my code calls a function that accesses my database and prints a name on the template.
def index(request):
if(request.GET.get('Recommend')):
sql_handler.recFunc()
context['name'] = sql_handler.name
return render(request, 'polls/index.html', context)
I have an array of checkbox values in Javascript that are calculated after the user presses "Recommend". I want to send it to my index view and use it as the parameter for another function.
So:
def index(request):
if(request.GET.get('Recommend')):
sql_handler.recommend()
context['name'] = sql_handler.name
//something??
tags = check_array_javascript
context['tags'] = tags
return render(request, 'polls/index.html', context)
How can I do this? I've been searching similar questions, but I'm new to Django and web development in general, so I either did not understand the answers or they didn't help me.
Alright, so for sending data from the client (JavaScript) to the backend (your Django app) you need to employ something called Ajax, it stands for Asynchronous JavaScript and XML.
Basically what it does is allowing you to communicate with your backend services without the need of having to reload the page, which, you would have to do using a normal POST or PUT form submission.
The easiest implementation is using jQuery. jQuery is first and foremost a DOM manipulation library but since its inception has grown to encompass much more than that.
A jQuery ajax call looks like this.
$(document).ready(function() {
$.ajax({
method: 'POST',
url: '/path/to/your/view/',
data: {'yourJavaScriptArrayKey': yourJavaScriptArray},
success: function (data) {
//this gets called when server returns an OK response
alert("it worked!");
},
error: function (data) {
alert("it didnt work");
}
});
});
This can then be checked for in your views.py
def index(request):
if request.is_ajax():
#do something
request_data = request.POST
return HttpResponse("OK")
I want to perform an action do file in controllers/static_pages_controller.rb:
def fileopen
my_file = File.new("public/CHNAME1.txt","w")
my_file.write "\tfasf"
my_file.close
end
(it work well when i define it in helper and call it in view.)
in myview.html.erb, i want some thing like <button id="button" onclick="readfile()" />
How can I do that?
I tried in application.js
function readfile() {
alert('readfile work')
$.ajax({
alert('ajax work')
url: "/fileopen",
type: "POST",
##don't know what to do to make fileopen work
}
});
}
routes.rb
match '/fileopen', to:'static_pages#fileopen', via: 'get'
and it's seem nothing happen. Only the first alert work.
In answer to your question directly, you have to be able to handle the JS request in the controller. This is typically done by using the respond_to block in Rails, like this:
def fileopen
respond_to do |format|
format.js {
my_file = File.new("public/CHNAME1.txt","w")
my_file.write "\tfasf"
my_file.close
}
end
end
This code may give you some sort of a response with your current code, but it might be the case that you need to appreciate better how Ajax & Rails work in order to help you better
How Ajax Works
Ajax is a javascript technology which sends an "asynchronous" request to other pages on your website. By their nature, asynchronous requests are done completely independently of your main HTTP request, and basically act like a "pseudo" browser -- working in the background
Ajax is used to pull data from JS-enabled endpoints (which are handled with the respond_to function in Rails, which you can then use to modify your page in some way. A lot of people get confused with Ajax, but it's actually quite simple -- it's just javascript which pulls data from another page, allowing you to manipulate your page with that data
Using Ajax In Your Views
The reason why this is important for you is because you mentioned you didn't know what to do with the success callback of your app. Hopefully my explanation will show you that the success part of the $.ajax call should be used to append the data you receive from the controller on your page
This can be done in this way:
$("#button").click(function() {
$.ajax({
url: "/static_pages/fileopen",
type: "POST",
data: {name: $(this).val()},
success: function (data) {
// append data to your page
$("page_element").html(data);
}
});
});
I'm very new to Django and there's something I'm trying to do that I don't seem to understand how. Right now I have elements of the page show/hide using javascript onclick. What I'm showing includes a dropdown box. What I would like to do is to call a python client side function I wrote, passing the choice and thus making changes to the database. The function is written and I have the frontend working but I don't understand how to take the submit button and get the javascript to not only show/hide elements but end up calling this function.
I don't want the page to refresh which my research so far leads me to believe I need to use AJAX and some sort of POST. Not really sure about that. I've got no experience with this kind of thing. I was wondering if I was on the right track or somewhere that might help me get there/a guide of some kind.
You are correct, you will need to use AJAX. Here is quick example:
template.html
<button type="button" id>Click Me!</button>
{% block inline_js %}
<script type="text/javascript">
$(document).ready(function () {
$(document).on("click",'#button',
function() {
$.ajax({
type: "POST",
data: { action: "delete"},
success: function(data){}
</script>
You can put your AJAX functions directly in your view. If you put nothing in your 'url' argument of your AJAX call, it will call itself (your view that called it).
views.py
def post(self,request, *args, **kwargs):
#Add a AJAX request check. If it is AJAX, redirect to AJAX function
if self.request.is_ajax():
return self.ajax(request)
#===========================================================================
# AJAX
#===========================================================================
def ajax(self, request):
response_dict= {
'success': True,
}
#Your SQL DROP code here...
return HttpResponse(simplejson.dumps(response_dict), mimetype='application/json')
You have to make POST request with JavaScript, and receive it on the server. The basic idea: you send data (JSON, maybe) from the client using XMLHttpRequest or $.ajax (if your project uses jQuery). It is in theory, I am not familiar with Django as well. But according to the docs Django can not handle AJAX by default, so it seems to be what you need: http://www.dajaxproject.com/dajaxice/