So I have a pretty simple python function sitting in the backend as follows:
def iterator():
for i in range(1000):
random_var = randrange(1000)
return jsonify({
"random_var" : random_var,
})
time.sleep(4)
When I run this function standalone, it runs for well a 1000 times very slowly generating a random number every 4 seconds. What I want is to connect this to html frontend via javascript and the code for that is as follows:
var Output_value = document.getElementById('Output_value');
$(function() {
$('#exe-btn').click(function(){
$.ajax({
url: '/display/',
success: function(data) {Output_value.innerHTML = data['random_var'];} });
}
);
});
This does work but not exactly as I expect it to.
Frontend
It displays the value once and then just stops and doesn't display it again unless I press the button again.
What I've researched so far:
The two potential candidates that I found to solve this included the setInterval and setTimeout functions in jquery, however the flaw with that approach is that they require the execution of the backend python function again and again, while what I require is that function to be executed only once.
What I want it to do
I want the backend iterator() to be executed only once and then the values of it to be returned to js so that the newest value is displayed on the html frontend and taking the place of the previous value.
What is happening right now is that the function in the backend is executed upon click but it stops streaming the output after returning one result.
The only way to do this is using websockets as Sergey said. The architecture for that is pretty simple though.
Keep an id in the html which will be used by the javascript to output the data it receives from the flask server. The flask server will use socket io and the javascript file will also use socket. The way it will work is that once the execute button on the frontend is pressed, the js calls an event in the backend which will receive it and then transmit out the data that you want to transmit. This is what the html code looks like:
<div class="Output">
<table>
<th ><h2>Python Library Output:</h2></th>
<th id="Output_value"> </th>
</table>
</div>
<div class= "execute_button">
<button id="exe-btn">Execute</button>
</div>
The exe-btn here is what we bind to jquery, which will call the module execution event in the backend. The JS code:
//Creating the event for execution of backend iterator script
//Setting up the event to be executed when the execute button is clicked and not otherwise
$(function() {
$('#exe-btn').click(function(){
socket.emit('module_trigger', {
data: 'Module event triggered'
})
}
);
});
socket.on( 'module_trigger_stream', function( msg ) {
console.log( 'Response 2')
console.log(msg['add'])
Output_value.innerHTML = msg['add']
//console.log(msg['message'])
}
);
Now the final step is to set up the handlers in the backend and also the method for creating the data. I use a simple random number generator which keeps emitting values after some time:
from flask import Flask, jsonify, render_template
from flask_socketio import SocketIO, send
import math
import time
#module imports
from random import randrange
import json
app = Flask(__name__,template_folder ='templates/')
app.config['SECRET_KEY'] = 'vnkdjnfjknfl1232#'
socketio = SocketIO(app)
#socketio.on('module_trigger')
def handle_module_trigger(jsons, methods=['GET','POST']):
print('Trigger successfully executed')
print(str(jsons))
for i in range(1000):
random_var = randrange(1000)
time.sleep(4)
print("The random variable is " + str(random_var))
out = {'add' : str(random_var)}
socketio.emit('module_trigger_stream',out)
#app.route("/",methods=['GET','POST'])
def render_output():
return render_template("index.html" )
if __name__ == "__main__":
socketio.run(app,debug=True)
Related
I have a website built using the django framework that takes in an input csv folder to do some data processing. I would like to use a html text box as a console log to let the users know that the data processing is underway. The data processing is done using a python function. It is possible for me to change/add text inputs into the text box at certain intervals using my python function?
Sorry if i am not specific enough with my question, still learning how to use these tools!
Edit - Thanks for all the help though, but I am still quite new at this and there are lots of things that I do not really understand. Here is an example of my python function, not sure if it helps
def query_result(request, job_id):
info_dict = request.session['info_dict']
machines = lt.trace_machine(inputFile.LOT.tolist())
return render(request, 'tools/result.html', {'dict': json.dumps(info_dict),
'job_id': job_id})
Actually my main objective is to let the user know that the data processing has started and that the site is working. I was thinking maybe I could display an output log in a html textbox to achieve this purpose.
No cannot do that because you already at server side therefor you cannot touch anything in html page.
You can have 2 ways to do that:
You can make a interval function to call to server and ask the progress and update the progress like you want at callback function.
You can open a socket connection in your server & browser to instantly update.
While it is impossible for the server (Django) to directly update the client (browser), you can you JavaScript to make the request, and Django can return a StreamingHttpResponse. As each part of the response is received, you can update the textbox using JavaScript.
Here is a sample with pseudo code
def process_csv_request(request):
csv_file = get_csv_file(requests)
return StreamingHttpResponse(process_file(csv_file))
def process_file(csv_file):
for row in csv_file:
yield progress
actual_processing(row)
return "Done"
Alternatively you could write the process to the db or some cache, and call an API that returns the progress repeatedly from the frontend
You can achieve this with websockets using Django Channels.
Here's a sample consumer:
class Consumer(WebsocketConsumer):
def connect(self):
self.group_name = self.scope['user']
print(self.group_name) # use this for debugging not sure what the scope returns
# Join group
async_to_sync(self.channel_layer.group_add)(
self.group_name,
self.channel_name
)
self.accept()
def disconnect(self, close_code):
# Leave group
async_to_sync(self.channel_layer.group_discard)(
self.group_name,
self.channel_name
)
def update_html(self, event):
status = event['status']
# Send message to WebSocket
self.send(text_data=json.dumps({
'status': status
}))
Running through the Channels 2.0 tutorial you will learn that by putting some javascript on your page, each time it loads it will connect you to a websocket consumer. On connect() the consumer adds the user to a group. This group name is used by your csv processing function to send a message to the browser of any users connected to that group (in this case just one user) and update the html on your page.
def send_update(channel_layer, group_name, message):
async_to_sync(channel_layer.group_send)(
group_name,
{
'type': 'update_html',
'status': message
}
)
def process_csv(file):
channel_layer = get_channel_layer()
group_name = get_user_name() # function to get same group name as in connect()
with open(file) as f:
reader=csv.reader(f)
send_update(channel_layer, group_name, 'Opened file')
for row in reader:
send_update(channel_layer, group_name, 'Processing Row#: %s' % row)
You would include javascript on your page as outlined in the Channels documentation then have an extra onmessage function fro updating the html:
var WebSocket = new ReconnectiongWebSocket(...);
WebSocket.onmessage = function(e) {
var data = JSON.parse(e.data);
$('#htmlToReplace').html(data['status']);
}
I have a CGI script which runs on local host. It has a Python function named compute(argument 1) and prints html tags along with a JavaScript function called onClick which gets called when a button is clicked on the browser. I am not sure how to communicate between JavaScript and Python here.
Here is a snippet of the CGI file
import cgi
def worker():
#Does something here and returns an python list called data
def compute(index)
#Does something with data[index] and returns another python list called data2
print ("Content-type: text/html")
print ()
print ("<html>")
print ("<head>")
print ("<title>Worker</title>")
print("""<script type = "text/javascript">
var count = 0;
function onClick(){
count = count+1;
document.getElementById("dis").innerHTML = count;
};
</script>""")
print ("</head>")
print ("<body>")
<button type = "button" class = "next" onclick = "onClick()" >Next → </button>
print("""Count: <p id = "dis" > 0 </p>""")
print ("</body>")
print ("</html>")
My question here is how do I make the JavaScript function OnClick() call the python function compute() by passing variable count as a parameter and then storing the resulting list returned by compute() as a JavaScript variable. It would be better if worker function does not have to be called again. It would be fine to even return a single variable in compute(index) rather than a python list and pass it to JavaScript. Thanks in advance :).
AJAX/JQUERY can send the data, JSON for instance, to your python endpoint. But your script is not a server. You need a server a.k.a middleware like apache. POST via AJAX to your server adress:
$(function(){
$.ajax({
url: "ServerAdress/cgi-bin/" + service.py,
type: "POST",
async:true,
data: {key_vehicule: 'mercedes'},
success: function(response){
},
error: function(){
},
complete: function(){
}
});
Be sure to have your server configured to execute the python script with the python interpeter and all the right it needs. In the python script, just importing the cgi is not enought, you need the to receive the data:
form = cgi.FieldStorage()
vehicule = form.getvalue('key_vehicule')
I have a python program and I want to print the output of that program on an HTML page using Node JS. The python script is being called using 'child_process'. I want to print the value of 't' on HTML page. Is there any way to do it?
script.py
import time
t=0
def main():
while 1:
t =t+1
print t
time.sleep(2)
# Start process
if __name__ == '__main__':
main()
Here I want to print the value of 't' after every 2 seconds.
app.js
var sys =require('sys');
var myPythonScript ="script.py";
var path =resolve("C:\\Python27\\python.exe");
var pythonExecutable = path;
const spawn = require('child_process').spawn;
const scriptExecution = spawn(pythonExecutable, [myPythonScript]);
scriptExecution.stdout.on('data', (data) => {
console.log(data);
});
});
This JS function is called when a button is clicked on the HTML page.
I have an idea for you- you can create a simple flask app (http://flask.pocoo.org/), that will listen to a specific port.
When this app will receive a REST API request, it will execute your's python code and return the result as a response.
In the javascript side, you will only need to create an ajax request which will provide the request to the flask app.
In this way, you can mix both python and javascript comfortably.
you can use urllib to send the output of your .py file :
import urllib
output="test"
url="post.php"
param=urllib.urlencode({'output':output})
urllib.urlopen(url,param)
to get the post in PHP :
<?php
$output = $_POST["output"];
echo $output;
?>
to get the post parameter in Node.js you can read this :
https://scotch.io/tutorials/use-expressjs-to-get-url-and-post-parameters
How do you extract POST data in Node.js?
https://dzone.com/articles/get-post-parameter-nodejs
Yes, you can code on python then using rapydscript and npm you can convert python code to Js code and it will print on the HTML page. The best part of rapydscript is, it doesn't need any server.
For example:
Below code is an alert function in python:
def greet():
alert("Hello World!")
This will convert to below Js code:
function greet() {
alert("Hello World!");
}
https://www.npmjs.com/package/rapydscript-ng
I'm trying to find out how to create a local connection between a Python server and a Javascript client using the JSON format for the data to be retrieved. Particularly, I need to make some queries on the HTML client side, send these queries to the server on JSON format and run them on the Python server side to search for data on a SQLite Database. And after getting the results from the database, send those results back to the client in JSON format too.
By now, I just can run the query on Python and code it on JSON like this:
import sqlite3 as dbapi
import json
connection = dbapi.connect("C:/folder/database.db")
mycursor = connection.cursor()
mycursor.execute("select * from people")
results = []
for information in mycursor.fetchall():
results += information
onFormat = json.dumps(results)
print(onFormat)
I know this code does something alike (in fact it runs), because it calls a service on a server which returns data in JSON format (but the server in this example is NOT Python):
<html>
<head>
<style>img{ height: 100px; float: left; }</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
</head>
<body>
<div id="images"></div>
<script>
$.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?",
{
tags: "mount rainier",
tagmode: "any",
format: "json"
},
function(data) {
$.each(data.items, function(i,item){
$("<img/>").attr("src", item.media.m).appendTo("#images");
if ( i == 3 ) return false;
});
});</script>
</body>
</html>
What I need is to know how should I run (locally) the python program to be an available running web-service and how should be the Javascript to retrieve the data from the python server.
I've looking for this on internet everywhere but I didn't find this answer anywhere because the only answers they give are on how to code JSON inside Python or inside Javascript but not connecting both. Hope somebody can help me on this!!!
Here's a "hello world" example of a flask web-application that can serve static html and javascript files, search database using parameter from a javascript request, and return results to javascript as json:
import sqlite3
from flask import Flask, jsonify, g, redirect, request, url_for
app = Flask(__name__)
#app.before_request
def before_request():
g.db = sqlite3.connect('database.db')
#app.teardown_request
def teardown_request(exception):
if hasattr(g, 'db'):
g.db.close()
#app.route('/')
def index():
return redirect(url_for('static', filename='page.html'))
#app.route('/json-data/')
def json_data():
# get number of items from the javascript request
nitems = request.args.get('nitems', 2)
# query database
cursor = g.db.execute('select * from items limit ?', (nitems,))
# return json
return jsonify(dict(('item%d' % i, item)
for i, item in enumerate(cursor.fetchall(), start=1)))
if __name__ == '__main__':
app.run(debug=True, host='localhost', port=5001) # http://localhost:5001/
else:
application = app # for a WSGI server e.g.,
# twistd -n web --wsgi=hello_world.application --port tcp:5001:interface=localhost
The database setup code is from Using SQLite 3 with Flask.
static/page.html and static/json-jquery.js files are from Ajax/jQuery.getJSON Simple Example, where the javascript code is modified slightly to pass a different url and nitems parameter:
$(document).ready(function(){
$('#getdata-button').live('click', function(){
$.getJSON('/json-data', {'nitems': 3}, function(data) {
$('#showdata').html("<p>item1="+data.item1+" item2="+data.item2+" item3="+data.item3+"</p>");
});
});
});
Your question amounts to "how do I make this python into a webservice".
Probably the most lightweight ways to do that are web.py and flask. Check them out.
If this is getting bigger, consider django with tastypie - that's a simple way to make a json-based api.
Update: Apparently, there is also a python-javascript RPC framework called Pico, to which Felix Kling is a contributor. The intro says:
Literally add one line of code (import pico) to your Python module to
turn it into a web service that is accessible through the Javascript
(and Python) Pico client libararies.
I found finally an easier way than Flask. It's a Python framework called Bottle You only need to download the library from the official web site and put all its files in your working directory in order to import the library. You can also install it using the setup python program included to avoid carrying with the sourcecode everywhere. Then, for making your Web Service Server you can code it like this:
from bottle import hook, response, route, run, static_file, request
import json
import socket
import sqlite3
#These lines are needed for avoiding the "Access-Control-Allow-Origin" errors
#hook('after_request')
def enable_cors():
response.headers['Access-Control-Allow-Origin'] = '*'
#Note that the text on the route decorator is the name of the resource
# and the name of the function which answers the request could have any name
#route('/examplePage')
def exPage():
return "<h1>This is an example of web page</h1><hr/><h2>Hope you enjoy it!</h2>"
#If you want to return a JSON you can use a common dict of Python,
# the conversion to JSON is automatically done by the framework
#route('/sampleJSON', method='GET')
def mySample():
return { "first": "This is the first", "second": "the second one here", "third": "and finally the third one!" }
#If you have to send parameters, the right sintax is as calling the resoure
# with a kind of path, with the parameters separed with slash ( / ) and they
# MUST to be written inside the lesser/greater than signs ( <parameter_name> )
#route('/dataQuery/<name>/<age>')
def myQuery(name,age):
connection= sqlite3.connect("C:/folder/data.db")
mycursor = connection.cursor()
mycursor.execute("select * from client where name = ? and age= ?",(name, age))
results = mycursor.fetchall()
theQuery = []
for tuple in results:
theQuery.append({"name":tuple[0],"age":tuple[1]})
return json.dumps(theQuery)
#If you want to send images in jpg format you can use this below
#route('/images/<filename:re:.*\.jpg>')
def send_image(filename):
return static_file(filename, root="C:/folder/images", mimetype="image/jpg")
#To send a favicon to a webpage use this below
#route('/favicon.ico')
def favicon():
return static_file('windowIcon.ico', root="C:/folder/images", mimetype="image/ico")
#And the MOST important line to set this program as a web service provider is this
run(host=socket.gethostname(), port=8000)
Finally, you can call the REST web service of your Bottlepy app on a Javascript client in this way:
var addr = "192.168.1.100"
var port = "8000"
function makeQuery(name, age){
jQuery.get("http://"+addr+":"+port+"/dataQuery/"+ name+ "/" + age, function(result){
myRes = jQuery.parseJSON(result);
toStore= "<table border='2' bordercolor='#397056'><tr><td><strong>name</strong></td><td><strong>age</strong></td></tr>";
$.each(myRes, function(i, element){
toStore= toStore+ "<tr><td>"+element.name+"</td><td>" + element.age+ "</td></td></tr>";
})
toStore= toStore+ "</table>"
$('#theDataDiv').text('');
$('<br/>').appendTo('#theDataDiv');
$(toStore).appendTo('#theDataDiv');
$('<br/>').appendTo('#theDataDiv');
})
}
I hope it could be useful for somebody else
For example:
#!/usr/bin/python
print "This is python."
print "<script type="text/javascript">
var pass_to_python = new Number(7)
</script>"
the_number = pass_to_python???
How do I get the pass_to_python in python?
With pyv8 you can execute javascript from within Python.
import PyV8
class Global(PyV8.JSClass):
pass
with PyV8.JSContext(Global()) as ctxt:
the_number = ctxt.eval("var pass_to_python = new Number(7)")
see http://code.google.com/p/pyv8/
You can GET or POST to the Python script. If you need to do this dynamically, you can use AJAX.
Here is a good link: How are POST and GET variables handled in Python?
i am using flask and ajax to pass values from javacript to python
function pass_values() {
var pass_to_python = new Number(7)
$.ajax(
{
type:'POST',
contentType:'application/json;charset-utf-08',
dataType:'json',
url:'http://127.0.0.1:5000/pass_val?value='+pass_to_python ,
success:function (data) {
var reply=data.reply;
if (reply=="success")
{
return;
}
else
{
alert("some error ocured in session agent")
}
}
}
);
}
python:
#app.route('/pass_val',methods=['POST'])
def pass_val():
name=request.args.get('value')
print('name',name)
return jsonify({'reply':'success'})
HTTP is a simple request-response protocol, it doesn't let you pause mid-stream and wait for more information from the client — and since your JS runs in the browser (JS can run on the server, but most people wouldn't be attempting this if they didn't need the code to run in the browser, so I'm assuming that using server side JS is out of the question) and the Python runs on the server, that is what you need for your code to work (as well as fixing your broken quote nesting in the Python code).
You need to load the complete document, and then issue a new HTTP request.
This might involve having the JS set location.href (making sure you have a fallback for non-JS clients), it might involve using XMLHttpRequest to load new data asynchronously, it might be best using another technique (it is hard to say for sure as your example simplifies too much to tell what X is)
I think using JSON is the best way.you can create a JSON file as intermidiary between JavaScript and Python, both languages can access and modify JSON file