Getting data from python flask to html through Javascript - javascript

I am learning data science but I am still new to flask, html and Js.
I have developed a ML model for home price prediction and would love to deploy it to Heroku.
The problem is the drop down menu in my frontend is not updated by the locations I have passed in my python flask backend.
here are the important parts of my code.
server.py:
from flask import Flask, request, jsonify, render_template
app = Flask(__name__)
#app.route('/locations')
def locations():
response = jsonify({
'locations': get_location_names()
})
response.headers.add('Access-Control-Allow-Origin', '*')
return response
app.js
function onPageLoad() {
console.log( "document loaded" );
$.get("{{ url_for('locations') }}",
function(data, status) {
console.log("got response for locations request");
if(data) {
var locations = data.locations;
var uiLocations = document.getElementById("uiLocations");
$('#uiLocations').empty();
for(var i in locations) {
var opt = new Option(locations[i]);
$('#uiLocations').append(opt);
}
}
});
index.html:
<!DOCTYPE html>
<html>
<head>
<title>Banglore Home Price Prediction</title>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"
type="text/javascript"></script>
<link rel="stylesheet" type= "text/css" href="{{url_for('static', filename = 'app.css')}}">
<script type="text/javascript" src ="{{url_for('static', filename = 'app.js')}}"></script>
</head>
the browser consoles prints "document loaded" which I placed in app.js but doesn't get the data from server.py.
I believe the issue is with the url_for statement but don't know how to go about it.

You can't use jinja2 expressions in a js file which is loaded as a static asset. - v25
You can add your Javascript in a <script> tag in the index.html file. Or you can hard code it.
I usually do not use either approachs. Instead, I render all the files with a custom python script before running the main app. I use a .bat file and type all the commands needed. You sometimes use Sass or any other thing that requires rendering... So it's helpful to be organized and write such a script. Use this approach if your JavaScript data doesn't change dynamically.
But if your script is dynamic, you can add a route that renders your file every time it is requested.
#app.route('/my_script.js')
def script():
return render_template('my_script.js', name='mark')
And in your /locations route:
<script src="{{url_for('script')}}"></script>
Jinja2 can parse any file regardless of it's type.

Related

Link to external file as in css o scripts but with database

What I know
As we all know in HTML files we usually use
<head>
<link rel="stylesheet" type="text/css" href="styles.css">
</head>
to link an external css stylesheet to a html file so this last one will be formatted as we need and also we use
<head>
<script src="somescript.js"></script>
</head>
to make our html file use an external script.
Question
Is it possible to use the same approach by linking into a Viewer.html file an external file (or even more than one) to load from a simple database saved for example as csv, txt, db, json, xml, and so on?
HTML Pseudo code Example:
<head>
<database src="somedata.db"></database>
</head>
Of course, once the data is available to the html file, a js will be used to put it where it has to go, for example into a table contained into the Viewer.html file.
Punctualizzations:
No server of any kind must be involved, just only local files approach.
No frameworks (no jquery, no Node...)
I'm looking an approach that makes use just of html (HTML 5) + javascript (ES6) and the db file (*.csv, *.txt, *.json, *.xml, *.db, ...) containing only utf8 text. The records and fields in it will follow my specifics:
text field 1|text field 2|text field...|text field N
text field 1|text field 2|text field...|text field N
text field 1|text field 2|text field...|text field N
where the pipe symbol | is my custom field separator and the newline is the record separator.
Try this solution if it feel your needs. It make use of json format:
index.html:
<html>
<head>
<script id="jsonDatas" type="application/javascript" src="./datas.json"></script>
<script src="./index.js"></script>
</head>
<body onload="javascript:checkDatas();">
<div id="datas"></div>
</body>
</html>
datas.json:
let datas = [
{"firstname":"alain","lastname":"deseine"},
{"firstname":"robert","lastname":"dupont"},
{"firstname":"john","lastname":"query"},
{"firstname":"albert","lastname":"dumoulin"},
{"firstname":"bob","lastname":"thesponge"}
];
index.js:
function checkDatas(){
console.log('%o', datas);
}
With this solution you will have access of datas in local file datas.json.
UPDATE
This is the only way to achieve what you describe with local files.
Adding let datas... to datas.json is mandatory to make browser loading datas in a javascript var that you can access later. Browsers will blocks every attempt to access src loaded content for security reasons.
You can also use XMLHttpRequest or fetch to achieve what you want, but for this you need to serve your files with an HTTP server (which can be local). HTTP Server is required because file:// protocol is not supported by CORS. And CORS will block every requests originating from another server accordingly to CORS HEADERs that are set.
To bypass CORS with chrome browser, you can start chrome with thiese flags: --disable-web-security --user-data-dir see for details: Disable same origin policy in Chrome There is aloso some chrome extensions that block CORS (not tested). But ATTENTION, these solutions need user actions and they dramatically restrict browser security.
You can load the data using prefetch/preload in the link.
<link id="user_data" rel="preload" as="fetch" crossorigin="anonymous" href="user-data.json">
But your browser will not take any action on it and you can't access the loaded data in the javascript context.
In order to use the data in the javascript context, you have to call the fetch() function with the link URL. This will not load the data twice since the data is already preloaded on the page load.
function readData(selector){
const link = document.querySelector(selector)
if(link){
return fetch(link.href)
.then( res => res.json()) // use res.text() for XML or CSV
}else{
throw 'Invalid selector on loadData(selector)'
}
};
function abc(){
//...
readData('#user_data').then(data => {
console.log(data)
})
//...
}
Alternative solution
You can define your own custom element to get the desired feature
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script src="data-loader.js"> </script>
<!-- this script must be called before the custom element -->
<data-loader id="json" src="data.json" onload="onJSONFetch"></data-loader>
<script src="script.js"> </script>
</head>
<body>
<data-loader id="phone-xml" src="phone-number.xml" onload="onXMLFetch(event)"></data-loader>
<data-loader id="user-data" src="user.csv"></data-loader>
</body>
</html>
data-loader.js
class DataLoader extends HTMLElement {
constructor() {
super();
}
connectedCallback() {
this.load();
}
attributeChangedCallback(name, oldValue, newValue) {
if (name === 'src') {
this.load();
}
}
load() {
const attr = this.getAttribute('src');
if (attr) {
fetch(attr).then(res => res.text()).then((data) => {
const event = new CustomEvent('load', { detail: data })
this.dispatchEvent(event);
const inlineEventHandlerName = this.getAttribute('onload');
const cb = inlineEventHandlerName && window[inlineEventHandlerName];
if((typeof cb) === 'function'){
cb(event);
}
})
}
}
}
customElements.define('data-loader', DataLoader);
script.js
const documentReady = () => {
document.querySelector('#user-data').addEventListener('load', function (event) {
console.log('user-data loaded', event.detail);
})
}
function onXMLFetch(event){
console.log('onXMLFetch', event.detail);
}
function onJSONFetch(event){
console.log('onJSONFetch', event.detail);
}
document.onreadystatechange = () => {
if (document.readyState === 'interactive') {
documentReady();
}
}
Thanks to Alaindeseine's answer I got some inspirations and I did this:
into any text file (so no necessarily *.json) I put these records (notice that the following is a template literal that makes use of back ticks `, not single ' or double " quotes):
var db =
`
my data field | my data field | my data field | my data field
my data field | my data field | my data field | my data field
my data field | my data field | my data field | my data field
`
Also because of the use of var instead of let, I have the advantage that the variable db gets hoisted so I can make use of it everywhere into the file and get its content from any other script or function. Also it can be potentially declared and overwritten multiple times after the previous one is elaborated so it will be also possible to queue multiple db files to show them into the same viewer.html, which is a great advantage, for my case at least.
I already tried this way with different text file types and works great. For example I've tested
*.json
*.tsv
*.csv
*.txt
*.db
*.ext
*(no extension at all)
So at this point I believe that this can also handle other generic files extensions, at the condition they contain text enclosed into a template literal, similarly to the above.
Of course into the html file I have to point/link to the database file as if it was a script like the following:
<script src="someDBfile.ext"></script> // This links the database
<script src="myJavascript.js"></script> //This links the javascript functions that get the data from the **db** variable and create the table/s to show the well formatted data contained into the database.
Nevertheless let's hope browser developers will implement in browsers a standard and more semantically correct way of doing this with an appropriate and semantically correct tag as
<database src="somedata.db"></database>
that works similarly.
Also such procedure allows me to have just one Viewer.html to use to show all compatible files, which is also great.

Accessing scala.js output in resources

I'm trying to build a application server using scala.js, scalatags, akka-http, and mill as my build tool. All goes well until the browser tries to find scripts with generated scala.js code. This is the scalatags code which successfully gets built and references the compiled scala.js code (HiPage.js - built as a ScalaJSModule in mill). When it is run the println below prints out:
file:/Users/nnovod/projects/lims/LIMS/resources/HiPage.js
which is indeed where I've placed the javascript output from scala.js
object HiPage {
val boot =
"Hi().main(document.getElementById('contents'))"
println(getClass.getResource("/HiPage.js").toString)
val skeleton =
html(
head(
script(`type`:="text/javascript", src:="/HiPage.js"),
link(
rel:="stylesheet",
href:="https://cdnjs.cloudflare.com/ajax/libs/pure/0.5.0/pure-min.css"
)
),
body(
onload:=boot,
div(id:="contents")
)
)
}
This eventually shows up in the browser as follows:
<html>
<head>
<script type="text/javascript" src="/HiPage.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/pure/0.5.0/pure-min.css"/>
</head>
<body onload="Hi().main(document.getElementById('contents'))">
<div id="contents"></div>
</body>
</html>
This is my akka-http route...
val route =
path("hello") {
get {
complete(
HttpEntity(
ContentTypes.`text/html(UTF-8)`,
HiPage.skeleton.render
)
)
}
}
The browser can never find the HiPage.js ("Failed to load resource: the server responded with a status of 404 (Not Found)"). HiPage.js is in a top level resources directory and is found by the println(getClass.getResource("/HiPage.js").toString) in the code. What do I have to do to get it seen when the browser requests it from the server?
Not sure if this is the best way but I finally was able to solve the problem by having all src references in script start with /resource and then editing my akka-http route to include the following:
pathPrefix("resource") {
extractUnmatchedPath { unmatched =>
val resource = unmatched.toString()
if (!resource.startsWith("/"))
reject()
else
getFromResource(resource.substring(1))
}
}

How to send RxPy data stream to frontend javascript

I'm trying to get python ReactiveX stream (using RxPy library) to be sent to a javascript on Web UI component, but I can't seem to find a way to do so. Also, I might need to get the data stream coming into the Javascript into a RxJS Observable of sorts for further processing.
Could you please help me understand how to achieve this?
I'm still getting a grip on ReactiveX so maybe there are some fundamental concepts I'm missing, but I'm struggling to find anything similar to this around the net.
This issue has come up as I'm working on a desktop app that takes data from a csv or a zeromq endpoint, and streams it to a UI where the data will be plotted dynamically (updated the plot as new data comes in). I'm using Electron to build my app, using python as my backend code. Python is a must as I will be extending the app with some TensorFlow models.
Following fyears really well made example as an initial structure, I have written some sample code to play with but I can't seem to get it to work.
I manage to get from the UI button all the way to the python scripts, but I get stuck in the return of the PricesApi.get_stream(...) method.
index.html
The front end is straight forward.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Electron Application</title>
</head>
<body>
<button id="super-button">Trigger Python Code</button>
<div id="py-output">
</div>
</body>
<script src="renderer.js" ></script>
</html>
api.py:
The ZeroRPC server file is like the one in the above mentioned link.
import gevent
import json
import signal
import zerorpc
from core_operator import stream
class PricesApi(object):
def get_stream(self, filename):
return stream(filename)
def stop(self):
print('Stopping strategy.')
def echo(self, text):
"""echo any text"""
return text
def load_settings():
with open('settings.json') as json_settings:
settings_dictionary = json.load(json_settings)
return settings_dictionary
def main():
settings = load_settings()
s = zerorpc.Server(PricesApi())
s.bind(settings['address'])
print(f"Initialising server on {settings['address']}")
s.run()
if __name__ == '__main__':
main()
core_operator.py
This is the file were the major logic will sit to get prices from zeroMQ subscription, but currently just creates an Observable from a csv.
import sys
import rx
from csv import DictReader
def prepare_csv_timeseries_stream(filename):
return rx.from_(DictReader(open(filename, 'r')))
def stream(filename):
price_observable = prepare_csv_timeseries_stream(filename)
return price_observable
rendered.js
finally, the javascript that should be receiving the stream:
const zerorpc = require('zerorpc');
const fs = require('fs')
const settings_block = JSON.parse(fs.readFileSync('./settings.json').toString());
let client = new zerorpc.Client();
client.connect(settings_block['address']);
let button = document.querySelector('#super-button');
let pyOutput = document.querySelector('#py-output');
let filename = '%path-to-file%'
button.addEventListener('click', () => {
let line_to_write = '1'
console.log('button click received.')
client.invoke('get_stream', filename, (error, result) => {
var messages = pyOutput;
message = document.createElement('li'),
content = document.createTextNode(error.data);
message.appendChild(content);
messages.appendChild(message);
if(error) {
console.error(error);
} else {
var messages = pyOutput;
message = document.createElement('li'),
content = document.createTextNode(result.data);
message.appendChild(content);
messages.appendChild(message);
}
})
})
I have been looking into using WebSockets, but failed in understanding how to implement it. I did find some examples using Tornado server, however I am trying to keep it as pure as possible and, also, it feels odd that having already a client/server structure from Electron, I'm not able to use that directly.
Also I'm trying to maintain the entire system a PUSH structure as the data requirements don't allow for a PULL type of pattern, with regular pollings etc.
Thank you very much in advance for any time you can dedicate to this, and please let me know if you require any further details or explanations.
I found a solution by using an amazing library called Eel (described as "A little Python library for making simple Electron-like HTML/JS GUI apps"). Its absolute simplicity and intuitiveness allowed me to achieve what I wanted a few simple lines.
Follow the intro to understand the layout.
Then your main python file (which I conveniently named main.py), you expose the stream function to eel, so it can be called from JS file, and pipe the stream into the JavaScript "receive_price" function which is exposed from the JS file!
import sys
import rx
from csv import DictReader
def prepare_csv_timeseries_stream(filename):
return rx.from_(DictReader(open(filename, 'r')))
def process_logic():
return pipe(
ops.map(lambda p: print(p)), # just to view what's flowing through
ops.map(lambda p: eel.receive_price(p)), # KEY FUNCTION in JS file, exposed via eel, is called for each price.
)
#eel.expose # Decorator so this function can get triggered from JavaScript
def stream(filename):
price_observable = prepare_csv_timeseries_stream(filename)
price_observable.pipe(process_logic()).subscribe() # apply the pipe and subscribe to trigger stream
eel.init('web')
eel.start('main.html') # look at how beautiful and elegant this is!
Now we create the price_processing.js file (placed in the 'web' folder as per Eel instructions) to incorporate the exposed functions
let button = document.querySelector('#super-button');
let pyOutput = document.querySelector('#py-output' );
let filename = '%path-to-file%'
console.log("ready to receive data!")
eel.expose(receive_price); // Exposing the function to Python, to process each price
function receive_price(result) {
var messages = pyOutput;
message = document.createElement('li');
content = document.createTextNode(result);
message.appendChild(content);
messages.appendChild(message);
// in here you can add more functions to process data, e.g. logging, charting and so on..
};
button.addEventListener('click', () => {
console.log('Button clicked magnificently! Bloody good job')
eel.stream(filename); // calling the Python function exposed through Eel to start stream.
})
The HTML stays almost the same, apart from the changing the script refs: /eel.js, as per Eel documentation and our price_processing.js file.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Let's try Eel</title>
</head>
<body>
<h1>Eel-saved-my-life: the App!</h1>
<button id="super-button">Trigger Python Code</button>
<div id="py-output">
</div>
</body>
<script type="text/javascript" src="/eel.js"></script>
<script type="text/javascript" src="price_processing.js"></script>
</html>
I hope this can help anyone struggling with the same problem.

Removing image accessibility

I have a web page that is feed from flask a link to a picture. I have it so the image disappears after a second or two. However, if you look at the page source you can still get the picture, via finding the exact url for the picture. How do I remove access to the url or remove the picture from the url so that the user can no longer view the image after it disappears.
Html Page:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Welcome</title>
<script src="//code.jquery.com/jquery-1.11.0.min.js"></script>
</head>
<body>
<script type="text/javascript">
$(document).ready(function () {
var c = document.getElementById('image');
console.log(typeof(c));
setTimeout(function () {
$('#des').css('visibility', 'hidden');
}, 1750);
fs.unlink("{{ url_for('static', filename = '1l1l1lI1lIlIlIlI1IlI1II1l1.jpg') }}");
removeElement(c)
storage.clear(c);
sessionStorage.clear(c);
});
</script>
<div id="des">
<img id="image" src="{{ url_for('static', filename = '1l1l1lI1lIlIlIlI1IlI1II1l1.jpg') }}"/>
</div>
</body>
</html>
main.py
from datetime import datetime
from flask import make_response
from functools import wraps, update_wrapper
from flask import Flask, request, session, g, redirect, url_for, abort, \
render_template, flash
app = Flask(__name__) # create the application instance
app.config.from_object(__name__) # load config from this file , flaskr.py
def nocache(view):
#wraps(view)
def no_cache(*args, **kwargs):
response = make_response(view(*args, **kwargs))
response.headers['Last-Modified'] = datetime.now()
response.headers['Cache-Control'] = 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0, max-age=0'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '-1'
return response
return update_wrapper(no_cache, view)
#app.route(r'/')
#nocache
def landingpage():
return render_template('index.html')
if __name__ == '__main__':
app.run()
First and foremost your images won't be protected if the user tries hard enough. You cannot prevent them for taking screen shots or anything of the sort.
There are several ways to help "prevent" them from getting your images:
You can load your image via ajax:
<script>
$('#my_image').attr('src', {{ url_for('image_path') }});
</script>
You can also load via an expiring hash key (database driven). So create a table with a mapping to the file name and then load the image using a route by passing in the hash key. If they try to open the image from a debugger, it can say it was expired and won't load the image.
def get_image(self, hash_key):
my_image = self.image_engine.get_image(hash_key)
if my_image.get('is_viewed') == 0:
real_key = my_image.get('real_key')
root_path = os.path.join(self.image_path, real_key)
# call a function to set the image as "viewed"
return send_from_directory(root_path, '{}.png'.format(real_key))
else:
return render_template('404.html')
Doing it like this means you can add expiration to the hash key, or even a flag that says it was already viewed and then render nothing, or a 404 if they try to load it again.
This method does require a lot more database calls, but if you are out to protect your images, you will have to look at some performance options.

Display the contents of a log file as it is updated

I have external programs such as ffmpeg and gstreamer running in the background and writing to a log file. I want to display the contents of this log with my Flask application, so that the user can watch the log update, like tail -f job.log would do in the terminal.
I tried to use <object data="/out.log" type="text/plain"> to point at the log file, but that failed to show the data, or the browser told me I needed a plugin.
How can I embed and update the log file in an HTML page?
Use a Flask view to continuously read from the file forever and stream the response. Use JavaScript to read from the stream and update the page. This example sends the entire file, you may want to truncate that at some point to save bandwidth and memory. This example sleeps between reads to reduce cpu load from the endless loop and allow other threads more active time.
from time import sleep
from flask import Flask, render_template
app = Flask(__name__)
#app.route('/')
def index():
return render_template('index.html')
#app.route('/stream')
def stream():
def generate():
with open('job.log') as f:
while True:
yield f.read()
sleep(1)
return app.response_class(generate(), mimetype='text/plain')
app.run()
<pre id="output"></pre>
<script>
var output = document.getElementById('output');
var xhr = new XMLHttpRequest();
xhr.open('GET', '{{ url_for('stream') }}');
xhr.send();
setInterval(function() {
output.textContent = xhr.responseText;
}, 1000);
</script>
This is almost the same as this answer, which describes how to stream and parse messages, although reading from an external file forever was novel enough to be it's own answer. The code here is simpler because we don't care about parsing messages or ending the stream, just tailing the file forever.
I am using frontail package from npm.
npm i frontail -g
frontail /var/log/syslog
visit http://127.0.0.1:9001 to view logs
Source: https://github.com/mthenw/frontail
This may not be the exact answer for the question(to embed an html page), but it solves the problem of many users who are looking specifically only for
Display the contents of a log file as it is updated
For me #davidism solution (accepted answer) worked only on Firefox. It didnt work in Chrome, Brave, Vivaldi. Maybe there was some kind of de-sync in backend and frontend loops? I dont know.
Anyway i used far simpler solution, without loop on the backend and javascript loop on frontend. Maybe it's "uglier" and may cause trouble for some very long logs, but at least it works on every browser i use.
#app.route('/stream')
def stream():
with open("job.log", "r") as f:
content = f.read()
# as you see, file is loaded only once, no loop here, (loop is on frontend side)
return app.response_class(content, mimetype='text/plain')
<!DOCTYPE html>
<html>
<head>
<!-- page auto-refresh every 10 seconds -->
<meta http-equiv="refresh" content="10">
<title>Some title</title>
</head>
<body>
<h1>Log file ...</h1>
<script>
// function for adjusting iframe height to log size
function resizeIframe(obj) {
obj.style.height = obj.contentWindow.document.documentElement.scrollHeight + 'px';
}
</script>
<!-- iframe pulls whole file -->
<iframe src="{{ url_for('stream') }}" frameborder="0" style="overflow:hidden;width:100%" width="100%" frameborder="0" scrolling="no" onload="resizeIframe(this)"></iframe>
</body>
</html>
As you see the only javascript code is used to adjust iframe height to current text size.

Categories