So, I am trying to use telethon,quart and websockets to create a dynamic page that only receives updates when the client receives a message event:
from telethon import TelegramClient, connection, events, utils
import hypercorn.asyncio
from quart import Quart, websocket
from variables import api_id, api_hash
from datetime import datetime
import asyncio
# Session name, API ID and hash to use; loaded from environmental variables
SESSION = 'quart';
# Telethon client
client = TelegramClient(SESSION, api_id, api_hash)
client.parse_mode = 'html'
###paginas html
teste_html = """
<!doctype html>
<html>
<head>
<title>Quart + Telethon</title>
<link rel="stylesheet" type="text/css" href="/static/style.css">
</head>
<body>
<h1>Página de teste!</h1>
<p>No momento, estamos testando o websocket</p>
<ul></ul>
<script type="text/javascript">
let socket = new WebSocket('ws://localhost:8000/random');
socket.onmessage = function(event) {
var messages_dom = document.getElementsByTagName('ul')[0];
var message_dom = document.createElement('li');
var cotent_dom = document.createTextNode('Received: ' + event.data);
message_dom.appendChild(cotent_dom);
messages_dom.appendChild(message_dom);
};
</script>
</body>
</html>
"""
###
app = Quart(__name__)
lock = asyncio.Lock();
#app.before_serving
async def startup():
await client.connect()
#app.after_serving
async def cleanup():
await client.disconnect()
#client.on(events.NewMessage)
async def receber(event):
global x;
try:
sender = await event.get_sender();
name = utils.get_display_name(sender);
message = name + "::::::" + event.text + "\n";
await lock.acquire();
x = message;
try:
file = open('msg.txt', '+a');
file.write(message);
file.close();
finally:
lock.release()
except(KeyboardInterrupt):
print("Adeus!");
except Exception as e:
print("\n======ERRO======\n");
print(e);
###websockets
#app.websocket('/random')
async def random():
global x;
import random
from time import sleep
try:
while True:
if x != 1:
print(x);
data = random.randint(0,5);
data = str(data);
await websocket.send(f"Quart enviou 1: {data}");
else:
besteira = 0;
sleep(1);
except Exception as e:
print(e);
print("erro 2");
#app.route('/')
async def hello_world():
await client.send_message('me', 'Hello World')
#return 'Message Sent!'
return teste_html;
async def main():
await hypercorn.asyncio.serve(app, hypercorn.Config())
if __name__ == '__main__':
x = 1;
try:
client.loop.run_until_complete(main())
except KeyboardInterrupt:
print("Adeus!");
except Exception as e:
print(e);
The idea is that, when a first new message is received, the websocket will start to send random numbers to the page (I am begginer), however, it just, locks, it not even writes on the msg.txt file, and here is the thin, this only happens if you use the if statement. If you take away the if statement, it runs, slow, but it runs, but if you put the if statement, it just locks.
If you can't help me, can you suggest me any quart/telethon/websocket tutorial? Or an example? If not,the is any other way to do this? Any better way to create a dynamic page?
Any help is very welcome.
The solution is that I am idiot, I was using synchronous sleep, I don't know exactly how this was breaking stuff, but it was, here is the fixed API:
#app.websocket('/random')
async def random():
global mensagem;
import random
#from time import sleep
mensagem_antiga = "";
try:
for i in range(1, 1000):
if mensagem_antiga != mensagem:
await websocket.send(f" {mensagem}");
mensagem_antiga = mensagem;
await asyncio.sleep(1);
except Exception as e:
print(e);
print("erro 2");
First time poster here. I am trying to put together a chatroom app using Flask-SocketIO and am having a hard time getting one tiny, but crucial, piece to work. When I call the emit method in the server and set the broadcast=True flag, the expected behavior is that it triggers the respective socket.on events on the client side for all users, including the sender. However, my code apparently only does so for all clients other than the sender. This is contrary to the documentation at https://flask-socketio.readthedocs.io/en/latest/, where it states:
When a message is sent with the broadcast option enabled, all clients connected to the namespace receive it, including the sender. When namespaces are not used, the clients connected to the global namespace receive the message.
I need for this to send the message and trigger the socket.on event for the sender as well as all other clients. Can anyone explain what I'm doing wrong here? Because it seems like such a basic feature that I must be missing something really obvious.
Server code:
import os
import requests
from flask import Flask, jsonify, render_template, request
from flask_socketio import SocketIO, emit, join_room, leave_room
app = Flask(__name__)
app.config['SECRET_KEY'] = os.getenv('SECRET_KEY')
socketio = SocketIO(app)
channels = ['a', 'b', 'c', 'testing', 'abc', '123']
ch_msgs = {'a': ['testing', '1234', 'high five'], 'b': ['hello']}
msg_limit = 100
#app.route('/')
def index():
return render_template('index.html')
#socketio.on('add channel')
def add(data):
new_channel = data
channels.append(new_channel)
ch_msgs[new_channel] = list()
emit('announce channel', new_channel, broadcast=True)
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
Javascript:
document.addEventListener('DOMContentLoaded', () => {
var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
socket.on('connect', () => {
document.querySelector('#add_channel').onclick = () => {
const new_channel = document.querySelector('#new_channel').value;
socket.emit('add channel', new_channel);
};
});
socket.on('announce channel', data => {
const li = document.createElement('li');
li.innerHTML = '#' + `${data}`;
$('#channel_list').append(li);
});
});
HTML/CSS snippet:
<h3>Chat Channels</h3>
<br>
<ul id='channel_list' style='list-style-type:none;'>
</ul>
<form class='channel'>
<input type='text' id='new_channel' placeholder='add new channel'><input type='submit' id='add_channel' value='Add'>
</form>
All clients in fact are receiving the message sent, including the sender, but the issue here is that the sender's page is refreshed as soon as the form is submitted, (which is the default behavior of forms) thus resulting in loss of all local data.
In order to prevent this, you need to use event.preventDefault() method. In your case, you may change the event handler for connect event like this:
socket.on('connect', () => {
document.querySelector('#add_channel').onclick = event => {
event.preventDefault();
const textinput = document.querySelector('#new_channel')
const new_channel = textinput.value;
textinput.value = ""; // Clear input after value is saved
socket.emit('add channel', new_channel);
};
});
I want to refresh my web page every time a file is uploaded to the folder.
I have a web service written in flask having the following handler
#app.route('/getlatest/')
def getlatest():
import os
import glob
newset = max(glob.iglob('static/*'),key=os.path.getctime)
Return newest;
This gives me the name of the latest file in the folder.
I have an Ajax call in my JS (client side) to constantly get data from above function.
function GetLatest()
{
$.ajax({
url: "http://localhost:5000/getlatest",
success: function(result)
{
if(previousName != result){
previousName = result;
$("#image").attr("src","/"+previousName);
}
}
});
}
function calling server every second.
(function myLoop (i) {
setTimeout(function () {
GetLatest();
if (--i) myLoop(i);
}, 1000)
})(100);
This Works fine [well almost].
My Question is: Is there any better way to do it[ there must be ]?
i'm open to technology choices what every they may be [node, angualr etc.]
yea you can use websockets(flask-socketio).That will let you to have an open connection between you and server and every time in the folder is a new photo will be displayed on a selected div
http://flask-socketio.readthedocs.io/en/latest/
https://pypi.python.org/pypi/Flask-SocketIO/2.9.1
So here is how i did it.
first of all thanks to
https://blog.miguelgrinberg.com/post/easy-websockets-with-flask-and-gevent
for explaining it perfectly.
What i have learnt reading a couple of blogs.
All communication initiated using http protocol is client server communication, and client is always the initiator. So in this case we have to use a different protocol: Web-sockets which allow you to create a full-duplex (two way) connection.
Here is the server code;
socketio = SocketIO(app, async_mode=async_mode)
thread = None
prevFileName = ""
def background_thread():
prevFileName = ""
while True:
socketio.sleep(0.5)
if(isNewImageFileAdded(prevFileName) == "true"):
prevFileName = getLatestFileName()
socketio.emit('my_response',
{'data': prevFileName, 'count': 0},
namespace='/test');
def getLatestFileName():
return max(glob.iglob('static/*'),key=os.path.getctime)
def isNewImageFileAdded(prevFileName):
latestFileName = max(glob.iglob('static/*'),key=os.path.getctime)
if(prevFileName == latestFileName):
return "false"
else:
return "true"
Creating separate thread to keep the socket open. emit sends the msg to the client from server...
#socketio.on('connect', namespace='/test')
def test_connect():
global thread
if thread is None:
thread = socketio.start_background_task(target=background_thread)
emit('my_response', {'data': 'Connected', 'count': 0})
And here is Client Side.[replaced ajax call with the following code]
var socket = io.connect(location.protocol + '//' + document.domain + ':' +
location.port + namespace);
socket.on('my_response', function(msg) {
setTimeout(function(){ $("#image").attr("src","/"+msg.data)},1000)
});
Please correct me if i am wrong.
I am trying to write a realtime web app which can respond real time messages updating in Flask framework. I am using code from http://flask.pocoo.org/snippets/116/ but JavaScript EventSource SSE not firing in browser. From the log I can see I've published the data successfully but the webpage(http:xxxx:5000/) does not get updated at all.
import gevent
from gevent.wsgi import WSGIServer
from gevent.queue import Queue
from flask import Flask, Response
import time
# SSE "protocol" is described here: http://mzl.la/UPFyxY
class ServerSentEvent(object):
def __init__(self, data):
self.data = data
self.event = None
self.id = None
self.desc_map = {
self.data : "data",
self.event : "event",
self.id : "id"
}
def encode(self):
if not self.data:
return ""
lines = ["%s: %s" % (v, k)
for k, v in self.desc_map.iteritems() if k]
return "%s\n\n" % "\n".join(lines)
app = Flask(__name__)
subscriptions = []
# Client code consumes like this.
#app.route("/")
def index():
debug_template = """
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<h1>Server sent events</h1>
<div id="event"></div>
<script type="text/javascript">
var eventOutputContainer = document.getElementById("event");
var evtSrc = new EventSource("/subscribe");
evtSrc.onmessage = function(e) {
console.log(e.data);
eventOutputContainer.innerHTML = e.data;
};
</script>
</body>
</html>
"""
return(debug_template)
#app.route("/debug")
def debug():
return "Currently %d subscriptions" % len(subscriptions)
#app.route("/publish")
def publish():
#Dummy data - pick up from request for real data
def notify():
msg = str(time.time())
for sub in subscriptions[:]:
sub.put(msg)
print 'data is ' + str(time.time())
gevent.spawn(notify)
return "OK"
#app.route("/subscribe")
def subscribe():
def gen():
q = Queue()
subscriptions.append(q)
try:
while True:
result = q.get()
ev = ServerSentEvent(str(result))
print 'str(result) is: ' + str(result)
yield ev.encode()
except GeneratorExit: # Or maybe use flask signals
subscriptions.remove(q)
return Response(gen(), mimetype="text/event-stream")
if __name__ == "__main__":
app.debug = True
server = WSGIServer(("", 5001), app)
server.serve_forever()
# Then visit http://localhost:5000 to subscribe
# and send messages by visiting http://localhost:5000/publish
Can you please shed some lights? I am testing with Chrome/34.0.1847.116. Thanks.
I'm looking at the following API:
http://wiki.github.com/soundcloud/api/oembed-api
The example they give is
Call:
http://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=json
Response:
{
"html":"<object height=\"81\" ... ",
"user":"Forss",
"permalink":"http:\/\/soundcloud.com\/forss\/flickermood",
"title":"Flickermood",
"type":"rich",
"provider_url":"http:\/\/soundcloud.com",
"description":"From the Soulhack album...",
"version":1.0,
"user_permalink_url":"http:\/\/soundcloud.com\/forss",
"height":81,
"provider_name":"Soundcloud",
"width":0
}
What do I have to do to get this JSON object from just an URL?
It seems they offer a js option for the format parameter, which will return JSONP. You can retrieve JSONP like so:
function getJSONP(url, success) {
var ud = '_' + +new Date,
script = document.createElement('script'),
head = document.getElementsByTagName('head')[0]
|| document.documentElement;
window[ud] = function(data) {
head.removeChild(script);
success && success(data);
};
script.src = url.replace('callback=?', 'callback=' + ud);
head.appendChild(script);
}
getJSONP('http://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=js&callback=?', function(data){
console.log(data);
});
A standard http GET request should do it. Then you can use JSON.parse() to make it into a json object.
function Get(yourUrl){
var Httpreq = new XMLHttpRequest(); // a new request
Httpreq.open("GET",yourUrl,false);
Httpreq.send(null);
return Httpreq.responseText;
}
then
var json_obj = JSON.parse(Get(yourUrl));
console.log("this is the author name: "+json_obj.author_name);
that's basically it
In modern-day JS, you can get your JSON data by calling ES6's fetch() on your URL and then using ES7's async/await to "unpack" the Response object from the fetch to get the JSON data like so:
const getJSON = async url => {
const response = await fetch(url);
if(!response.ok) // check if response worked (no 404 errors etc...)
throw new Error(response.statusText);
const data = response.json(); // get JSON from the response
return data; // returns a promise, which resolves to this data value
}
console.log("Fetching data...");
getJSON("https://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=json").then(data => {
console.log(data);
}).catch(error => {
console.error(error);
});
The above method can be simplified down to a few lines if you ignore the exception/error handling (usually not recommended as this can lead to unwanted errors):
const getJSON = async url => {
const response = await fetch(url);
return response.json(); // get JSON from the response
}
console.log("Fetching data...");
getJSON("https://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=json")
.then(data => console.log(data));
Because the URL isn't on the same domain as your website, you need to use JSONP.
For example: (In jQuery):
$.getJSON(
'http://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=js&callback=?',
function(data) { ... }
);
This works by creating a <script> tag like this one:
<script src="http://soundcloud.com/oembed?url=http%3A//soundcloud.com/forss/flickermood&format=js&callback=someFunction" type="text/javascript"></script>
Their server then emits Javascript that calls someFunction with the data to retrieve.
`someFunction is an internal callback generated by jQuery that then calls your callback.
DickFeynman's answer is a workable solution for any circumstance in which JQuery is not a good fit, or isn't otherwise necessary. As ComFreek notes, this requires setting the CORS headers on the server-side. If it's your service, and you have a handle on the bigger question of security, then that's entirely feasible.
Here's a listing of a Flask service, setting the CORS headers, grabbing data from a database, responding with JSON, and working happily with DickFeynman's approach on the client-side:
#!/usr/bin/env python
from __future__ import unicode_literals
from flask import Flask, Response, jsonify, redirect, request, url_for
from your_model import *
import os
try:
import simplejson as json;
except ImportError:
import json
try:
from flask.ext.cors import *
except:
from flask_cors import *
app = Flask(__name__)
#app.before_request
def before_request():
try:
# Provided by an object in your_model
app.session = SessionManager.connect()
except:
print "Database connection failed."
#app.teardown_request
def shutdown_session(exception=None):
app.session.close()
# A route with a CORS header, to enable your javascript client to access
# JSON created from a database query.
#app.route('/whatever-data/', methods=['GET', 'OPTIONS'])
#cross_origin(headers=['Content-Type'])
def json_data():
whatever_list = []
results_json = None
try:
# Use SQL Alchemy to select all Whatevers, WHERE size > 0.
whatevers = app.session.query(Whatever).filter(Whatever.size > 0).all()
if whatevers and len(whatevers) > 0:
for whatever in whatevers:
# Each whatever is able to return a serialized version of itself.
# Refer to your_model.
whatever_list.append(whatever.serialize())
# Convert a list to JSON.
results_json = json.dumps(whatever_list)
except SQLAlchemyError as e:
print 'Error {0}'.format(e)
exit(0)
if len(whatevers) < 1 or not results_json:
exit(0)
else:
# Because we used json.dumps(), rather than jsonify(),
# we need to create a Flask Response object, here.
return Response(response=str(results_json), mimetype='application/json')
if __name__ == '__main__':
##NOTE Not suitable for production. As configured,
# your Flask service is in debug mode and publicly accessible.
app.run(debug=True, host='0.0.0.0', port=5001) # http://localhost:5001/
your_model contains the serialization method for your whatever, as well as the database connection manager (which could stand a little refactoring, but suffices to centralize the creation of database sessions, in bigger systems or Model/View/Control architectures). This happens to use postgreSQL, but could just as easily use any server side data store:
#!/usr/bin/env python
# Filename: your_model.py
import time
import psycopg2
import psycopg2.pool
import psycopg2.extras
from psycopg2.extensions import adapt, register_adapter, AsIs
from sqlalchemy import update
from sqlalchemy.orm import *
from sqlalchemy.exc import *
from sqlalchemy.dialects import postgresql
from sqlalchemy import Table, Column, Integer, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
class SessionManager(object):
#staticmethod
def connect():
engine = create_engine('postgresql://id:passwd#localhost/mydatabase',
echo = True)
Session = sessionmaker(bind = engine,
autoflush = True,
expire_on_commit = False,
autocommit = False)
session = Session()
return session
#staticmethod
def declareBase():
engine = create_engine('postgresql://id:passwd#localhost/mydatabase', echo=True)
whatever_metadata = MetaData(engine, schema ='public')
Base = declarative_base(metadata=whatever_metadata)
return Base
Base = SessionManager.declareBase()
class Whatever(Base):
"""Create, supply information about, and manage the state of one or more whatever.
"""
__tablename__ = 'whatever'
id = Column(Integer, primary_key=True)
whatever_digest = Column(VARCHAR, unique=True)
best_name = Column(VARCHAR, nullable = True)
whatever_timestamp = Column(BigInteger, default = time.time())
whatever_raw = Column(Numeric(precision = 1000, scale = 0), default = 0.0)
whatever_label = Column(postgresql.VARCHAR, nullable = True)
size = Column(BigInteger, default = 0)
def __init__(self,
whatever_digest = '',
best_name = '',
whatever_timestamp = 0,
whatever_raw = 0,
whatever_label = '',
size = 0):
self.whatever_digest = whatever_digest
self.best_name = best_name
self.whatever_timestamp = whatever_timestamp
self.whatever_raw = whatever_raw
self.whatever_label = whatever_label
# Serialize one way or another, just handle appropriately in the client.
def serialize(self):
return {
'best_name' :self.best_name,
'whatever_label':self.whatever_label,
'size' :self.size,
}
In retrospect, I might have serialized the whatever objects as lists, rather than a Python dict, which might have simplified their processing in the Flask service, and I might have separated concerns better in the Flask implementation (The database call probably shouldn't be built-in the the route handler), but you can improve on this, once you have a working solution in your own development environment.
Also, I'm not suggesting people avoid JQuery. But, if JQuery's not in the picture, for one reason or another, this approach seems like a reasonable alternative.
It works, in any case.
Here's my implementation of DickFeynman's approach, in the the client:
<script type="text/javascript">
var addr = "dev.yourserver.yourorg.tld"
var port = "5001"
function Get(whateverUrl){
var Httpreq = new XMLHttpRequest(); // a new request
Httpreq.open("GET",whateverUrl,false);
Httpreq.send(null);
return Httpreq.responseText;
}
var whatever_list_obj = JSON.parse(Get("http://" + addr + ":" + port + "/whatever-data/"));
whatever_qty = whatever_list_obj.length;
for (var i = 0; i < whatever_qty; i++) {
console.log(whatever_list_obj[i].best_name);
}
</script>
I'm not going to list my console output, but I'm looking at a long list of whatever.best_name strings.
More to the point: The whatever_list_obj is available for use in my javascript namespace, for whatever I care to do with it, ...which might include generating graphics with D3.js, mapping with OpenLayers or CesiumJS, or calculating some intermediate values which have no particular need to live in my DOM.
You make a bog standard HTTP GET Request. You get a bog standard HTTP Response with an application/json content type and a JSON document as the body. You then parse this.
Since you have tagged this 'JavaScript' (I assume you mean "from a web page in a browser"), and I assume this is a third party service, you're stuck. You can't fetch data from remote URI in JavaScript unless explicit workarounds (such as JSONP) are put in place.
Oh wait, reading the documentation you linked to - JSONP is available, but you must say 'js' not 'json' and specify a callback: format=js&callback=foo
Then you can just define the callback function:
function foo(myData) {
// do stuff with myData
}
And then load the data:
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = theUrlForTheApi;
document.body.appendChild(script);