So I was implementing a chat room. I'll start off with the schema that I used.
I have a room table, that basically stores the information regarding the chatroom like the number of participant, the topic etc etc.
I have a users table that stores the users info.
I have a posts table that stores the posts. This has a foreign key from Users and from room tables.
also, I have one final table that is to have a relation between users and rooms. So it just has the roomid and the userid from the users who are a part of the room.
Now, I have three divs on page, one for the chatarea, the other where the people online are shown and then there is a text area to post the message.
What I am doing currently is, to have a javascript function loadChats(), now this method calls a php file that just fetches all the posts in that particular room till now. And the same is dumped into my div ie "chatroom".
Also, similarly, I have a loadParticipants() that load the users every other second.
I am using jquery.post for the purpose and in the end of the method, I do a setTimeout in the end of the function. Now here are my questions
Ofcourse i can make this better. Any suggestions? I was thinking of a few.
On every call to php, I get the entire chathistory and send it back to browser, ofcourse I can check if the count of messages is the same as it is on the client side, and if it is, then I wont send the messages. But is it going to make it any better? How?
Also, making a call to server side every other second seems a bit too much of an overkill. Is there any way to do it like, if some new chat is added to posts table, then that particular chatroom is notified and updated? i.e. instead of constantly pinging the server to ask for new request, just ask it once, and wait if there is anything new or not. When that request is completed, it pings the server again for the next update.
You should look into websockets (I've never used them with PHP but this seems really promising: http://socketo.me/). What you can do is have the server push any new messages to the client whenever they come in, and have each of the clients push to the server, etc. This way you won't have to keep pinging the server over and over every 2 seconds, and loading tons of data to compare. When there's a new message, the server saves it to some database and then pushes that message through all the open sockets. Same thing with logging in/logging off.
edit: Just looked through the page even more and their tutorial even goes through how to get it set up with a basic chatroom-esque functionality.
Related
I am fairly new to javascript, I do know basics. I am looking to build my own (from scratch) java script library just like google analytics.js that will track user behavior on websites. Basically I'm looking to collect data like
Click through data
Dwell time
Page hits etc..
I spent lot of time trying to find website/tutorials to get me started on this but I keep ending up on google analytics.js or some private tools.
What I am looking for :
Is there any good starting point/resource/website which can help me build this js library
Are there reference for archetecture of end to end system including back-end?
Any open-source library that I can directly use?
Some things I already looked into
Chaoming build your own analytics tool
Splunk BYO analytics
At it's most basic, the architecture of such an application would only require a client, server, and database.
You can use basic javascript functions to record specific user actions on the frontend and then push them to your server. To identify your users you can set a cookie with a unique id. Then, everytime you send data to your server, you will get the specific user request as well so you can keep track of their actions. (Be careful of privacy laws first though).
For page hits, simply send a response to the server everytime someone opens your site - so call this function as soon as your Javascript loads. On the server, send a request to increment the appropriate value in your database.
For user dwell time, write a function that records the date when the user first hits your site and then count how long they stay there. Push your data to the server every so often and save updates to the user record by adding the new time spent to the current time spent. You could also watch for when a user is about to exit out of the site and then send the data all at once that way - although this method is more fragile.
For clicks and hovers, set up onclick and mouseover event handlers on your links or whatever elements you want to track. Then push the url of the link they clicked or whatever data you want - like "Clicked navbar after 200 seconds on site and after hovering over logo`.
If you want suggestions on specific technologies, then I suggest Node.js for your server side code and MongoDB for your database. There are many tutorials out there on how to use these technologies together. Look up javascript events for a list of the different things you can watch for on the frontend.
These are the building blocks you need. Now you just have to work on defining the data you want and using these technologies to get it.
I am trying to create a social network with live chat system, so that users can have notification that they have a new message or receive a message after it was sent from another user in real time.
I am new to this, I have made front end (div that will hold messages that are fetched from DB, in form of a paragraph) and DB design, but I am not sure what to use for back end. My best solution so far is to make Ajax call for every user in every few seconds interval, but this looks like inefficient solution for many registered users.
I have searched the web and haven't found any good and up-to-date solutions and I would appreciate if someone could share some experience or point me in the right direction.
Few ways to do it:
websocket (with socketio it's the best)
Server Sent Event Long Pooling Pooling (Ajax)
The best now is websocket. But you can have some problems if your chat needs to work behind some firewall. But the overall perf if you use websocket, you will use something like 80% less resources.
I've built an API that delivers live data all at once when a user submits a search for content. I'd like to take this API to the next level by delivering the API content to a user as the content is received instead of waiting for all of the data to be received before displaying.
How does one go about this?
Easiest way to do in Django is using Django Endless Pagination
I think the better way to apply it is setting limit in your query. For example, If you have 1000 of records in your database, then retrieving all data at once takes time. So, if a user search a word 'apple', you initially send the database request with limit 10. And, you can set pagination or scroll feature at your front-end. If the user click next page or scroll your page, you can again send the database request with another limit 10 so that the database read action will not take more time to read the limited data.
From your explanation
We're pulling our data from multiple sources with each user search.
Being directly connected to the scrapers for those sources, we display
the content as each scraper completes content retrieval. I was
originally looking to mimic this in the API, which is obviously quite
different from traditional pagination - hope this clarifies.
So you in your API, you want to
take query from user
initiate live scrapers
get back the data to the user when scrapers finish the job !
(correct me if im wrong)
My Answer
This might feel little complicated, but this is the best one I can think of.
1. When user submits the query:
1. Initiate the live scrapers in to celery queue (take care of the priority).
2. Once the queue is finished, get back to the user with the information you have via sockets(this is how facebook or any website sends users notifications`. But in your case you will send the results html data in the socket.
3. Since you will have the data already, moved into the db as you scraped, you will can paginate it like normal db.
But this approach gives you a lag of a few seconds or a minute to reply back to the user, meanwhile you keep theuser busy with something on the UI front.
What I want to do is, when something new comes into the database, it gets the latest value entered and plays a specific song depending on what it says.
For example, on my computer, I run this website. Say somebody else around the world picked up their phone and clicked to play a song, that song would automatically play on all devices running the page. When somebody chooses a new song, the song should change.
I have made several tries at this, mainly trying to use setInterval in JavaScript however it just updates and updates all the time as you would expect.
I guess what I really am looking for is a way to track when the database has been updated, any solutions or code examples I can use?
If I understand you right, you can use push notifications.
https://firebase.google.com/docs/notifications/
Here's a tutorial how to set up a js client
https://firebase.google.com/docs/cloud-messaging/js/client
In this case you won't have to do unnecessary requests to the server to fetch updated. Just send a push notification to the client and do necessary actions on the client side.
I'm currently fooling around with AJAX. Right now, I created a Markdown previewer that updates on change of a textarea. (I guess you know that from somewhere... ;-) ).
Now, I'm trying to figure out, how to update a page upon an event is fired from another client. So to say an asynchron message board. A user writes something, an event is called, the post is written.
But on the other clients' pages, the new post is of course not yet available until they reload and get the updated list of posts from the database.
Now, how can you get this to work asynchronously? So in that moment when one client does something, the other clients all get to know that he did something?
I don't think this can be done completely in AJAX, but I also have no idea whatsoever how to implement this on server-side, as it would require a page reload to inform the other clients of the event.
I'm thinking of creating a file or database entry that hashes the current state of data. Whenever a client loads the page, he saves this hash. Then, a timer (does this exist in JavaScript?) checks for the hash every few seconds.
As soon as anyone changes the databse, the hash is recalculated. If the script sees that the hash was changed and is different to the one saved, it reloads the contents form the database and saves the new hash.
Is that even going to work?
Polling that is light as possible is really the best solution here. Even if you did use a socket or something... That's still basically a live connection waiting around that will likely have to poll itself (albeit in a more effecient way).
20 queries in 10 minutes that have responses like {"updates":false} shouldn't even be putting a dent in your application. I mean imagine someone browsing your site requesting 20 pages and the related images/scripts/etc (even if some caching is involved), there could easily be hundreds of requests requiring all sorts of wasted database queries to information to be displayed on the page they don't actually care about.
You could use polling. For example each client might be sending continuous AJAX requests to the server say each 30 seconds to see if new posts are available and if yes, show them:
setInterval(function() {
// TODO: Send an AJAX request here to the server and fetch new posts.
// if new posts are available update the DOM
}, 30 * 1000);
On the other hand when someone decides to write a new post you send an AJAX (or not AJAX) request to the server to store this post in the database.
Another less commonly used approach is the concept of Comet and the HTML 5 WebSockets implementation which allow the clients to be notified by the server of changes using push.