We've built a chat web-app, user get data from APIs first time and then send data back to server with those APIs, and we're using websocket to handle new data (new messages and other stuff).
Today one of our users told us that wrong number of messages showed up in the list of new messages, after hours of investigation I've noticed that user has wrong system time (about 3 minutes difference from our server time).
In our app, we check if user opened that conversation and then send current time back to server (from javascript new Date()).
I don't want to user lose any new message, even when they marked that conversation as read and during marking process new message arrived, if I set time from server they will lose that new message, if I use browser time that's not matching exact to our server.
I really don't know handling this case is important or not, and other apps handling this case or not, and if handling this how?
Every bit of help is really appreciated
Edit
I don't want to send user check-time on every request, for this reason our app sends server checked time 10 seconds after user interacted with app (marked that as read).
Send message with something like Last-Event-ID generated from server.
and decide which message you should send. not just user time.
Related
I have a configuration page where the user can select at what time to receive certain information from my API, for example they could select to get the information every day at 5PM or every Friday at 5PM. After this is set, the user should receive a Slack message at the time they defined, for example, every day at 5PM they would receive a message...
I save the user's timezone so that I can send the message to them in Slack at the correct time for them.
That being said, how can I schedule this message to send out from my node.js app? I would have a few users in the system, all who would have likely chosen different times so I would need a timer per user?
The only thing I can think of is scanning all users in the system, getting their selected time and then sending the message to them...but this doesn't seem scalable.
I'm not looking for a complete solution for this, just some pointers for how to design this sort of functionality.
I also looked into Slack scheduled messages but this isn't exactly what I'm looking for. Note: I can already send messages to Slack, I'm more interested in how to build the timer mechanism.
Thanks in advance!
EDIT:
Did a bit more research and it looks like node-schedule could be an option to schedule jobs: https://github.com/node-schedule/node-schedule#readme
With using this package, is the approach that I scan all users in my database at let's say midnight everyday and schedule jobs based on their settings...then those jobs execute at their scheduled time and the user receives the message in Slack. Is this a good approach?
For the core logic I would suggest something like this:
Store the timing of delivery (e.g. 5 AM ever Friday) for every user in your database
Then have a worker process that is running on a regular basis, e.g.
every 5 minutes
When it runs it checks if there are any due messages to be delivered
If yes it sends the message with your API information to the user and store the last time of sending for the user
This approach is resilient to downtime. It will just resume sending due messages once the worker process is running again after a downtime.
It is also scaleable: If needed you can run multiple worker processes (make sure to design your workers to support concurrent processing, e.g. with transactions)
Some additional things to consider:
Would limit the number of messages sent per run to avoid timeouts and having too many workers running in parallel
You need some error handling if sending message to Slack fails
To avoid timezone complexities I would suggest to convert all timings to UTC for processing in your app
Is this a regular task? In other words, it's executed every day at the same time for user X? If so, node-schedule seems fine, and it can run the same job regularly, you just have to set it up properly via a cron-like string (see the instructions in the README). If a user changes their setting, you then modify the previous job. The downside with node-schedule is that you need to set it up everytime your application is loaded, which can take a while and consume lots of resources if you have too many users.
Alternatively, if the number of users is big or you prefer to keep your application stateless, you can set a number of slots for sending these messages (and run that in a separate process from your main application). Let's say, a slot every 30 min. Then you set timers for those time slots (using node-schedule if you like, it will be just 48 timers), fetch the list of users for that time slot from the database, and send the messages.
Overall, NodeJS/JavaScript is pretty efficient with this sort of timer-based scheduling. If you want an in-depth dive into the reasons, see this: https://nodejs.org/de/docs/guides/event-loop-timers-and-nexttick/
You also need to consider what happens in case your application suffers from downtime. Should users be guaranteed to receive those messages, even if they are late? But that's another story :-)
I'm currently developing a website for a chat application (everything in the serverside is Django).
One of the problems I faced was how to keep the website (once a user is logged and has everything rendered) updated if the user needs to receive something new (a new message, notifications, etc).
The solution I came with was to first create a URL to send a get request, and the response would be a list of unseen notifications for the user. Then, in the HTML, a JavaScript code to send request to this URL and receive the unseen notifications.
This way, upon loading, the page will send a get and receive all unseen notifications and save them in a variable, after that it will keep sending the request every half a second and check if the rendered data is the most recent data, and if it's not, reload the page to refresh all the rendered data.
Now, this works ok, but I'm not sure this is how it should be done, as I'm bombarding with requests my server (currently, everything in development and just a couple of users at the time nothing exploded) and the client is sending request all the time. Is this how other webs (for example Facebook) keep themselves updated in case a new notification appears without the need of the user manually reloading the page?
Thanks in advance!
I have created a chat with node.js and socket.io.
When a user sends a message, I insert it directly in the DOM and emit the message to the server, so it can be emitted to all the other clients.
The problem is that it seems the timestamp set with Date.now() on the server is different from the timestamp set on the client with exact same command.
This makes the interface a bit weird because a message sent at a later point in time can show a timestamp prior to previously sent messages.
One solution would be to calculate the time difference when the user joins the chat room and subtract this difference when a new message is added, but should this really be necessary or is this the common way to solve this problem? Could this also be the solution to cope with timezones etc?
Different time zones between the client and the server possibly. It will not work with multiple users across the world. This one will be helpful:
How to ignore user's time zone and force Date() use specific time zone
Also, if you append the timestamp to the DOM immediately, and then get the date again from the server, there will be a difference in any case because you have to consider the time that the request needs to reach the server.
So I was implementing a chat room. I'll start off with the schema that I used.
I have a room table, that basically stores the information regarding the chatroom like the number of participant, the topic etc etc.
I have a users table that stores the users info.
I have a posts table that stores the posts. This has a foreign key from Users and from room tables.
also, I have one final table that is to have a relation between users and rooms. So it just has the roomid and the userid from the users who are a part of the room.
Now, I have three divs on page, one for the chatarea, the other where the people online are shown and then there is a text area to post the message.
What I am doing currently is, to have a javascript function loadChats(), now this method calls a php file that just fetches all the posts in that particular room till now. And the same is dumped into my div ie "chatroom".
Also, similarly, I have a loadParticipants() that load the users every other second.
I am using jquery.post for the purpose and in the end of the method, I do a setTimeout in the end of the function. Now here are my questions
Ofcourse i can make this better. Any suggestions? I was thinking of a few.
On every call to php, I get the entire chathistory and send it back to browser, ofcourse I can check if the count of messages is the same as it is on the client side, and if it is, then I wont send the messages. But is it going to make it any better? How?
Also, making a call to server side every other second seems a bit too much of an overkill. Is there any way to do it like, if some new chat is added to posts table, then that particular chatroom is notified and updated? i.e. instead of constantly pinging the server to ask for new request, just ask it once, and wait if there is anything new or not. When that request is completed, it pings the server again for the next update.
You should look into websockets (I've never used them with PHP but this seems really promising: http://socketo.me/). What you can do is have the server push any new messages to the client whenever they come in, and have each of the clients push to the server, etc. This way you won't have to keep pinging the server over and over every 2 seconds, and loading tons of data to compare. When there's a new message, the server saves it to some database and then pushes that message through all the open sockets. Same thing with logging in/logging off.
edit: Just looked through the page even more and their tutorial even goes through how to get it set up with a basic chatroom-esque functionality.
We do an online survey so we want to calculate the time spent. using javascript and php we have discovered the time spent is not 100% accurate.
The original script is sending server requests every 5 seconds and updates the time in the database.
I made a research and discovered setTimeOut and setInterval are not accurate at all. So what's the best way to do that?
I replaced the Ping function with another one that calculates the difference between previous packet timestamp and now(); however it's not accurate at all.
Please advice if there are any other solutions to the problem described.
Why don't you listen 'onunload' event of the document and send a single message in order to know when a user is leaving your page. Consider this; When a user starts filling your server you send a message and when he/she leaves your page, you send another one and measure the time between these messages in order to sum up the total time
I imagine most implementations of setTimeout and setInterval are set to wait at least the amount of time you specify. If you want accurate readings of how long the browser is open, use javascript's date functions to calculate it on the client side, then send to the server.
If you want to send requests to the server every 5 seconds, you can send the current time (using new Date().getTime()) in the request that is sent from client to server.
Save the time the 1st request was sent in the database. This is the time the user started the survey.
When each subsequent packet arrives, subtract the time the 1st request was send to get the total time the user has spent on the survey so far. When the user clicks the button to finish the the survey you could send a final request indicating the survey is complete.
You may also want to send requests on document blur and focus events. These events track when the user leaves and comes back to your page without closing the page down.