I have a webpage that's for a fund raiser, and I'd like to update the amount raised live. So if a user is looking at the total raised, and somewhere else another user donates, then the first user would see the amount tick up and change to the updated amount.
I have a CouchDB for a database and Lambdas using Node.js running. How can I set this up?
It's a well known software development pattern. It's usually achieved by using "push" or "pull" approach.
In a "pull" approach a client constantly asking for data from backend. In your particular scenario it could be a timer on a page that at specified interval polls server for data and updates page if needed.
In a "push" approach backend "pushes" changes to client(s) when they become available. Web sockets would do the job in your particular case but implementing it is much more complicated than a "pull" approach.
Related
Update: I am getting the impression that this is not even the right website to post this. If someone can point me in the right direction, I'd be appreciative...
I have an existing PHP+MySQL application that wasn't built to render "real-time" or similarly live-style data. But now I need to build in a way to pull nearly real-time data into the application and keep the data on the page fresh. This live data is only for 1 page in the application.
Looked at things like socket.io and PHP-based websockets libraries, but it seemed like overkill because the data is basically coming from 1 source and being delivered to 1 person (the client). Multiple other users could have this process running, but each one would bring their own data endpoint. That's... like a year down the road. But good to think about. Would ideally have hundreds, or thousands of users on the system, pulling their live-ish data. So I want this to be as streamlined and low-impact as possible.
Users must be authenticated and authorized to consume the data. This is already baked into the current system.
The API to get the data (which has already been built by another vendor) is also NOT streaming. It's set on a 20-second cron, so the new data is available every 20 seconds, which satisfies the client's needs.
My current plan is to do something like this...
Data is pulled on a cron every 20 seconds, organized, and stored into the database (complete)
Adjust #1 so it also does any additional proprietary calculations on data AND compiles + writes a JSON file on the server (unique to the user) which is the exact data needed for the front end (DB data is needed for other pages)
Create small PHP-based service which validates a client-provided JWT and reads the JSON file out
Write AJAX front end to poll endpoint from #3 every X seconds using a JWT for authorization
This all seems sort of like I might be reinventing the wheel, or missing something. The fact that this is an existing PHP based application (LAMP) does have some limiting factors, but I feel like there's got to be a more efficient way to handle this... It's pretty new to me. Also, I'm open to other technologies that'll run on the LAMP stack, if it'll make things better.
I would say go for the API solution in the beginning :) Since it fits the architecture more and is for sure the least amount of work. Also if there will be problem with the "live" feeling of the data you can fix it by polling more often or introducing long polling, assuming you change the cron job time.
I mean in the end it is all about impact for the time spent, don't start implement features that customers don't care about :)
The biggest problem to solve is to implement it in a way that fits your requirements and is somewhat future extendable. You still have to deal with issues like resolution, time outs, reducing server processing when requesting data and so on!
For me, if you need to maintain a global service state because a single client(s) request could affect all other connected client request(s) then most all server-side scripting languages are not the best choice! Also to further add, if you plan on implementing something like this with PHP, you will be setting your self up for a living nightmare! Why, because simply put, PHP(s) socket(s) implementation is that bad!
I have a website that manages realworld tabletop games via a php, jquery, bootstrap and mysql setup.
It has been running very well for a number of years, but I am implementing a team game concept, which allows 2 "captains" to manage the pairings at the same time. The page itself does what I want it to do when one captain does all the data entry, but it is not really optimal for both to be doing it at the same time.
Once both players for a game have been selected the row turns green
The goal is that as a Captain selects a player from a drop down box, it should somehow update the other captains screen and vice versa.
Should I have some kind of timer going, and every X seconds refresh the page, form, etc? Has anyone done something similar to this in the past?
I am thinking of having a table in my database with each field on the form, and when it was last updated, then I could loop through the table and only update the most recent ones, but I feel this could be an extra layer that just may be over complicating it.
Any pointers would be appreciated
Refreshing the whole page is definitely possible, but I wouldn't recommend it.
You could execute an ajax call for every x second with setInterval(). Requesting all data from the server and see if everything is loaded. You also would have to send the new data back to the server when the player changes a field.
A better approach for this would be the usage of sockets. They synchronize data across different browsers (almost) instantly. Without the need to constantly request data from the server.
You can take a look at socket.io for more information. This is a javascript package to make the implementation of sockets fairly simple in javascript.
I've read a few StackOverflow posts related to this subject but I can't find anything specifically helps me in my scenario.
We have multiple monitoring instances within our network, monitoring different environments (Nagios, Icinga, more...). Currently I have a poller script written in PHP which runs every minute via cron, it asks the instance to return all of its problems in JSON, the script then interprets this and pushes it in to a MySQL database.
There is then an 'overview' page which simply reads the database and does some formatting. There's a bit of AJAX involved, every X seconds (currently use 30) it checks for changes (PHP script call) and if there are changes it requests them via AJAX and updates the page.
There's a few other little bits too (click a problem, another AJAX request goes off to fetch problem details to display in a modal etc).
I've always been a PHP/MySQL dev, so the above methodology seemed logical to me and was quick/easy to write, and it works 'ok'. However, the problems are: database constantly being polled by many users, mesh of javascript on the front end doing half the logic and PHP on the back doing the other half.
Would this use case benefit from switching to NodeJS? I've done a bit of Node.JS before but nothing like this. Can I subscribe to MySQL updates? Or trigger them when a 'data fetcher' pushes data in to the database? I've always been a bit confused as I use PHP to create data and javascript to 'draw' the page, is there still a split of NodeJS doing logic and front end javascript creating all the elements, or does NodeJS do all of this now? Sorry for the lack of knowledge in this area...
This is definitely an area where Node could offer improvements.
The short version: with websockets in the front-end and regular sockets or an API on the back-end you can eliminate the polling for new data across the board.
The long version:
Front-end:
You can remove all need for polling scripts by implementing websockets. That way, as soon as new data arrives on the server, you can broadcast it to all connected clients. I would advise Socket.io or the Primus websocket wrapper. Both are very easy to implement and incredibly powerful for what you want to achieve.
All data processing logic should happen on the server. The data is then sent to the client and should be rendered on the existing page, and that is basically the only logic the client should contain. There are some frameworks that do all of this for you (e.g. Sails) but I don't have experience with any of those frameworks, since they require you to write your entire app according to their rules, which I personally don't like (but I know a lot of developers do).
If you want to render the data in the client without a huge framework, I highly recommend the lightweight but incredibly useful Transparency rendering library. Using this, you can format a Javascript object on the server using Node, JSONify it, send it to the client, and then all the client would have to do is de-JSONify it and call Transparency's .render.
Back-end:
This one depends on how much control you have over the behaviour of the instances you need to check. I assume you have some control, since you can get all their data in a nice JSON format. So, there are multiple options.
You can keep polling every so often. This is the easiest solution since it requires no change to the external services. The Javascript setInterval function is very useful here. Depending on how you connect with the instances, you might be able to use a module like Request to do the actual request, so that takes out a bunch more of the heavy lifting.
The benefit of implementing the polling in your Node app as well, is that you will receive the data in your Node app and that way you can immediately broadcast it to the clients, even before inserting it into a database. This will greatly reduce the number of queries on your database.
An alternative to polling would be to set up a simple Express-based API where the applications can post their 'problems', as you call them. This way your application will get notified the moment a problem occurs, and combined with the websockets connection to the client this would result in practically real-time updates.
To be more redundant, you would have a polling timer alongside the API, so that you can check the instances in case there's something wrong that causes them to not send over any more data.
An alternative to the more high-level API would be to just use direct socket communication, which is basically the same approach only using a different set of functions.
Lastly, you could also keep the PHP-based polling script. This would be the most efficient solution since you wouldn't go and replace everything. Then from the Node app that's connected to the clients with websockets, you could set an interval to query the database every so often and broadcast the updates. This will still greatly reduce the number of queries, since no matter how many clients are connected there will only be one query, the response of which then gets sent to all connected clients.
I hope my post has give you some ideas of how you could implement your application using Node. Keep in mind though that I am just one developer, this is how I would approach building your application in Node. There will definitely be others who have different opinions.
I am trying to write a change listener based on NodeJS or PHP. We have a huge database in MSSQL server. I want the change listener to be listening for a change in database for example a specific column changes in database and the change listener gets the information of that particular row that has been modified and than perform operations on it. I am not trying to make a real-time application for the users. I am just trying to log the changes of one local database in an remote database and I am require todo it in NodeJS or PHP. You can see the image i am posting below.
http://i.stack.imgur.com/c3jD6.png
The part That I totally Understand
I know that what I can do is make a request to database every x amount of time. I can use sockets or long polling etc. (NOTE: Please correct me if I am wrong)
The part that I dont understand
How I will be able to get only that particular row that has been modified through sql query? (NOTE: I can just get the data from that database, I am not able to modify anything in database). Database is so huge I will not be fetching all the rows again and again. I have not idea on this.
Note
I am not doing any replication. What I am actually doing is monitoring a column in SQL server for changes and according to that change I perform certain operations in Remote database.
You cannot practically detect changes. Only SQL Server can track changes, and it exposes this tracking via Change Tracking or Change Data Capture. Your app, completely irrelevant of the access technology used (C#, C++, Node.js, ruby or whatever), can then interrogate the change tracking infrastructure and learn what rows have changed since last time it checked. Do not try to roll your own in-house developed change tracking mechanism, be it by triggers or 'changed_at' column or whatever. Many braves give up the ghost trying, save the headaches. Use the built-in off-the-shelf technologies for this task (linked above).
You say 'I am not able to modify anything in database' then the answer is simple: you are doing a fool's errand. Is impossible to detect changes w/o changing the database, is simple as that (comparison methods need not apply for any significant size). If they don't let you change the database (and enable one of the change tracking methods) then simply move on to a different project.
Basically, I have created a NODEJS app that uses Jade as its templating engine, along with Express and a MySQL database.
I am looking to create a new page which allows user to share a portion of text, and then a div underneath it named "Wall" will update dynamically with the new status.
Basically, it would ideally be similar to Facebook where something is typed, shared and then the page updates below dynamically. I'm also looking to have the wall page update when a new post have been shared from a users friend. All updates shared by the users would be sent to a database.
I have conducted a lot of searches but seem unable to gather a right answer.
I have narrowed it down to the use of either of the following: JQuery, Ajax, PHP.
Since the site I am building is built in JS - what is my best option?
I'm pretty new to all of this, but I assume when a user clicks share it calls a JS file which then stores the update in the database. But how do I get my "Wall" to refresh upon new content?
Any help greatly appreciated.
You've posed a conceptual question. So I'll do my best to explain some of the conceptual options you can choose to further explore and do your own research on how to best implement it with your project.
You have two paths to go here.
You can have your own wall update (do a refresh / re-render on the UI side) upon a successful AJAX write to your database, this would be something you implement in your AJAX callback function - basically the JS function that gets executed after your write request (submitting the new post) to the database returns successful.
A whole other branch of options you could explore, is implement either of the following options to basically "listen" in for changes server-side, and have the re-rendering react as the callback you use:
Polling - basically issue a request every X number of
seconds to check if there have been updates, or change of state on
server-side.
WebSockets - checkout Socket.io. Through this you can "push" messages from the server-side to your clients. As a note, WebSockets are not universally supported in every browser and from past experience I've found WebSocket protocols even differ by browser versions. So if you need universal support, I'd go with a polling method.
Good luck on your project, hope this helps!
Use...
setTimeout(function(){
/* update wall here */
}, 1000)
to poll your "Wall" backend and updated the content.