I'm using Node.js to build a site which involves getting posts from a public Facebook page.
I currently have the server updating my sqlite3 database from Facebook every n hours which works fine but I was wondering if there is some way I can set up an event which calls the update function whenever the page makes a post?
This would mean the site was always up to date with the page and there wouldn't be the delay of waiting until the next update time
You can use Webhooks: https://developers.facebook.com/docs/graph-api/webhooks
...those can only be used if you manage the Page though, there is no way to subscribe to changes to a Page you don't own.
Related
I am fairly new to javascript, I do know basics. I am looking to build my own (from scratch) java script library just like google analytics.js that will track user behavior on websites. Basically I'm looking to collect data like
Click through data
Dwell time
Page hits etc..
I spent lot of time trying to find website/tutorials to get me started on this but I keep ending up on google analytics.js or some private tools.
What I am looking for :
Is there any good starting point/resource/website which can help me build this js library
Are there reference for archetecture of end to end system including back-end?
Any open-source library that I can directly use?
Some things I already looked into
Chaoming build your own analytics tool
Splunk BYO analytics
At it's most basic, the architecture of such an application would only require a client, server, and database.
You can use basic javascript functions to record specific user actions on the frontend and then push them to your server. To identify your users you can set a cookie with a unique id. Then, everytime you send data to your server, you will get the specific user request as well so you can keep track of their actions. (Be careful of privacy laws first though).
For page hits, simply send a response to the server everytime someone opens your site - so call this function as soon as your Javascript loads. On the server, send a request to increment the appropriate value in your database.
For user dwell time, write a function that records the date when the user first hits your site and then count how long they stay there. Push your data to the server every so often and save updates to the user record by adding the new time spent to the current time spent. You could also watch for when a user is about to exit out of the site and then send the data all at once that way - although this method is more fragile.
For clicks and hovers, set up onclick and mouseover event handlers on your links or whatever elements you want to track. Then push the url of the link they clicked or whatever data you want - like "Clicked navbar after 200 seconds on site and after hovering over logo`.
If you want suggestions on specific technologies, then I suggest Node.js for your server side code and MongoDB for your database. There are many tutorials out there on how to use these technologies together. Look up javascript events for a list of the different things you can watch for on the frontend.
These are the building blocks you need. Now you just have to work on defining the data you want and using these technologies to get it.
I added setInterval method for downloading one of data. But the intervel is cleared when navigated to next page? is the functionality is correct?
Regards,George
Javascript variables (and setInterval generate a reference that you can save in a variable) are scoped by page.
Which means that when you are changing page, you make another request that reset your state.
Available solutions are the following :
Avoid reloading the page and use AJAX to refresh your page, if you app is mostly rely on async data I suggests you using some frontend framework to make Single Page Application like React, Vuejs, Angular.
You can also persist in localstorage or IndexedDB some data that can be given to a script that will run after page loading and create your interval.
You might also take a look at https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers
I believe I can use utilize the following functions to create delayed JavaScript alerts:
setTimeout(), clearTimeout(), setInterval() and clearInterval()
But how can I use them accross postbacks. For example:
A user has a list of reminders stored in the database. When user logs into site I can access those reminders and call something like:
setTimeout(function() { alert("Reminder 1"); }, 10000);
Depending on when the reminder due date/time is.
However as the user switches pages before the alert notification happens I believe the setTimeout call won't persist.
Is the only way to get this to work to look up the users reminders on every single postback and do a setTimeout on every page or in some sort of master page?
This is not really the way to do this. If your trying to build some sort of notification system then ideally it should be driven server side.
In a normal multi-page site every time you render a page the server would check for notifications and then in your template render an alert or some UI feature that tells the user they have notifications.
That set-up won't be active, IE if a new notification is posted on the server the client page won't know about it. Unless you use a notification message api for example pubnub.
Something like pubnub would let you send the message to your page, and in your JS code get a callback, so you can render it in your dom.
Since you have a multi-page app you would need to fetch / request for the notifications on every page render. Running a timeout to trigger for an ahead of time moment is not an ideal solution.
Another way to do it is to poll your server from your js code say every 5 mins and ask for notifications.
That has a price tag on all those repeated calls to your server. The best bet is to use something like pubnub but that comes at an extra laden of code server side to make it work.
I'm currently fooling around with AJAX. Right now, I created a Markdown previewer that updates on change of a textarea. (I guess you know that from somewhere... ;-) ).
Now, I'm trying to figure out, how to update a page upon an event is fired from another client. So to say an asynchron message board. A user writes something, an event is called, the post is written.
But on the other clients' pages, the new post is of course not yet available until they reload and get the updated list of posts from the database.
Now, how can you get this to work asynchronously? So in that moment when one client does something, the other clients all get to know that he did something?
I don't think this can be done completely in AJAX, but I also have no idea whatsoever how to implement this on server-side, as it would require a page reload to inform the other clients of the event.
I'm thinking of creating a file or database entry that hashes the current state of data. Whenever a client loads the page, he saves this hash. Then, a timer (does this exist in JavaScript?) checks for the hash every few seconds.
As soon as anyone changes the databse, the hash is recalculated. If the script sees that the hash was changed and is different to the one saved, it reloads the contents form the database and saves the new hash.
Is that even going to work?
Polling that is light as possible is really the best solution here. Even if you did use a socket or something... That's still basically a live connection waiting around that will likely have to poll itself (albeit in a more effecient way).
20 queries in 10 minutes that have responses like {"updates":false} shouldn't even be putting a dent in your application. I mean imagine someone browsing your site requesting 20 pages and the related images/scripts/etc (even if some caching is involved), there could easily be hundreds of requests requiring all sorts of wasted database queries to information to be displayed on the page they don't actually care about.
You could use polling. For example each client might be sending continuous AJAX requests to the server say each 30 seconds to see if new posts are available and if yes, show them:
setInterval(function() {
// TODO: Send an AJAX request here to the server and fetch new posts.
// if new posts are available update the DOM
}, 30 * 1000);
On the other hand when someone decides to write a new post you send an AJAX (or not AJAX) request to the server to store this post in the database.
Another less commonly used approach is the concept of Comet and the HTML 5 WebSockets implementation which allow the clients to be notified by the server of changes using push.
My Django app displays data from a database. This data changes without user intervention, i.e. behind the scenes. Whenever it changes, I would like the webpage to update the changed sections without a full page reload.
Obviously AJAX springs to mind. When the page is loaded initially (or manually, fully re-loaded later on), the rendered template loads a JavaScript that runs window.onload = update("all"), update(...) in turn triggers a number of XMLHTTPRequests which again return data that gets transformed into HTML pieces for the corresponding sections. All works fine. At the initial page load.
Now I find myself in a Python function that saves a new object to the database.
How do I tell the browser to run update(...) ?
Do I need to somehow manually issue a request to a url that is mapped to a view which in turn renders a template that contains the JavaScript code to run update(...) ??? Oh my!
I feel like I'm not following the usual approaches.
Maybe I'm just standing to close in front of the problem.
Can anyone help me ?
2021 update: Use channels: https://channels.readthedocs.io/en/latest/
You have two choices
Have the browser poll using setTimeout()
Look into Comet -- this is a technique for pushing data from the server to the browser.
Here's an article on Comet in Django
two approaches:
just update the database and wait until the next AJAX query. That means it should do the query periodically, you'll have to balance between immediacy and server load. It helps a little if you can do a cheap query to just verify if there has been an update. Maybe make that check rely only on memcached instead of going to the DB
use comet. In short: the client does an AJAX query asking for the update. the server sees there's no update, so it doesn't answer. Instead, the connection is kept open for a long time. Eventually either the update comes and the server finally answers, or the client times out and kill the connection. In that case, the client should immediately reissue the query to keep waiting for the update.
You can also use The Websocket API https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply.