Want to create a new application for website traffic monitoring ( analysis ) [closed] - javascript

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I want to create my own application for monitoring traffic of my website without using any third party tools like google analytics. In which I want to log screenshots, user details, page details and cookies. So what technology should I opt so as to achieve this goal and which will be best suited and what work flow should I follow.
I've never done this kind of work previously so I'm new to this. Any help would be greatly appreciated.
The technologies I know are : javascript, nodejs, django(python).

You will have to break down your application as it is a humongous task to create something like GA.
You will have to track many user activities(click,spend time etc).so you can do that in plain js or use a cross-platform lightweight library(angular.js) that can make your life a little easy.
Now since you will have to send large set of traced data to you database with minimum latency,use Node.js in this scenario.Simple Ajax call would also work but then it would be very slow.
Now comes your database.Prefer NoSql since it suits your requirement of unstructured data ,preferrablly MongoDb which can help you with its own mapReduce,large storage capacity
.Since there will be lot of calculation involved you can use your python knowledge which can help you process data a lot faster.you can use other languages as well(eg.Go)
Your processed data and results can then be stored in Redis(which acts as a caching layer).
you can use sophisticated graphic library like d3.js,Highcharts.js for displaying Graphical data on the client-side.
There are a lot of factors that can be involved.This is just a very basic outline of what you could do.

Related

Which database should I use in this budget app project? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 23 hours ago.
Improve this question
: )
On my adventure with coding, I got to the point when I want to learn about databases but I don't quite get where to start.
I know that we have SQL and noSQL and I get the difference more or less.
The project:
It's a basic budget app. For personal and learning use only (me + a bunch of friends). I store the data in local storage. It's not really convenient + I want to learn the alternatives.
The project has been deployed via Netlify.
In the future, I'd like to write a small app which would work as a survey app and then save the app in some database so that I can analyse it (I actually need such an app to finish my Master's thesis and can't find anything). There would be mostly closed questions. Then, I'd like to check eg how many ppl chose A in question 1 and simultaneously B in question 2. Here, the SQL database would be more convenient, is it right?
I use JS and React.
My question is:
is there any free server online that I could use as a database?
I thought about JSON file. Please, correct me if I'm wrong, but I cannot make a fetch request from the app to the app. I understand that I would need a server or API. I don't want my computer to be the server as it's not sustainable.
The same thing with node.js, right? I can't use it inside the app to change a file in the app unless I use localhost, which I don't.
I also thought about Mongodb. I don't quite get how it works. I saw a tutorial that instructed me to download it. Then I will use my computer as the server, right? Is there an option to use some online database, preferably for free?
I know that Amazon, Google and Microsoft offer sth, but it looks a bit perplexing to me and before I dive into this, I'd like to know if I'm at least going in the right direction. I saw a tutorial about Google sheets as a free online database. It looks complicated but would fulfil my need. Do you have any experience with that?

How do I make a web app with Google APIs? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 10 months ago.
Improve this question
I am pretty new to web dev, and I wanted to create a simple UI in Javascript with the Google Maps Places API to familiarize myself with everything. My question is, would I just make direct URL requests with the user’s variables, or would I have to create something server side, and if so, what would I have to do?
It depends on your application's functionality. Generally, you could keep all everything on the client-side. If you app is a server-rendered one, meaning you would be using a JavaScript framework like React or Vue, you could use just the website state to keep track of users' variables and send out requests to the Google APIs to do the heavy work.
You could also look into tools like Firebase, also created by the folks at Google to handle storage or user authentication. There are rich docs on how to include Firebase in your web application for your framework of choice. This way you can create language-agnostic backend resources that conform to all of the security measures and scales automatically so you can focus on the users' experience.
In case you would like to offer some less-common functionality or have complete control over the backend you could write up an API of your own and have it deployed on a separate backend server.

MySQL vs JSON vs Firebase as a database option [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I am currently developing a site to track credits gained from exams, and how these stack up against the user's goal credits.
I know JavaScript fairly well, but don't know any PHP.
What would be the best option to use as a database? Please, I am a beginner and I will be getting answers from advanced programmers. To me, your opinions count so please don't close this as asking for opinions.
MySQL
I could use this on a PHP site, but it would involve me learning PHP. What are the benefits and would they outweigh the hassle of learning a whole new language?
JSON
Would it work if I used JSON files for data storage, and used xmlHttpRequest() to read and write information?
Firebase
Would Firebase work as a fully operating database?
If you know javasript very well and aren't willing to learn a new server side language, I would recommend using Firebase. They way it works is very similar to if you had built your own REST api and just made ajax calls to it.
I wouldnt recommend using json to store data. Anytime you want to read there would be more data sent over the network than necessary and when you get enough data it would be impractical. Databases exist to avoid storing data in a file and simply reading/writing to the file.
Using php wouldnt be the best idea. It is a very powerful language, but it is incredibly easy to write major bugs with it. If you were to use it I would recommend learning a web framework because it would fix most of the bugs that you would write yourself. Heres a list of a bunch of php frameworks http://www.phpframeworks.com/. I think that there would be more benefits to you actually learning php. For starters, every developer should have more than 1 language in their toolbox. You should be able to approach a problem and know what the best way to solve it is and that starts with the exposure to different languages and technologies. Learning another language will take a decent amount of time, but you will get much more out of it than releasing your site a little bit earlier.
So to sum it up, I would use php(or any other scripting language for that matter) with a web framework. Your site would be more secure with the web framework and you will become a much better, more valuable developer by taking the time to learn something new.

How to implement Google Analytics and save daily statistics of a webpage in a database [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I would like to know how to implement Google analytics and make an SQL query from PHP to save daily webpage statistics.
Thank you for your answers.
There are a lot of issues that you will need to overcome to get this to work. It's not as simple as just adding a file to your site. You will first need to set up analytics on your site using the javascript that Google provides.
Next you will need to connect to the Google servers either using a PHP API like GAPI, curl or OAuth. You could also try a JSON solution. I personally like using the OAuth method for the ease of use. There are OAuth libraries that can do the heavy lifting, too.
After you have the data you need, you need to add it to the database. You can do this within the PHP for the request, or with AJAX. Both methods are decent. The SQL itself is up to you. If you are using some sort of CMS, try using their database methods instead of writing your own. If you are writing one from scratch, I would suggest the PDO API in PHP. It takes care of a lot of the security vulnerabilities you usually have with SQL and it's well documented.
All-in-all, there are a lot of moving parts to this process. I would suggest looking for someone who has all, or most, of the leg work already done before diving into it yourself.

Real time collaborative use interface on the web [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm wondering what the technologies and best practices are behind real time collaboration in web interfaces.
An example of this is of course Google Wave. Another is PivotalTracker.com.
I'm particularly interested in any work (frameworks, plugins, etc) people are doing with Ruby on Rails here.
I imagine it would have to use Javascript underneath at some level, but you would need a way to abstract this out. Probably polling the server periodically to see if changes have been made, and also a way to resolve conflicts if in the middle of editing something the server comes back and says someone else has updated it.
Thank you!
Wave has operational transform that has a nice property of being easily combinable. You have two users, each of them does "something" in the user interface and two "somethings" can be combined into final document. That allows you to skip the problems with conflict resolution.
A nice way to enable real-time updates to state of the app is by using Comet, which is essentially a geeky codename for keeping an alive, long standing, unterminated get/post request to the server, that server finishes and responds to when something happens on the server. It allows sending to the client instantaneous updates without having the client periodically poll.
I can't really say how to abstract this away in javascript/r'n'r, many of the underlying technical details are hard enough and application specific that no framework supports them out of the box.

Categories