I have a MySQL database and retrieve data using php on a website where I would like to visualize the data in different ways. For this I also need to transform the data (e.g. creating sums, filtering etc.).
My question is now at which step of the data flow this kind of transformation makes most sense, especially regarding performance and flexibility. I am not a very experienced programmer, but I think these are the options I have:
1.) Prepare views in the database which are already providing the desired transformed data.
2.) Use a PHP script that SELECT's the data in the transformed way.
3.) Just have a SELECT * FROM table statement in PHP and load everything in a json, read it in js and transform the data to a desired version.
What is best practice to transform the data?
It all depends in the architectural design of your application.
At this time SOA (Service Oriented Architecture) is a very used approach. If you use It, logic use to be in the services. Database is used as a data repository, and UI manage final data in light weight format, only really needed information.
So in this case, e.g. your option number 2 is most appropriated.
Related
I am getting a list of items in a JSON(through AJAX) and creating the required markup by JS and appending in the view. On the other hand I have seen few examples which do not use this practice, and send the complete or partially complete markup through AJAX and then simply append them to the document. So definitely the markup is being generated on the server.
So I am curious which one is the better approach and why. One thing I can clearly see is that the later approach does not expose the JSON structure to the UI.
You're right about exposing the json structure but perhaps the main thing to consider is loading times, both client side and server side. If the server is dedicated and powerful enough, generating the markup in the server will be fast and will keep the site snappy on the client side.
If not, you'll want the client to do as much of the processing as possible. Generating markup on the client side is not a bad thing. The popukar angularjs framework builds markup on the fly client side and is very effective.
The choice depends on several factors like
what we are trying to achieve - Requirement must be picture clear.
what is the amount of work involved with these choices alone
which choice will have better performance
and in few scenarios which choice will get the job done quick. - where performance is compromised over delivery
Lets take an example to explain better.
1) Create a table to display all the details of employee.
Here since this is a stand alone table we don't have to worry about passing a JSON and then writing logic inside the JS to build the table. This can be done quickly without the need of JS. -- HTML Wins
2) Create a table to display all the details of employee. Also build a dynamic bar graph to display the details of the employee group on click of any row in the table.
Here the scenario is to build a table also on click of any table row get the employee group and generate a bar graph to show total employees in that group or may be any other stuff.
Here if we have built out table with just HTML then to get the data for the bar graph either we need to make another server call OR do a for loop of all tr's and then extract data from each cell by cell which is a pain.
If you use JSON to dynamically build the table structure, We can use the same JSON data and then build the graph and quick without other overhead, As we have the data ready to manipulate. -- JSON wins
This is just a small example. Also this concept is totally debatable, it is choice of the developer and what he is comfortable with, along with few other factors as mentioned above. I think you get the big picture. Hope this is helpful.
I'm not really comfortable working with Ajax as I just started using it.
My question is as follows:
What is the best way to manage data fetched using Ajax?
I have a script that fetches data from a database and displays it in different ways depending on the users filters and order criterias. So far, I query the database for every requests and was thinking if it could be better to fetch all data at once, store it in an array of objects and run queries like ordering and category filtering locally using Javascript.
Any gains in termms of speed and/or performance?
Thank you.
Show an example first. An example of the data that is fetched from database and the required way to arrange them. You can use XML or JSON or straight JavaScript object. the question is which one do you like to work with? I would choose the returned data from database to be in XML format. Show us an example of code where you don't know what to do.
In my application I receive json data in a post request that I store as raw json data in a table. I use postgresql (9.5) and node.js .
In this example, the data is an array of about 10 quiz questions experienced by a user, that looks like this:
[{"QuestionId":1, "score":1, "answerList":["1"], "startTime":"2015-12-14T11:26:54.505Z", "clickNb":1, "endTime":"2015-12-14T11:26:57.226Z"},
{"QuestionId":2, "score":1, "answerList":["3", "2"], "startTime":"2015-12-14T11:27:54.505Z", "clickNb":1, "endTime":"2015-12-14T11:27:57.226Z"}]
I need to store (temporarily or permanently) several indicators computed by aggregating data from this json at quizz level, as I need these indicators to perform other procedures in my database.
As of now I was computing the indicators using javascript functions at the time of handling the post request and inserting the values in my table alongside the raw json data. I'm wondering if it wouldn't be more performant to have the calculation performed by a stored trigger function in my postgresql db (knowing that the sql function would need to retrieve the data from inside the json raw data).
I have read other posts on this topic, but it was asked many years ago and not with node.js, so I thought people might have some new insight on the pros and cons of using sql stored procedures vs server-side javascript functions.
edit: I should probably have mentioned that most of my application's logic already mostly lies in postgresql stored procedures and views.
Generally, I would not use that approach due to the risk of getting the triggers out of sync with the code. In general, the single responsibility principle should be the guide: DB to store data and code to manipulate it. Unless you have a really pressing business need to break this pattern, I'd advise against it.
Do you have a migration that will recreate the triggers if you wipe the DB and start from scratch? Will you or a coworker not realise they are there at a later point when reading the app code and wonder what is going on? If there is a standardised way to manage the triggers where the configuration will be stored as code with the rest of your app, then maybe not a problem. If not, be wary. A small performance gain may well not be worth the potential for lost developer time and shipping bugs.
Currently working somewhere that has gone all-in on SQL functions.. We have over a thousand.. I'd strongly advise against it.
Having logic split between Javascript and SQL is a real pain when debugging issues especially if, like me, you are much more familiar with JS.
The functions are at least all tracked in source control and get updated/created in the DB as part of the deployment process but this means you have 2 places to look at when trying to follow the code.
I fully agree with the other answer, single responsibility principle, DB for storage, server/app for logic.
I'm trying to make pull XML or Csv data into a HTML file then I want to use math to add up the values and show the result on the page ( I'm basically trying to display invoices on a web browser)
My skill set is HTML/CSS and I understand a little JavaScript
I've managed to pull XML data into HTML using http request and style that information using xslt
Really what I'm asking is what is the best solution to my needs is it using the above method then using xquiry to add up values or would I need to learn a bit of Ajax, Json and calculate the values with JavaScript?
You really should learn AJAX in order to fetch and manipulate data instead of fetching presentation parts. That's the way everyone follows as it allows more responsive interactions with the user and a cleaner architecture in case of complex interactions.
But that doesn't mean you must abandon XML : originally AJAX was built on XML (the X in AJAX) and not JSON.
Personally I prefer JSON, and I think it will be easier to manage in the long term, but if the server side is hard to change, you can fetch the XML (look for example at jquery's ajax function), build javascript objects using it, and then change your screen using those data. If later you decide to use JSON instead of XML, you'll just have to change the "parsing" part of the client code.
"I'm trying to make pull XML or Csv data into a HTML file then I want to use math to add up the values and show the result on the page"
You can do this with either XSLT or javascript. However, with XSLT things can become pretty complicated, depending on what version you're using. XSLT 1.0 has pretty limited set of functions for aggregating results. For all XSLT, you can't reassign variables you'll have to solve many of these things with recursion. In my opinion, not really a comfortable method.
Regardless of the choice between XSLT and Javascript, I would also question the architecture that would put this kind of logic in the presentation layer in the browser. I think it would be better if the server side would perform all the calculations that are required, and limit the browser's tasks to styling the output.
I'm designing an interface to display sets of data. I'm interested in organizing the information in a customized format using DIVs and SPANs to identify where specific data will be (not in the typical row-by-row format tables provide) but still having the flexibility of sorting and organizing the data that most JS libraries provide for tables. Is there a way to get the best of both worlds without rolling my own solution?
I want to sort the data on the fly using JS. I don't want to do crazy sorting... tasks like "group by type", Alphabetize, Sort Date, Ascending/Descending, etc. I know I could presort and display on the backend but I want to reduce server load and wish to keep unnecessary computations off the server. If there's a way to sort and organize on the client side, that would be best. My preference is to work with PHP on the backend and using the jQuery lib on the frontend. I'm also using mySQL if that helps.
It seems like you want the TinySort plugin.
TinySort will sort any nodetype by
it’s text- or attribute value, or by
that of one of it’s children.
Here's the documentation.