As I can't orientate freely in the topic of building dynamic sites, it is quite hard to me to google this. So I'll try to explain the problem to you.
I'm developing a simple social network. I've built a basic PHP API represented by the files like "get_profile.php", "add_post.php", etc. with the POST method that is used to pass some data. Then I try to get the data using JS AJAX (php functions return it by JSON), which means I get all the data that I need to show on a page after the page is loaded. That causes decreasing of a page loading speed and I feel like this structure is really wrong.
I hope you'll explain me how to build a proper structure or at least give me some links to read. Thanks.
Populate the HTML with the (minimum) required data on the server side and load all other necessary data on the client side using AJAX (as you already do).
In any case, I would profile your application to find the most important bottle necks. Do you parallelize AJAX requests?
Facebook, for example, doesn't populate its HTML with the actual data on the server side, but provides the rough structure, which is later filled using AJAX requests.
If I understood your architecture right, it sounds ok.
Advices
Making your architecture similar to this allows you to deliver templates for the page structure that you then populate with data from your ajax request. This makes your server faster also since it doesn't have to render the HTML also.
Be careful with the amount of requests you make though, if each client makes a lot of them you will have a problem.
Try and break your application into different major pieces and treat each one in turn. This will allow you to separate them into modules later on. This practice is also referred as micro-services architecture.
After you broke them down try and figure user interaction and patterns. This will help you design your database and model in a way in which you can easily optimise for most frequest use-cases.
The way of the pros.
You should study how facebook is doing things. They are quite open about it.
For example, the BigPipe method is the fastest I have seen for loading a page.
Also, I think you should read a bit about RESTful applications and SOA type architectures.
Related
I'm creating a statistics web page which can see sensitive information.
The webpage has a sort of table which has massive data in it, editable and stored in Server's database. But It needs to be hidden before the user got proper authentications(Like log-in). (Table itself and it's code too). But I found that most of the questions in stack overflow say it is basically impossible. But when I see lots of well-known websites, it seems they are hiding them well. So I guess there are some solutions to the problem.
At first, I build a full-stack of React - Express - Node - MariaDB toolchain.
The react client is responsible for rendering contents of a webpage and editable tables and request for submitting edited content.
The node with express are responsible for retrieving data from DB, updating DB (Provides data to manipulate from client-side -- that's all)
It comes to a problem when I'm considering security on client-side code. I want to hide all content of the page (not just data from the server, but also its logic and features)
To achieving my goals, I consider several things, but I doubt if it is right and working well if I create.
Using Serverside rendering -- Cannot use due to performance reason and lack of resources available
Serverside rendering can hide logic from the user cause it omits the only HTML from the server and all actions are submitted to the server and the server handle the actions and provide its result.
So I can provide only the login page at first, and if login is successful, I can send the rest of HTML and it's logics from the server.
The problem is that my content in the webpage is massive and will be interacted with the user very often, and applying virtualization on my table (by performance reason), it's data and rendering logic should be handled by the web browser.
Combining SSR and Client-Side Rendering
My inspection for this is not sure, I doubt if it is possible.
Use SSR for hiding content of the site from unauthorized users, and if authorized, the web browser renders its full content on demand. (Code and logics should be hidden before authorization, the unauthorized user only can see the login page)
Is it possible to do it?
Get code on demand.
Also my inspection, this is what I am looking for. But I strongly doubt if it is possible.
Workflow is like below
If a user is not logged in:: User only can see the login page and its code
If the user is logged in:: User can see features of the page like management, statistics, etc.
If the user approaches specific features:: Rendering logic and HTTP request interface is downloaded from the server (OR less-performance hindering logic or else...) and it renders what users want to see and do.
It is okay not to find ways from the above idea. Can you provide some outlines for implement such kind of web page? I'm quite new to Web Programming, so I cannot find proper ways. I want to know how can I achieve this with what kinds of solutions, library, structure.
What lib or package should I use for this?
How can I implement this?
OR can you describe to me how modern websites achieve this? (I think the SAP system quite resembles with what I wanna achieve)
Foreword
Security is a complex topic, in which it is not possible to reach 0 threat. I'll try to craft an answer that could fullfil what you are looking for.
Back end: Token, credentials, authentication
So, you are currently using Express for your back end, hence the need to sort of protect the access from this part, many solution exist, I favor the token authentication, but you can do something with username/password (or this) to let the users access the back end.
From what you are describing you would use some sort of API (REST, GraphQL etc.) to connect to the back-end and make your queries (fetch, cross-fetch, apollo-link etc.) and add the token to the call to the back end in the headers usually.
If a user doesn't have the proper token, they have no data. Many sites use that method to block the consumption of data from the users (e.g. Twitter, Instagram). This should cover the security of the data for your back end, and no code is exposed.
Front-end: WebPack and application code splitting
Now the tricky part, so you want the client side not to have access to all the front-end at once but in several parts. This has 2 caveats:
It will be a bit slower than in normal use
Once the client logged in once, he will have access to the application
The only workaround I see in this situation is to use only server side rendering, if you want to limit to the bare minimum the amount of data the client has on your front end. Granted it is slow, but if you want maximum protection that is the only solution.
Yet, if you want to still keep some interactions and have a faster front end, while keeping a bit of security, you could use some code splitting with WebPack. I am not familiar with C so I can't say, but the Multiple page application of WebPack, as I was mentionning in the comment, should give you a good start to build something more secure.
First, you would have for example 2 html files for entering the front end: one with the login and one with the application. The login contains only the Javascript modules that are for entering the application and shouldn't load the other Javascript modules.
All in all, entrypoints are the way you can enter the application, this is a very broad topic that I can't cover in this answer, but I would recommend you to follow WebPack's tutorial and find out how you can work this out.
I recommend the part on code splitting, but all the tutorial is worth having a look.
Second, you will have to tweak the optimisation module. It is usually a module that tries to reduce the size of the application by merging methods that are used by different parts or that are redundant: you don't want this.
In your case, you don't want un-authenticated users to have access. So you would have to probably change things there (as well another broad topic to be covered in a single answer, since you would have to decide what you keep for optimisation and what you remove for security), but here is the link to the optimisation module and a heads up, you will have to modify the SplitChunksPlugin not to do this optimisation.
I hope this helps, there are many solutions are hand and this is not a comprehensive guide but that should give you enough materials to get to what you need.
Choosing the "right" web framework is quite challenging task, at least in Java we have a lot of them. But looking at JavaScript frameworks like AngularJS I doubt if we really need something heavy at server. Usually web framework is responsible for routing, templating, building pretty URLs and some other stuff. With AngularJS we can move all these responsibilities to client side. Then the backend becomes nothing more than REST listener and data validator. A thin layer between your application logic and client view. So why do we need web frameworks now if all we want is a REST listener?
At the moment I found two points which must be handled by server side: authentication/authorization and things requiring 'pushing' like Comet. Are these criterias enough to choose the "right" framework?
I'll give you one more capability that I've seen require back-end server support. It's those pages which generate a file. For example, they get a file from a third party and then hand it to the client as though they produced it directly, or they are generating a JPEG/PNG/GIF image on the fly, or perhaps a CSV/XLS dump of data. There may be ways to generate those on the fly from the front end and make them available for download, but sometimes the back-end is just easier for those.
Other than that, your assessment is 100% correct. You literally need less server for apps built with AngularJS than was needed with the previous request/response model of ASP/JSP/PHP/etc.
However, just because you need less doesn't mean you need nothing. Issues like data caching and how user sessions are handled can still come up even for smaller servers as you scale. But it has definitely opened things up for tech like Node.js to be considered that I would not have given much thought to a few years back.
I'm working on a project and there is some battle between how some JS filtering should be implemented and I would like to ask you guys some input on this.
Today we have this site that displays a long list of repeated entries of data and some JS filtering would be nice for the users to navigate through. The usual stuff: keyword, order, date, price, etc. The question is not the use of JS, which is obvious, but the origin of the data. One person defends that the HTML itself should be used and that the JS should parse through it making the user's desired filtering. Another person defends that we should use a JSON generated in the server, and that JSON should be the data's origin.
What you guys think on this? What are the pros and cons?
As a final request, I would like you to be the most informative as possible since your answers will be used and referenced for all us in the company. (Yes, that is how we trust you!:)
The right action is matter of taste and system architecture as well as utility.
I would go with dynamically generated pages with JS and JSON -- These days I think you can safely assume that most browsers has Javascript enabled -- however you may need to make provisions for crawler (GoogleBot, Bing, Ask etc) as they may not fully execute all JS and hence may not index the page if you do figure out some kind of exception for supporting those.
Using JS+JSON also means that you make your code work so that support for mobile diveces is done client side, without the webserver having to create anything special.
Doing DOM manipulation as the alternative would not be my best friend, as the logic of the page control and layout is split-up in two places -- partly in the View controller on the webserver, and partly in the JavaScript -- it is in my opinion better to have it in one place and have the view controller only generate JSON and server the root pages etc.
However this is a matter of taste, and im not sure that I would be able to say that there is one correct and best solution.
I think it's a lot cleaner if the data is delivered in JSON and then the presentation HTML or view of that data is generated from that JSON with javascript.
This fits the more classic style of keeping core data structures separate from views. In this manner you can generate all types of views without having to constantly munge/revise the way you store, access and manipulate the data. You can even build classes and methods to develop a clean interface on your data that is entirely independent of how that data is displayed.
The only issue I see with that is if the browser doesn't support javascript and that browser is a desired viewer. In that case, you have to include a default HTML version from the server that will obviously not be manipulated and the JSON will be ignored.
The middle ground is that you include both JSON and the "default", initial HTML view of that data in rendered HTML. The view comes up quickly and non-JS browsers can see something useful. But, then any future manipulation of the view (sorting, for example) uses the JSON data and generates a new clean view from the JSON data. No data is then ever "parsed" from the HTML view.
In larger projects, this also can facilitate the separation of presentation from data manipulation so different people may work on creating HTML views vs. manipulate the data (like sorting).
I would make the multiple ajax calls to the server and have it return the sorted/filtered data. If you server backend is fast than it won't be very taxing and you could even cache the data between requests.
If you only have 50-100 items than it would be reasonable to send it all to the client and have javascript sort and filter it.
Some considerations to help make the decision
Is the information sensitive and unique? (this voids and benefit to caching in my first point)
What is the most common request that will happen and are you optimizing for that?
How much data is there? (tens of rows, hundreds, thousands, millions)?
Does you site have to work with JavaScript turned off? (supporting older browsers?)
Is your development team more comfortable doing this in the front-end or back-end?
The answer is that it depends on your situation.
Web-Applications these days make extensive use of Javascript, for example various Google Products like Gmail and Calendar.
I'm struggling to how NOT having duplicated logic server and client side.
When requesting a page or state of the application, i would prefer to send the complete UI, meaning: not just some javascript, which in turn makes a dozen ajax requests and builds the user interface.
But here lies the problem, the logic deciding what to show or not has to be written once in the server-side and once in the client-side language.
Then i was wondering if it was somehow possible to process your javascript logic server-side and send the whole to the client, who in turn can continue using the application with all the advantages of a responsive ui, but without disadvantage of the initial loading/building of the user interface due dependency of background ajax requests.
I hope the explanation of my problem is a bit clear, because i'm not the most fluent English writer. If you understand what i mean and if you can describe the problem a little better, please do... thanks!
So my question is:
Is something like this possible and or realistic?
What is your opinion on how to tackle this problem?
;-)
When we started our web app, we had the same kind of questions.
It may help you to know how we ended:
The backend (business logic, security) is totally separated from the frontend (gui)
frontend and backend communicate through JSON services exclusively
the JSON is rendered client-side with the PURE templating library
and the backend is Erlang (anything streaming JSON would be ok too, but we liked its power)
And for your question, you have to consider the browser as totally unsafe.
All the security logic must come from the backend.
Hiding or showing some parts of the screen client side is ok, but for sure the backend decides which data is sent to the browser.
Seems you describe Jaxer.You can write everything in JS. Also, there is GWT that allows to write whole thing on Java
Then i was wondering if it was somehow
possible to process your javascript
logic server-side and send the whole
to the client, who in turn can
continue using the application with
all the advantages of a responsive ui,
but without disadvantage of the
initial loading/building of the user
interface due dependency of background
ajax requests.
Maybe the apps you're looking at just use Ajax poorly.
The only content you can pre-process on the server is the content you already know the user wants. For example, in an email app, you could send them a complete view of their inbox, pre-processed on the server and fetched with a single request, as soon as they log in. But you might use AJAX to fetch a particular message once they click on it. Sending them all the messages up front would be too slow.
Used correctly, AJAX should make your pages faster, because it can request tiny updates or changes of content without reloading the whole page.
But here lies the problem, the logic
deciding what to show or not has to be
written once in the server-side and
once in the client-side language.
Not necessarily. For example, in PHP, you might write a function like displayWidgetInfo(). You could use that function to send the initial widget information at page load. If the user clicks the widget to change something, send an AJAX request to a PHP script that also uses displayWidgetInfo() to send back new results. Almost all your logic stays in that single function.
Your instincts are correct: it's bad to duplicate code, and it's bad to make too many requests for one page. But I think you can fix those problems with some refactoring.
I understand what you're saying.
But I don't think you should be having much 'logic' about what to build, on the client side. If you did want to go with a model like you're proposing (not my cup of tea, but why not), I don't see why you'd end up with much duplicated.
Where you would normally show a table or div, you would just output JavaScript, that would build the relevant components on the client side.
I would consider it just as another 'View' into your data/business logic model.
Have you go a small example of a problem you're coming up against?
I understand your question in this way:
Suppose we have an html form on web-page. There is a field for name and surname. We have to check it for validity both on client-side (with JS) and Sever-side (on php script while processing form inputs). So here is the duplication - regex check on both sides. So what is the way to prevent it and combing these logics?
I am trying to get into web development, specially interested building the front-end, UI part of websites while learning JavaScript maybe with AJAX technology. (I have a UI, HCI background.)
However, I have absolutely no previous knowledge about server-end web development either. To my understanding, frameworks like Django seem to pretty good at this (correct me if I am misunderstanding).
So the question is: how much Django, or Rails do I need to know, if my interest is primarily the user interface part of web development. Can I just let somebody else do the back-end stuff?
Pardon me for my imprecise choice of terminologies.
You need to know a bit about the server side. Here's what you need to know.
If you have a heavy JavaScript website, you're likely going to want to pass information from the server to clients with JSON (JavaScript Object Notation). This is just a way to format data into strings that JavaScript knows how to convert to objects.
So, each of your server-side functions that send data to the client will return JSON. If you have someone writing the server-side for you, that's all you should have to know. You're JS functions will receive JSON, and then you deal with it.
If you have to write the server-side yourself, then that involves 1) getting data from database 2) formatting the data 3) converting to JSON.
I have open-sourced a commenting widget that accepts JSON messages, and gives examples of how you would set up the Django server code. Maybe it will help you: http://www.trailbehind.com/comment_widget/
You can make a career of front-end user interface development without know a ton about server code. You would do well though to have at least a rudimentary understanding of what happens on the server when you send it a request, where your data comes from, and what the life-cycle of a web page is. This assumes that you have the support of back-end developers. As you mentioned Ajax in your question that implies that you want your web sites to actually do something, which will require things to happen on the back-end (such as storage, manipulation of data, logging in a user, etc.).
As with all things, the more you know, the easier it will be to get what you want from the dedicated professionals. I would suggest that you learn about programming in general, not try an learn a language and framework. In particular, try to understand datatypes, server settings (like timeouts, post versus get, etc.), security and database interactions as they exist beyond JavaScript/ECMAScript. That way when a developer is explaining why they cannot do something you have requested or are offering alternatives, you are speaking the same language.
Yes and no. Typically what people think of AJAX, such as posting a comment on YouTube and seeing the comment appear instantly with a thank you message, for example, requires a server side language handling the requests, looking up data and returning results as html snippets, JSON data, or XML.
However, an AJAX call can be made to static resources as well. You could have an XML file or html snippet stored statically on your web server and have it loaded. The uses for this sort of static loading are generally fewer because if you already have the static html or data in file next to your regular page, why not just put that data directly into the page?
It helps to set up a local server and write a few lines of code to service your AJAX calls. You can do a lot of JavaScript learning with just a little back-end learning.
If you're new in web development you'd rather wait with Ajax and server-side languages until you've learnt the basics with HTML, CSS and JavaScript, especially if you want to mostly work with the user interface and not the funcionality.
As you said you can let somebody else do the back-end and focus on front-end (JavaScript, HTML, CSS).
You would need to communicate with the back-end developer when storing or processing data from the server.
As mentioned before back-end development knowledge would be useful but if you have someone doing it, it's not essential for beginning.