I am developing a web application with angularjs as the front-end and a CRUD service at the backend. One of the requirements is to allow the user to upload a csv file containing a list of items to be created. This can be implemented on the front-end by parsing the file in javascript and making create API call to the server for each item. However, I am an not sure if this approach is better than passing the file to the server and doing all the processing there. What are advantages/disadvantages of both these approaches? What is the common practice in such a scenario?
There are 4 things that I would use to make this decision:
Do you have very high load. If you parse it on the client you are using the clients CPU. Parsing it on the server could cost you by needing more CPU's.
Access to developer talent, is your team more productive programming it on the client or the server side.
If the answer to the above does not give a clear answer, then I would put it on the server side as it would be easier to test.
Will the "upload TSV" functionality be used by other parties/apps, who consume your API -- or is only the frontend using this functionality ?
Since I have implemented this scenario, couldn't resist from responding. I believe following things would be considered(Addition to points mentioned above) :
The Size of the file, (Huge files freeze UI, no brainer) it can even crash some not so modern browsers.
Does the file need parsing/sanitizing the contents?( you would not want the garbage to make its way to your server)
Does the User need a feedback of the load summary details after the upload?(Aync vs Sync) - This tied back to #1
Regardless, you'll end up using some variation of the bulk copy at the backend.
Well I think its advisable to parse files at the backend. You get so many options like
saving the file for reference
reducing the load on your user's resource (RAM and CPU depending on the size of the file and the operation being done on the file before pushing to backend)
Can re-initiate activity on file if there is an error during batch( if the error is code you can reproduce and help out client because you've got the file😉)
Unless files are alway say some <1mb csv or txt just do stuff backend
I hope this helps 😏.
Related
There is an excellent tutorial on how to build a Ajax/PHP file upload including a progress handler provided at http://www.developphp.com/video/JavaScript/File-Upload-Progress-Bar-Meter-Tutorial-Ajax-PHP.
The receiver of the transmission (file_upload_parser.php) is implemented in php. However I need to replicate its functionality in Python, which turned out to be significantly more complicated than doing a simple translation.
The main problem seems to be to recreate php's global $_FILES-array (http://php.net/manual/de/reserved.variables.files.php) in Python, probably in the shape of a dictionary. I did not succeed using Pythons cgi.FieldStorage(), neither using the json module. Further, I had trouble keeping the connection established to listen for subsequent posts that might arrive (run_forever and the socket-approach suggested at Connect JS client with Python server turned out to not serve this purpose). There is plenty of documentation on how clients can write the file to the server using post. However I struggled to find information on how accomplish the read, that should occur on the receiving end of the transmission. At least not in a way that I could be implemented to yield the "file_upload_parser.php" functionality, which is key.
I have a simple audio sequencer application using javascript and HTML5. The goal of the application is to let users make a beat and save it so other people can open it up, add on to it, then re save. The issue I'm having is the save/open feature. How Do I add this? Or at least where can I start looking fro this solution?
If you are asking for an only-JS solution to this:
Opening Files
You can use the FileAPI to open the files from your users local machine, as long as its your users who are selecting the files manually(you are not digging into the file system yourself).
Here is nice demo for this functionality: http://html5demos.com/file-api
Saving Files
However, saving files was a W3C recommendation that got ditched so your only choice here is to allow your users to download the file. See this relevant question I made some time ago
If I were doing this for a class or prototype I might use PUT or POST requests to save the beat sequence in json form to a web server. When it's time to save your work, take your beat sequence data, convert from a js (I presume) object to a json string and PUT it to an appropriate url on your server. Give it a name and use the name in the url path, like "myserver.com/beats/jpbeat1.json".
When you want to retrieve that beat, issue an HTTP get to the url you saved. You can, of course also maintain and rewrite a directory page in json or html, or you can just navigate the directory offered by your web server. Most developers will think this crude, I imagine, but in a race for time, crude can be good if it meets the requirements.
Choose a web server, the easiest one for you. I might use Amazon S3. Costs money, but available to the internet, has authentication, security features, and supports http GET, PUT of static files. There are many other site hosting services.
For a local install, I might use Node.js. Also very easy to set up to serve static content. For example, the Express framework has static content service and easy examples how to set it up if you are comfortable with Javascript. With Node, you can make the site a bit fancier, by writing server side code to maintain an index, for example. Any real world implementation would require at least some server side code.
There are many other choices. Most developers would want a database, but you don't have to have one if simple files are ok for your class project. It all depends on your requirements.
I hope those ideas are helpful.
As I can't orientate freely in the topic of building dynamic sites, it is quite hard to me to google this. So I'll try to explain the problem to you.
I'm developing a simple social network. I've built a basic PHP API represented by the files like "get_profile.php", "add_post.php", etc. with the POST method that is used to pass some data. Then I try to get the data using JS AJAX (php functions return it by JSON), which means I get all the data that I need to show on a page after the page is loaded. That causes decreasing of a page loading speed and I feel like this structure is really wrong.
I hope you'll explain me how to build a proper structure or at least give me some links to read. Thanks.
Populate the HTML with the (minimum) required data on the server side and load all other necessary data on the client side using AJAX (as you already do).
In any case, I would profile your application to find the most important bottle necks. Do you parallelize AJAX requests?
Facebook, for example, doesn't populate its HTML with the actual data on the server side, but provides the rough structure, which is later filled using AJAX requests.
If I understood your architecture right, it sounds ok.
Advices
Making your architecture similar to this allows you to deliver templates for the page structure that you then populate with data from your ajax request. This makes your server faster also since it doesn't have to render the HTML also.
Be careful with the amount of requests you make though, if each client makes a lot of them you will have a problem.
Try and break your application into different major pieces and treat each one in turn. This will allow you to separate them into modules later on. This practice is also referred as micro-services architecture.
After you broke them down try and figure user interaction and patterns. This will help you design your database and model in a way in which you can easily optimise for most frequest use-cases.
The way of the pros.
You should study how facebook is doing things. They are quite open about it.
For example, the BigPipe method is the fastest I have seen for loading a page.
Also, I think you should read a bit about RESTful applications and SOA type architectures.
I know that it probably isn't, but is it possible to write/read a file on the server without server side languages. I've looked in a bunch of places and found oblique references to reading files on the server using XmlHttpRequest. I unfortunately have not found any "good" info on reading files.
I have found absolutely nothing when it comes to writing files without server side languages though. I am somewhat new to coding (4 to 5 months) and started out with HTML/JS, so I don't know any server side languages and when I look at them I get a headache trying to understand. I know AJAX interacts with the server, so I thought there must be some way to write a file on the server with AJAX.
Please tell me if I'm just being naive or if there it actually some truth behind my hypotheses. If there is a way to do so, then it would be nice to give me a link to an explanation or to give me one yourself.
I know that it probably isn't, but is it possible to write/read a file on the server without server side languages.
At a fairly abstract level: You have to have a server. That server has to be written in some language.
All regular web servers support the reading of files from file systems. Generally you specify a directory to be the root, then the local part of the URL gets mapped on to that directory and its sub-directories.
Writing is trickier. The HTTP specification includes the PUT verb, but not many servers have built in support for doing anything useful with it … and you'd almost always want to add some sort of access control before allowing anyone with access to the Internet to write files.
Expressing the logic behing the access control and where the file should be placed is something that is usually most easily handled with a programming language.
I've looked in a bunch of places and found oblique references to reading files on the server using XmlHttpRequest. I unfortunately have not found any "good" info on reading files.
XMLHttpRequest is just the standard API that browsers provide to JavaScript for making HTTP requests. It is entirely client side. You give it a URL, you get the data the server returns in a callback.
I have found absolutely nothing when it comes to writing files without server side languages though. I am somewhat new to coding (4 to 5 months) and started out with HTML/JS, so I don't know any server side languages and when I look at them I get a headache trying to understand.
JavaScript is a server side language. It isn't the easiest one to get to grips with, but it is an option. See node.js.
I'd recommend getting to grips with Python and using Django for someone new to programming.
I lean towards Perl and Catalyst for my own projects.
I know AJAX interacts with the server, so I thought there must be some way to write a file on the server with AJAX.
Ajax just means "Making an HTTP request from JavaScript without leaving the page". You can send a file, but once the request gets to the server, the server has to handle it.
The simplest method would most likely be to just use ajax to call a php script on the server. PHP is fully capable of reading/writing files on a server.
http://www.tizag.com/phpT/filewrite.php
http://www.tizag.com/ajaxTutorial/ajaxphp.php
Well let us understand the concept first.
AJAX is basically used to load the data from the server to the browser behind the scenes.
Next, If i presume that you know how to load the data asynchronously from the server to the browser behind the scenes. You might be using the POST or GET method.
Using the xmlRequestObject.send() function you can send parameters to the server file. But again just sending the data to the server is not enough. There needs to be some code on the server side to take that data and write that. For that you require a scripting language like PHP or use ASP.NET.
In my R&D, I did not find any such techniques that allow writing data to files on server without the use of server side scripting languages. If you find any..Let me know.
I am trying to get into web development, specially interested building the front-end, UI part of websites while learning JavaScript maybe with AJAX technology. (I have a UI, HCI background.)
However, I have absolutely no previous knowledge about server-end web development either. To my understanding, frameworks like Django seem to pretty good at this (correct me if I am misunderstanding).
So the question is: how much Django, or Rails do I need to know, if my interest is primarily the user interface part of web development. Can I just let somebody else do the back-end stuff?
Pardon me for my imprecise choice of terminologies.
You need to know a bit about the server side. Here's what you need to know.
If you have a heavy JavaScript website, you're likely going to want to pass information from the server to clients with JSON (JavaScript Object Notation). This is just a way to format data into strings that JavaScript knows how to convert to objects.
So, each of your server-side functions that send data to the client will return JSON. If you have someone writing the server-side for you, that's all you should have to know. You're JS functions will receive JSON, and then you deal with it.
If you have to write the server-side yourself, then that involves 1) getting data from database 2) formatting the data 3) converting to JSON.
I have open-sourced a commenting widget that accepts JSON messages, and gives examples of how you would set up the Django server code. Maybe it will help you: http://www.trailbehind.com/comment_widget/
You can make a career of front-end user interface development without know a ton about server code. You would do well though to have at least a rudimentary understanding of what happens on the server when you send it a request, where your data comes from, and what the life-cycle of a web page is. This assumes that you have the support of back-end developers. As you mentioned Ajax in your question that implies that you want your web sites to actually do something, which will require things to happen on the back-end (such as storage, manipulation of data, logging in a user, etc.).
As with all things, the more you know, the easier it will be to get what you want from the dedicated professionals. I would suggest that you learn about programming in general, not try an learn a language and framework. In particular, try to understand datatypes, server settings (like timeouts, post versus get, etc.), security and database interactions as they exist beyond JavaScript/ECMAScript. That way when a developer is explaining why they cannot do something you have requested or are offering alternatives, you are speaking the same language.
Yes and no. Typically what people think of AJAX, such as posting a comment on YouTube and seeing the comment appear instantly with a thank you message, for example, requires a server side language handling the requests, looking up data and returning results as html snippets, JSON data, or XML.
However, an AJAX call can be made to static resources as well. You could have an XML file or html snippet stored statically on your web server and have it loaded. The uses for this sort of static loading are generally fewer because if you already have the static html or data in file next to your regular page, why not just put that data directly into the page?
It helps to set up a local server and write a few lines of code to service your AJAX calls. You can do a lot of JavaScript learning with just a little back-end learning.
If you're new in web development you'd rather wait with Ajax and server-side languages until you've learnt the basics with HTML, CSS and JavaScript, especially if you want to mostly work with the user interface and not the funcionality.
As you said you can let somebody else do the back-end and focus on front-end (JavaScript, HTML, CSS).
You would need to communicate with the back-end developer when storing or processing data from the server.
As mentioned before back-end development knowledge would be useful but if you have someone doing it, it's not essential for beginning.