We Keep Coding

Django: fastest way to update the data that was once sent from template to view

I am working on an data visualisation app that will allow the user to filter the data that he sees by various criteria.
I want to keep as much logic as possible on Python/Django side, like this:
Data is passed from Django view to the template.
On the frontend, user filters the data through various controls: dropdowns, sliders etc.
The controls inputs are sent back to Django view (via AJAX post request?), which returns filtered data and sends it back to the template.
4.The template - the visualization - is updated with the filtered data.
Is this a good approach? My concern is that a lot of data will be flying around and the app might be unresponsive.
Another, possibly faster idea is to filter the data on client's side in JavaScript - but I would really like to leverage the great Python data munching libraries instead.

If you want to use DRF API, then go with it. A lot of websites have filtering features. I'd suggest you to take a look at django_filter package. It's possible to integrate it with DRF.
The worst thing in filtering data on client side is that you can't use pagination. Imagine that you have 500+ objects to filter, javascript filtering function is what really will make your app slow.
At the same time, if you have 20-30 objects to filter and this number won't grow up, then you can go with JS only and single endpoint: getAll()
Common approach is to set up javascript on_change handler and construct GET requests like(example from real project) this:
https://yourbackend.com/api/product/?status=not_published,published,inactive&search=132&moderation_status=declined,on_moderation,not_ready&ordering=desc&price_max=1000&page=1
DRF + django_filters will work just fine that, with minimum of your code
involved.
Well known pitfall on js side is to make request without timeout, eg user writes text and on every keyUP() event request being sent. Or he moves the slider and a lot of requests being made - you'll need to make request when users stop it, eg 300ms after he chosen value. See this question for reference.
Sure, there's one more point. Your database have to be normalised and have proper indexes. But you have to look at this side if you'll have really slow SQL queries.
Summing up: I'd choose thin js layer and do most of work on backend.

How to keep a list of items updated on a website using Javascript?

Coming from Python/Java/PHP, I'm now building a website. On it I want a list of items to be updated in near-realtime: if items (server side) get added to or deleted from the list, this should be updated on the webpage. I made a simple API call which I now poll every second to update the list using jQuery. Because I need some more lists to be kept updated on the same page I'm afraid this will turn into more than 10 server calls per second from every single open browser, even if nothing gets updated.
This seems not like the logical way to do it, but I don't really know how else to do it. I looked at Meteor, but since the webpage I'm building is part of a bigger system I'm rather restricted in my choices of technology (basic LAMP setup).
Could anybody enlighten me with a tip from the world of real-time websites on how to efficiently keep a list updated?

You can use WebSocket(https://code.google.com/p/phpwebsocket/ ) technology.
but php is not the best language for implement it

A way to work this is using state variables for the different types of data you want to have updated (or not).
In order to avoid re-querying the full tables even if the data set in them has not changed in relation to what a particular client has displayed at any given time, you could maintain a state counter variable for the data type on the server (for example in a dedicated small table) and on the client in a javascript variable.
Whenever an update is done on the data tables on the server, you update the state counter there.
Your AJAX polling calls would then query this state counter, compare it to the corresponding javascript variable, and only do a data-update call if it has changed, updating the local javascript variable to what the server has.
In order to avoid having to poll for each datatype separately, you might want to use an JS object with a member for each datatype.
Note: yes this is all very theoretical, but, hey, so is the question ;)

Where to put/generate large javascript data files in front end with cakephp

I have an app that uses a large database to fill in google maps and charts data. This ends up being about 5000 lines per column and about 20 columns. The issue I am running into is whether to put this data in the view template, which makes my source code several thousand lines long, or generating a javascript file for each instance and including them in the view. The issue I am running into with that method is that I am generating files with no way to delete them out of the webroot folder (without a cron job to go through and delete old ones). I was wondering what the solution is for this.

of course, but you as a developer are responsable for fast delivering websites. you cannot fetch all of the data. for example when using google maps it is common practice that you display a limited number of data according to the displayed area ( rectangle ). When using charts, then you should answer only with the already aggregated data.
there is almost never the reason to display all the data to the user. Ask yourself, what will you do when you see 5000 thousands lines of code on the website. do you need it all at once? No.
Use AJAX to fetch only the rows you need right now.

Can I access my requestScoped JavaBean-objects both client side and server side?

Short Version:
In a java web app, I can set javabean objects as requestscoped parameters, and access detailed fields, even hierarchical, within these objects through i.e. JSTL/EL while building a website.
Can I in any way access the full extent of these JavaBeans, in i.e. javascript-functions that are fired on, for instance, onclick of some elements within my web page?
Long version:
I am making a java web-app, and I am trying to learn the basics, so I am not using any frameworks like Spring or Struts, but I am building the app by the front controller pattern.
I have a page, which should be able to create new, recieve, edit and/or finally update data in my database. The database has foreign keys, and my choices in the editor should depend on the number of elements of other linked tables in the database.
I would like my editor to be able to:
Create the editor menu based on secondary tables of the database (static until leaving the site)
Load data from a database-element
Edit data in html-elements
Undo changes (which is basically repeating step 2, if data is available still)
Save data, and reset editor.
Point 4 is the center of attention here. I wish to be able to do step 4 without reloading the whole page. If I am able to do this, I figure step 2 should also be executed client side, as it does the same thing, only first time. It feels like a setup like this will grant me a good seperation of creating the form itself server side, and let step 2-4 happen client side, until step 5 again requires server side action.
I am not sure how to approach this goal though, or if it is a good idea. It is only a problem, when data is loaded from the database, and I want to store that data client side. Right now I am building the form in jsp/html/jstl, and I am using requestScoped java objects to do it, through a HttpServletRequest-object from the Servlet Controller. I have been trying to use these objects in javascript functions, with limited success. I have been able to extract all data, even hierirchal object's fields, except those in collections. Unfortunately these are essential to my editor page.
I have been looking into JSON for this, but is seems like i need to do big adjustements in my java code to implement this. Is it worth it?
finally, to repeat the question: How can I access requestScoped JavaBean-objectdata to be available client side, in i.e. Javascript?

Reduce requested file size or reduce number of browser calculations?

I have some data that I want to display on a web page. There's quite a lot of data so I really need to figure out the most optimized way of loading and parsing it. In CSV format, the file size is 244K, and in JSON it's 819K. As I see it, I have three different options:
Load the web page and fetch the data in CSV format as an Ajax request. Then transform the data into a JS object in the browser (I'm using a built-in method of the D3.js library to accomplish this).
Load the web page and fetch the data in JSON format as an Ajax request. Data is ready to go as is.
Hard code the data in the main JS file as a JS object. No need for any async requests.
Method number one has the advantage of reduced file size, but the disadvantage of having to loop through all (2700) rows of data in the browser. Method number two gives us the data in the end-format so there's no need for heavy client-side operations. However, the size of the JSON file is huge. Method number three has the advantage of skipping additional requests to the server, with the disadvantage of a longer initial page load time.
What method is the best one in terms of optimization?

In my experience, data processing times in Javascript are usually dwarfed by transfer times and the time it takes to render the display. Based on this, I would recommend going with option 1.
However, what's best in your particular case really does depend on your particular case -- you'll have to try. It sounds like you have all the code/data you need to do that anyway, so why not run a simple experiment to see which one works best for you.