here is the problem. I have no right to create a database and I receive a csv that countain MASSIVE amount of data each day. (More than 200 000 rows)
Data that I must make accessible for everybody on the intranet. So I created a simple html/php page that extract all the rows and display those informations in a table with a filter on every column with a simple fgetcsv.
Problem is that the web browser is not suited to display that much informations at the same time so it makes it crash or freeze for a while, and you can't do anything for a while.
I wanted to know if anyone knew a way to say to the page "load only the first 100 rows for exemple, then automatically create a next page that will load and display the next 100 rows etc."
I manages to DISPLAY only the first x rows and then when you clicked a button the table would expand with the next x rows, but they are still all loaded at once. The y remaining are just hidden, so the browser still dies or freeze.
Any idea ?
Thanks
It's a generic pagination question really. It doesn't matter if your data is stored in database or in a CSV file.
Just pass some offset argument to your PHP script via query string or URL rewriting and use it to select only part of your CSV list.
Like this: /big-table.php?page=3.
// Getting passed argument.
$pageNumber = (int) $_GET['page'];
// Items per page default.
$itemsPerPage = 100;
// Calculating offset.
$offset = ($pageNumber - 1) * $itemsPerPage;
Then use the $offset and $itemsPerPage to retrieve only part of your CSV file by limiting scope of your CSV parsing loop.
You can also pass items per page value as an argument to your script in order to control this value from your web interface. For example, if you want to create a dropdown menu with ability to select 10, 50, 100 items per page, etc.
And if you want it - you could always use AJAX to fetch more items dynamically, it doesn't really affect your pagination implementation server-side, only an output format (JSON instead of HTML).
Of course database implementation will work faster and I would recommend to opt for it instead if possible. And/Or you can use some caching layer to speed things up.
You could use the jquery Datatables plugin http://datatables.net/
It's quite simple to do what you want using that.
Refer to either this example: http://datatables.net/examples/data_sources/ajax.html or http://datatables.net/examples/data_sources/server_side.html
The most sensible thing is to demand database access. Right now, you're told to build a car but without using wheels and an engine.
Right now, you could use PHP to split the big csv in multiple smaller files of n rows long. You have to do this only once, or once a day/hour if the big csv is updated. Then you load each of these files only when you need them, either by navigating to another page or dynamically using Javascript.
Related
I looked at "Generating HTML Page on the fly" on this website, but most of it was over my head.
I have a 2 part question that I would like assistance with please.
I want to fill a narrow vertical container, <div id=”counter”> with the numbers 1 .. <xx>.
<xx> is determined by the record count of a database, filtered “on-the-fly”, by the user choosing a category (no problem there – I have an SQL background)
Eg. Category1: 1 .. 200
Category2: 1 .. 6
These numbers could change over time, as I want to allow users to add content to the database (vetted of course).
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
I have created a small website of a similar nature to that, hard-coding all the images and links, but I am looking at 3000+ images (as a starting point here), and they differ per page.
I have created this scenario many times in stand-alone apps and from past experience, I thought perhaps, I could create a javascript routine which would use a loop to
• print the numbers to the <div> using the getelementbyID ( ).
• Fill an array with the record number, a title and an image link.
Question 1: Is this possible or am I beating a “dead horse”?
If it is possible, any suggestions would be gratefully accepted.
Part 2:
My current idea is that, as the user hovers the mouse over any number, a mouseover ( ) event will occur that will read the appropriate array record and display the <title> as a tool-tip-text.
If the user clicks the number, a function (I have yet to write) will read the appropriate array record and attach the image link to an <a> tag, and subsequently display the appropriate image to the screen.
Question 2: repeat of question 1.
I have viewed a number of website source code pages (of similar ideas eg. Surgicalexam.com), but they have all been hard-coded and are distinct pages per category.
Why are you so sure about that? You can't see php-code, because it is executed on the server. There is no way to know if it was hardcoded or by php
Answer:
It is possible.
If I understand this correctly, you want to read some data from a database and if the user clicks / hovers something, you want to load more data?
You have to splitt this into two things:
Load data with PHP from the db (Server side)
If you want a live, visual feedback you need JavaScript (and/or CSS3) to do changes. (Client side)
One possible solution is to create a API with php (maybe REST-like) and then call that api with JavaScript.
You could also do everything with PHP but this will require a reload of the website on every click. PHP cannot do changes On-The-Fly.
First of all you should learn the basics about web development.
And most important: If you decide to learn Web-Programming: learn about security, too. For example things like Cross Site Scripting and SQL-Injection. Never trust data coming from a client (e.g. JavaScript)!
I have a php page that shows track listings of classical CD's. The table draws data from two very similar but different mysql tables which sometimes leads to (partial) duplication of table rows.
These are the two possibilities (with the number being the track number on the CD):
1 Work title
1 Subwork title
Work title (possibly altered)
2 Work title
In the first case I want to remove the track number. In the second case I want to retain the work title in the first line, add the track number of the second line and do nothing with the work title of the second line. So the result should look like:
Work title
1 Subwork title
2 Work title (possibly altered)
Can this be accomplished in php or should I look in the direction of javascript or jQuery for a solution? I tried using
$("div").empty();
but couldn't make it work.
All suggestions are welcome.
The most efficient way would be to process the data on the server (PHP) as it is retrieved from the database. This way you aren't transferring large data sizes to the client only to have the front-end (javascript) to discard anything of the data this isn't needed. This will create a not so desirable user experience.
If your PHP script is generating your entire table at once, then you can do it in PHP. If your PHP script only creates part of the table, sends it to the user, and it is later modified by something like an AJAX call, then you have to do it in Javascript.
In one of my requirements, I get the log file (can be around 10MB) from my WebServer, and display a table with its content as log entries (using javascript). The problem I face is that it creates too many table rows, due to which page becomes less responsive.
I want to know if too many objects in HTML makes a page slow ?
Also, what should be workaround for this ?
Thanks in advance.
What does your javascript look like? Depending on how you're building the DOM, it could be the bottleneck. For example:
var html = [];
for(var i=0;i<rows.length;i++) {
html.push('<tr><td>' + rows[i] + '</td></tr>');
}
$('#tbody').append(html.join(''));
is going to be a lot faster than
for(var i=0;i<rows.length;i++) {
$('#tbody').append('<tr><td>' + rows[i] + '</td></tr>');
}
This is assuming you're using jquery. Also, as others have mentioned, use pagination to keep limit the number of elements on the screen.
If your HTML is complex, you may want to look into a template engine, and some help choosing the one that is right for you. I use dust because linkedin did all the hard evaluation work.
So, given this is an AJAX call I see a few options (in order of best performance):
Add paging to the service call and allow the AJAX query to only pull down specific batches of entries. Then implement next/previous page buttons that would make successive calls to retrieve more/less information. e.g. /ajax/log would be your initial call, then clicking next button would call /ajax/log?page=2; Previous button re-calls /ajax/log?page=1.
Pull all the data down in a single call, but only selectively output it to the DOM. KnockoutJS makes ouputting information in a paging format very simple (and there are examples here on Stackoverfow).
Dump it as-s and add a script like jQuery DataTables. This will hide rows and implement paging based on what's been pushed in to the table itself. It requires very little additional programming other than load up the table via ajax, then call $('mytable').datatable().
I want to create a list using Angular, where the data (say like over 1000 items) is retrieved from the server using JSON call. In my plan, the list will only show x amount of items in the list, and will load x more items into it when the user pull up the list.
So my question here is, how to write the expression for the ng-repeat so that
It will only show x amount of items when first loading the list
It will only load x amount of items when user pulled up the list
I think i read somewhere about ngRepeat being able to do this kind of feature, to limit and control the amount of items shown or loaded onto the list, but i forgot where i read it.
But someone told me that better way of doing this is by limiting the amount of data retrieved when using JSON call rather than letting Angular do it.
So i'm kind of confuse in here.
Take a look at ngInfiniteScroll http://binarymuse.github.io/ngInfiniteScroll/.
Last fall, i want to use a similar solution and ended up there.
I have a database no more than say 100k which I'd like to use as reference documentation for my software. It's just a simple table really - around 5 columns by a couple of hundred rows. I am looking for a decent Javascript database library; one which would feature:
Sorting by column
A tiny size. Has to be small as it is sent to the user (since I don't want anything server-side). Say no more than 50-100k.
"Update-as-you-type" functionality, and by that I mean, you can type in a filter box, and the rows filter out instantly on the HTML page, or near instantly as you're typing (client-side processing only). If no input in the filter is given, all the results would display on a single large HTML page.
Searches that allow for partial matches of any cell in the table, and preferably allow NOT, OR and obviously AND.
Furthermore, it should be free/cheap, easy to use and install, perhaps working on a CSV data file for its data.
Is there anything out there that fits the bill?
Check jQGrid OR DataTables it has most of that what you are looking for.