We Keep Coding

architecture for get and store api request data

This is more of a architectural questions. An external platform had product and price information for let's say, books. There is an API available to get this information.
What I read is that it should be possible to create a function in Javascript and connect the Javascript to a page where you want to show the data on my own website. This would mean that for each page request an API-call is made. Since the requested information only changes once a day maximum this does not sound the most efficient solution.
Can someone advise a better solution? Something into the direction of a similar php or javascript function that does the request on the background, schedule an update and import the data into mysql? If so, what language would be most common.
I need the solution for a Joomla/php/mysql environment

Here's a simple idea - fetch and store results from the API (ones you think aren't gonna change in a day), either on disk, or in the database, and later use these stored results to retrieve what you otherwise would've fetched from the API.
Since storing anything in frontend JS across page reloads isn't easy, you need to make use of PHP for that. Based on what's given, you seem to have two ways of calling the API:
via the frontend JS (no-go)
via your PHP backend (good-to-go)
Now, you need to make sure your results are synced every (say) 24 hours.
Add a snippet to your PHP code that contains a variable $lastUpdated (or something similar), and assign it the "static" value of the current time (NOT using time()). Now, add a couple of statements to update the stored results if the current time is at least 24 hours greater than $lastUpdated, followed by updating $lastUpdated to current time.
This should give you what you need with one API call per day.
PS: I'm not an expert in PHP, but you can surely figure out the datetime stuff.

It sounds like you need a cache, and you're not the first person to run into that problem - so you probably don't need to reinvent the wheel and build your own.
Look into something like Redis. There's an article on it available here as well: https://www.compose.com/articles/api-caching-with-redis-and-nodejs/

SQL Data & JavaScript - Best Practice

Let's say a website needs to pull information from a specific table in a database based on a user's menu selection. That table's data is then fed into some JS equations and thrown onto the page.
What is the best way to go about pulling that table's information? I've read that trying to access an SQL database via JavaScript is bad practice, so is there another way to do this? I know about PHP's json_encode, but I guess I'm not entirely sure
What the syntax is if I'm calling PHP from a JS script, and
If that's 'best' practice. Still relatively new to this, so I'd like to do this right.
Another option as far as I'm concerned is attempting to pull ALL of the possible tables (not a security concern) at once on page load. I expect that'd introduce a good deal of latency, though.

It looks to me that you are not really sure what technique to use. Here are some options. I'm not going to type them here because, there is enough to find about each one:
plain php: w3schools
pure ajax call: stackoverflow
jquery: jquery
Ajax calls are more user friendly and many times more efficient because, you don't have to refresh the page. I usually get all information at once( as long your mysql data is not to big). As for security: You use php either way so it doesn't matter if you use Ajax or not. Oh and don't select valuable data of users data (like password or their emails). I hope you get more overview after this :)

which is better, searching in javascript or database?

I have a grid(employee grid) which has say 1000-2000 rows.
I display employee name and department in the grid.
When I get data for the grid, I get other detail for the employee too(Date of Birth, location,role,etc)
So the user has option to edit the employee details. when he clicks edit, I need to display other employee details in the pop up. since I have stored all the data in JavaScript, I search for the particular id and display all the details. so the code will be like
function getUserDetails(employeeId){
//i store all the employeedetails in a variable employeeInformation while getting //data for the grid.
for(var i=0;i<employeeInformation.length;i++){
if(employeeInformation[i].employeeID==employeeId){
//display employee details.
}
}
}
the second solution will be like pass employeeid to the database and get all the information for the employee. The code will be like
function getUserDetails(employeeId){
//make an ajax call to the controller which will call a procedure in the database
// to get the employee details
//then display employee details
}
So, which solution do you think will be optimal when I am handling 1000-2000 records.
I don't want to make the JavaScript heavy by storing a lot of data in the page.
UPDATED:
so one of my friend came up with a simple solution.
I am storing 4 columns for 500 rows(average). So I don't think there should not be rapid slowness in the webpage.
while loading the rows to the grid, under edit link, I give the data-rowId as an attribute so that it will be easy to retrieve the data.
say I store all the employee information in a variable called employeeInfo.
when someone clicks the edit link.. $(this).attr('data-rowId') will give the rowId and employeeInfo[$(this).attr('data-rowId')] should give all the information about the employee.
instead of storing the employeeid and looping over the employee table to find the matching employeeid, the rowid should do the trick. this is very simple. but did not strike me.

I would suggest you make an AJAX call to the controller. Because of two main reasons
It is not advisable to handle Database actiity in javascript due to security issues.
Javascript runs on client side machine it should have the least load and computation.
Javascript should be as light as possible. So i suggest you do it in the database itself.

Don't count on JavaScript performance, because it is heavily depend on computer that is running on. I suggest you to store and search on server-side rather than loading heavy payload of data in Browser which is quite restricted to resources of end-user.
Running long loops in JavaScript can lead to an unresponsive and irritating UI. Use Ajax calls to get needed data as a good practice.

Are you using HTML5? Will your users typically have relatively fast multicore computers? If so, a web-worker (http://www.w3schools.com/html/html5_webworkers.asp) might be a way to offload the search to the client while maintaining UI responsiveness.
Note, I've never used a Worker, so this advice may be way off base, but they certainly look interesting for something like this.

In terms of separation of concerns, and recommended best approach, you should be handling that domain-level data retrieval on your server, and relying on the client-side for processing and displaying only the records with which it is concerned.
By populating your client with several thousand records for it to then parse, sort, search, etc., you not only take a huge performance hit and diminish user experience, but you also create many potential security risks. Obviously this also depends on the nature of the data in the application, but for something such as employee records, you probably don't want to be storing that on the client-side. Anyone using the application will then have access to all of that.
The more pragmatic approach to this problem is to have your controller populate the client with only the specific data which pertains to it, eliminating the need for searching through many records. You can also retrieve a single object by making an ajax query to your server to retrieve the data. This has the dual benefit of guaranteeing that you're displaying the current state of the DB, as well as being far more optimized than anything you could ever hope to write in JS.

Where to render Ajax search results in an object oriented approach using Coldfusion?

I'm updating a Coldfusion8/MySql site with a fairly complex search from "spaghetti to object" (= separate view, controller and process - no framework, everything handled by Jquery Mobile).
I need to run the search query through Jquery-AJAX and am now posting the search form to my searchProcess.cfc, which does the database queries.
Question:
I'm not sure where to render the results?
The results will be fairly complex (database with a few million records, rows of 40 fields) and should end up in a single result layout or a multiple result layout file.
I was thinking of constructing the files inside the cfc and handing them back via cfsavecontent, but I'm reading everywhere this is a no-no...
What are the alternatives then?
I could set up a template_single.cfm and template_multi.cfm, pass back pure search results as AJAX response and then fire another AJAX call from the success handler to call the template and then render the output of this 2nd call. This seems awfully complicated, plus I don't see where I can fit my pagination in there without passing around large datasets.
So I'm looking for some advice on how to handle search-results in an object-oriented-way?
Thanks for input!
EDIT:
After a few more hours of googling, I'm currently looking at the following option:
1.) run a single database query to return paginated results - as per here
2.) send data with 0-25 records back to AJAX in JSON
3.) trying to use a template cf/js in a loop (length 1 or length 25) - as per here
This would mean data-transfer only for 1-25 raw records in JSON. If I try to render in the success handler, I'm not having to http-request another template.
Does this approach make sense?

First off, I see nothing wrong with putting display logic in a .cfc that is specifically for display logic. I know it's not strict MVC, but depending on what you're doing, it can work just fine. What I don't like is .cfc's that do any sort of output within the function. I always hand back any data from the function.
/end opinion
For this particular problem, I second EDIT idea of setting up the view to be almost all HTML/jQuery with AJAX calls for paginated recordsets. As far as the single/multiple results, I'd go with separate jQuery functions depending on which one you needed. The nice thing about this is that you could have the multiple recordset display call the single record display to view a single record (while still retaining the full recordset in the Dom).
In both cases, I highly recommend getting a Javascript Dump function (helps so much in visualizing the DOM data.)
PS. If anybody finds any newer/better JS Dump functions that work like cfdump, please, please, please let me know!

Load up entire table again or add row manually?

This is an ajax questions. I have a table this table shows users information at certain time depending on what settings the user sets.
Now in some cases a user will see the information right away and in some cases they won't it all depends on when they want to see the information.
Now what should I do?
Should I do? do a post and post their data and then do a ajax get after to get the table and render it?
*I probably could it all in the post but unless some huge performance gain is gained I rather not otherwise I have to do mix "success/fail" messages and the table to be rendered all in the same response.
So each one seems to have pluses and minuses.
Ajax way
don't have to worry about having a
JavaScript solution that queries the
database to figure out what their
timezone is and then determine if the
row should be added or not and any
other headaches that comes with
javascript dates.
Each row could potential have a
different style to. This would
mean I would have to possibly do a
query to the database and figure it
out or have hidden field in the page
for easy access. With Ajax way I
would not have to worry about it.
don't have to worry about making a
manual row in javascript/jquery
syntax what can be a pain to do if
you have many columns.
Javascript way
Problem less of a performance hit
since only have to potentially make
one new or do nothing. Where
otherwise I have to generate a new
table regardless and if it has lots
of rows in it that could be slow.
Have to rebind all jquery plugins
that would be on the table. Or use
jquery.live for everything else.
So I am not sure to me it seems like a hard choice.

Unless I misunderstand what you want to do, why not do both in one solution?
Return a JSON response, so, when a user logs in, you post the information using an ajax call, and just return the data from the database. I tend to return either data or an error message, but you could have two objects in your json string, one for a possible error message and the other being the data that is being returned.
The javascript then can process the data as is needed.
So, you do both, it isn't an either/or decision.