We Keep Coding

architecture for get and store api request data

This is more of a architectural questions. An external platform had product and price information for let's say, books. There is an API available to get this information.
What I read is that it should be possible to create a function in Javascript and connect the Javascript to a page where you want to show the data on my own website. This would mean that for each page request an API-call is made. Since the requested information only changes once a day maximum this does not sound the most efficient solution.
Can someone advise a better solution? Something into the direction of a similar php or javascript function that does the request on the background, schedule an update and import the data into mysql? If so, what language would be most common.
I need the solution for a Joomla/php/mysql environment

Here's a simple idea - fetch and store results from the API (ones you think aren't gonna change in a day), either on disk, or in the database, and later use these stored results to retrieve what you otherwise would've fetched from the API.
Since storing anything in frontend JS across page reloads isn't easy, you need to make use of PHP for that. Based on what's given, you seem to have two ways of calling the API:
via the frontend JS (no-go)
via your PHP backend (good-to-go)
Now, you need to make sure your results are synced every (say) 24 hours.
Add a snippet to your PHP code that contains a variable $lastUpdated (or something similar), and assign it the "static" value of the current time (NOT using time()). Now, add a couple of statements to update the stored results if the current time is at least 24 hours greater than $lastUpdated, followed by updating $lastUpdated to current time.
This should give you what you need with one API call per day.
PS: I'm not an expert in PHP, but you can surely figure out the datetime stuff.

It sounds like you need a cache, and you're not the first person to run into that problem - so you probably don't need to reinvent the wheel and build your own.
Look into something like Redis. There's an article on it available here as well: https://www.compose.com/articles/api-caching-with-redis-and-nodejs/

SetInterval mySQL query every second

I would like to implement a very small and simple group chat into my website without using any 3rd party libraries or anything.
Therefore I thought of using jQuery Ajax and Javascript's setInterval function.
On page load I am going to have a div, with all the chat messages inside, that have been posted so far + a hidden input with the max(id) of the chat-table in the database. Then I would start setInterval with an ajax request to the database every second, which returns all new messages (id > the one in the hidden input), adds them to the DOM and updates the hidden input to the new max(id).
Furthermore I thought of deleting all messages older than 48 hours from the database, to keep the chat-table very small.
Do you have any concerns about this? Will it significantly impair the performance of the site?
The site may have up to 100 concurrent users at a time, but only half of them will have access to the chat.

There are a couple things to take notice with this:
The front end (your jQuery code) can't talk to the database. You Need a server set up. since you are using Javascrpt/jquery in the front end I suggest node, it's the same language so you won't have to learn another language.
There is a lot of things you'll need, a server (i suggest node(express)), web socket exposure(I suggest socket.io), a database (I suggest redis for simplicity).
without using 3rd party libs is just not an option, even simple chats are a lot more complex and difficult than originally thought.

JavaScript: Stable method of updating connected clients variables

I have just stepped into the world of Web Development, and I am developing a small browser game that simply allows connected users to take control of an object (a triangle currently!), and simply move around the screen area.
Currently, I store the clients co-ordinate position in a MySQL database, and update that position using AJAX, roughly 30 times per second.
Other clients positions are also polled roughly 30 times per second.
My problem however, is that this seems to be causing an hour long IP lockout for the client, which I assume is automatically occurring on my Host's end. Would this perhaps be a normal default precautionary action? I was under the impression that 30 AJAX polls in a second was not a particularly stressful amount, however as I mentioned this is a new field for me. I'm fearful I've created some miniscule DOS attack!
If so, I would be grateful if someone with experience in this matter could point me to a more efficient method of handling the kind of interactivity I have described. This is all leading up to a six-month project I will be working on alone for my final year University project, so I'm more than happy to put the extra hours in to learn a better solution.

What you should do is known as "hybrid-polling". Basically you have a long running method server side which is running an "infinite" loop which runs once every 33ms (30 times per second). This loop will shoot data out to a part of your front end if the data has changed. When the data gets to be too large in the buffer for the method to handle, the method exits. The whole time your client is polling to see if new data was written. If the method exits, the client must restart the method. This is a hybrid approach, where the client polling is only checking client side data, except when the method exits, in which case the client must poll again to restart the server method, which then runs once every 33ms and pushes data out to the client.

Look up Comet (compatible with older browsers but not as efficient as possible), BOSH, or Web Sockets (ideal but not compatible with older browsers) for other approaches.

How are timed html forms, like the kind you encounter when doing online tests, properly coded using PHP, MySQL, HTML, CSS and JavaScript?

In other words, what technology would take care of doing the time tracking? Would it be the JavaScript? I don't see being able to keep track of such things with PHP.
Here's what I need to accomplish:
I need to have a long form spanning many web page reloads because it is pretty much an online test where each page load displays a new question. However, the entire form, which constitutes one test has a time limit. When the time expires if the user in question has not completed the test then he/she cannot submit a partially completed test nor attempt to do the entire test all over again within the same day (either calendar date or with 12/24 hours in between.) The user may, however, comeback the next day and attempt to finish the entire test again within the allotted time. I know I have added a lot of details and I did this just to show context, nevertheless, the main difficulty for me would be in how accomplish the time expiry feature. That is, somehow, within a series of pages that make up a form representing an online test I want to track the time starting from the first question (one page load) and upon time expiry for the test to be disabled.
Has anyone ever done this? Would anyone have any tips for me on how to accomplish this?
Any advice I can get would totally be appreciated in advance.

If you do track time on client-side - always validate it on the server-side.
Never trust the client, by itself, to validate the time. As mentioned in the comments, client-side time validation is only good for cosmetic features, never for actual validation.
The easiest way to accomplish this is to add a unique token to the form (which is not spoofable) on first navigation. Cookies, or any other sort of session management technique you get from your framework will suffice.
On form submission you can first validate this on client side and return an error if time has passed, even before actually sending the form. If successful, submit the form, and make sure you validate the token upon processing on the server.

There are two ways you could measure the "time they spent" on the form.
When the first page of the form is severed, in the PHP create a session variable containing the date. When they finish the form, you subtract the current date form the beginning date. This gives you the total time it took from when the form was served and finished. However, this is not 100% accurate as there could downtime for other reasons such as slow internet.
You could have JavaScript record the time on that page (I answered how to do that here: here) Using AJAX, this time could be sent that way or by using GET posts. The time would then be kept by PHP somehow and added up in the end.
Hope this helps! Just ask if you want an example.

In the most general terms, you'll need to set up a session on the server side to track each user and test. When the user begins the test, you stamp a variable (server side) with the test start time. As the user progresses through the test and requests additional pages, check whether the difference between the current time and that variable have exceeded the time allowed for the test. If the test has expired, instead of delivering the next test page, you can close up the test and deliver a "time's up" page to the user.
I don't know what server-side environment you're using, but it almost certainly has some sort of session management framework available. To reliably control the testing environment, you have to manage this from the server side.

Ajax "Is there new content? If so, update page" - How to do this without breaking the server?

It's a simple case of a javascript that continuously asks "are there yet?" Like a four year old on a car drive.. But, much like parents, if you do this too often or, with too many kids at once, the server will buckle under pressure..
How do you solve the issue of having a webpage that looks for new content in the order of every 5 seconds and that allows for a larger number of visitors?

stackoverflow does it some way, don't know how though.
The more standard way would indeed be the javascript that looks for new content every few seconds.
A more advanced way would use a push-like technique, by using Comet techniques (long-polling and such). There's a lot of interesting stuff under that link.
I'm still waiting for a good opportunity to use it myself...
Oh, and here's a link from stackoverflow about it:
Is there some way to PUSH data from web server to browser?

In Java I used Ajax library (DWR) using Comet technology - I think you should search for library in PHP using it.
The idea is that server is sending one very long Http response and when it has something to send to the client it ends it and send new response with updated data.
Using it client doens't have to ping server every x seconds to get new data - I think it could help you.

You could make the poll time variable depending on the number of clients. Using your metaphor, the kid asks "Are we there yet?" and the driver responds "No, but maybe in an hour". Thankfully, Javascript isn't a stubborn kid so you can be sure he won't bug you until then.

You could consider polling every 5 seconds to start with, but after a while start to increase the poll interval time - perhaps up to some upper limit (1 minute, 5 minute - whatever seems optimal for your usage). The increase doesn't have to be linear.
A more sophisticated spin (which could incorporate monzee's suggestion to vary by number of clients), would be to allow the server to dictate the interval before next poll. The server could then increase the intervale over time, and you can even change the algorithm on the fly, or in response to network load.

You could take a look at the 'Twisted' framework in python. It's event-driven network programming framework that might satisfy what you are looking for. It can be used to push messages from the server.

Perhaps you can send a query to a real simple script, that doesn't need to make a real db-query, but only uses a simple timestamp to tell if there is anything new.
And then, if the answer is true, you can do a real query, where the server has to do real work !-)

I would have a single instance calling the DB and if a newer timestamp exists, put that new timestamp in a application variable. Then let all sessions check against that application variable. Or something like that. That way only one innstance are calling the sql-server and the number of clients does'nt matter.
I havent tried this and its just the first idéa on the top of the head but I think that cashe the timestamp and let the clients check the cashe is a way to do it, and how to implement the cashe (sql-server-cashe, application variable and so on) I dont know whats best.

Regarding how SO does it, note that it doesn't check for new answers continuously, only when you're typing into the "Your Answer" box.
The key then, is to first do a computationally cheap operation to weed out common "no update needed" cases (e.g., entering a new answer or checking a timestamp) before initiating a more expensive process to actually retrieve any changes.
Alternately, depending on your application, you may be able to resolve this by optimizing your change-publishing mechanism. For example, perhaps it might be feasible for changes (or summaries of them) to be put onto an RSS feed and have clients watch the feed instead of the real application. We can assume that this would be fairly efficient, as it's exactly the sort of thing RSS is designed and optimized for, plus it would have the additional benefit of making your application much more interoperable with the rest of the world at little or no cost to you.

I believe the approach shd be based on a combination of server-side sockets and client-side ajax/comet. Like:
Assume a chat application with several logged on users, and that each of them is listening via a slow-load AJAX call to the server-side listener script.
Whatever browser gets the just-entered data submits it to the server with an ajax call to a writer script. That server updates the database (or storage system) and posts a sockets write to noted listener script. The latter then gets the fresh data and posts it back to the client browser.
Now I haven't yet written this, and right now I dunno whether/how the browser limit of two concurrent connections screws up the above logic.
Will appreciate hearing fm anyone with thoughts here.
AS