I'm trying to GET a large amount of data from the API (over 300k records). It has pagination (25 records per page) and request limit is 50 request per 3 minutes. I'm using PHP curl to get the data. The API needs JWT token authorization. I can get a single page and put its records into an array.
...
$response = curl_exec($curl);
curl_close($curl);
$result = json_decode($response, true);
The problem is I need to get all records from all pages and save it into array or file. How to do it? Maybe I should use JS to do it better?
Best regards and thank you.
Ideally use cron and some form of storage, database or a file.
It is important that you ensure a new call to the script doesn't start unless the previous one has finished, otherwise they start stacking up and after a few you will start having server overload, failed scripts and it gets messy.
Store a value to say the script is starting.
Run the CURL request.
Once curl has been returned and data is processed and stored change the value you stored at the beginning to say the script has finished.
Run this script as a cron in the intervals you deem necessary.
A simplified example:
<?php
if ($script_is_busy == 1) exit();
$script_is_busy = 1;
// YOUR CURL REQUEST AND PROCESSING HERE
$script_is_busy = 0;
?>
I would use a series of requests. A typical request takes at most 2 seconds to fulfill, so 50 requests per 3oo secs does not require parallel requests. Still you need to measure time and wait if you don't want to be banned for DoS. Note that even with parallelism, curl supports it as far as I remember. When you reach the request limit you must use the sleep function to wait until you can send new requests. For PHP the real problem that it is a long running job, so you need to change settings, otherwise it will timeout. You can do it this way: Best way to manage long-running php script? As of nodejs, I think it is a lot better solution for this kind of async tasks, because the required features come naturally with nodejs without extensions and such things, though I am biased towards it.
Okay. I misinterpreted what you needed. I have more questions.
Can you do one request and get your 50 records immediately? That is assuming when you said 50 requests per 3 minutes you meant 50 records.
Why do you think there is this 50/3 limitation?
Can you provide a link to this service?
Is that 50 records per IP address?
Is leasing 5 or 6 IP addresses an option?
Do you pay for each record?
How many records does this service have total?
Do the records have a time limit on their viability.
I am thinking if you can use 6 IP addresses (or 6 processes) you can run the 6 requests simultaneously using stream_socket_client().
stream_socket_client allows you to make simultaneous requests.You then create a loop that monitors each socket for a response.
About 10 years ago I made an app that evaluated web page quality. I ran
W3C Markup Validation
W3C CSS Validation
W3C Mobile OK
WebPageTest
My own performance test.
I put all the URLs in an array like this:
$urls = array();
$path = $url;
$url = urlencode("$url");
$urls[] = array('host' => "jigsaw.w3.org",'path' => "/css-validator/validator?uri=$url&profile=css3&usermedium=all&warning=no&lang=en&output=text");
$urls[] = array('host' => "validator.w3.org",'path' => "/check?uri=$url&charset=%28detect+automatically%29&doctype=Inline&group=0&output=json");
$urls[] = array('host' => "validator.w3.org",'path' => "/check?uri=$url&charset=%28detect+automatically%29&doctype=XHTML+Basic+1.1&group=0&output=json");
Then I'd make the sockets.
foreach($urls as $path){
$host = $path['host'];
$path = $path['path'];
$http = "GET $path HTTP/1.0\r\nHost: $host\r\n\r\n";
$stream = stream_socket_client("$host:80", $errno,$errstr, 120,STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT);
if ($stream) {
$sockets[] = $stream; // supports multiple sockets
$start[] = microtime(true);
fwrite($stream, $http);
}
else {
$err .= "$id Failed<br>\n";
}
}
Then I monitored the sockets and retrieved the response from each socket.
while (count($sockets)) {
$read = $sockets;
stream_select($read, $write = NULL, $except = NULL, $timeout);
if (count($read)) {
foreach ($read as $r) {
$id = array_search($r, $sockets);
$data = fread($r, $buffer_size);
if (strlen($data) == 0) {
// echo "$id Closed: " . date('h:i:s') . "\n\n\n";
$closed[$id] = microtime(true);
fclose($r);
unset($sockets[$id]);
}
else {
$result[$id] .= $data;
}
}
}
else {
// echo 'Timeout: ' . date('h:i:s') . "\n\n\n";
break;
}
}
I used it for years and it never failed.
It would be easy to gather the records and paginate them.
After all sockets are closed you can gather the pages and send them to your user.
Do you think the above is viable?
JS is not better.
Or did you mean 50 records each 3 minutes?
This is how I would do the pagination.
I'd organize the response into pages of 25 records per page.
In the query results while loop I'd do this:
$cnt = 0;
$page = 0;
while(...){
$cnt++
$response[$page][] = $record;
if($cnt > 24){$page++, $cnt = 0;}
}
header('Content-Type: application/json');
echo json_encode($response);
Is it possible to redirect a user to a different page through the use of PHP?
Say the user goes to www.example.com/page.php and I want to redirect them to www.example.com/index.php, how would I do so without the use of a meta refresh? Is it possible?
This could even protect my pages from unauthorized users.
Summary of existing answers plus my own two cents:
1. Basic answer
You can use the header() function to send a new HTTP header, but this must be sent to the browser before any HTML or text (so before the <!DOCTYPE ...> declaration, for example).
header('Location: '.$newURL);
2. Important details
die() or exit()
header("Location: https://example.com/myOtherPage.php");
die();
Why you should use die() or exit(): The Daily WTF
Absolute or relative URL
Since June 2014 both absolute and relative URLs can be used. See RFC 7231 which had replaced the old RFC 2616, where only absolute URLs were allowed.
Status Codes
PHP's "Location"-header still uses the HTTP 302-redirect code, this is a "temporary" redirect and may not be the one you should use. You should consider either 301 (permanent redirect) or 303 (other).
Note: W3C mentions that the 303-header is incompatible with "many pre-HTTP/1.1 user agents. Currently used browsers are all HTTP/1.1 user agents. This is not true for many other user agents like spiders and robots.
3. Documentation
HTTP Headers and the header() function in PHP
What the PHP manual says
What Wikipedia says
What the W3C says
4. Alternatives
You may use the alternative method of http_redirect($url); which needs the PECL package pecl to be installed.
5. Helper Functions
This function doesn't incorporate the 303 status code:
function Redirect($url, $permanent = false)
{
header('Location: ' . $url, true, $permanent ? 301 : 302);
exit();
}
Redirect('https://example.com/', false);
This is more flexible:
function redirect($url, $statusCode = 303)
{
header('Location: ' . $url, true, $statusCode);
die();
}
6. Workaround
As mentioned header() redirects only work before anything is written out. They usually fail if invoked inmidst HTML output. Then you might use a HTML header workaround (not very professional!) like:
<meta http-equiv="refresh" content="0;url=finalpage.html">
Or a JavaScript redirect even.
window.location.replace("https://example.com/");
Use the header() function to send an HTTP Location header:
header('Location: '.$newURL);
Contrary to what some think, die() has nothing to do with redirection. Use it only if you want to redirect instead of normal execution.
File example.php:
<?php
header('Location: static.html');
$fh = fopen('/tmp/track.txt', 'a');
fwrite($fh, $_SERVER['REMOTE_ADDR'] . ' ' . date('c') . "\n");
fclose($fh);
?>
Result of three executions:
bart#hal9k:~> cat /tmp/track.txt
127.0.0.1 2009-04-21T09:50:02+02:00
127.0.0.1 2009-04-21T09:50:05+02:00
127.0.0.1 2009-04-21T09:50:08+02:00
Resuming — obligatory die()/exit() is some urban legend that has nothing to do with actual PHP. It has nothing to do with client "respecting" the Location: header. Sending a header does not stop PHP execution, regardless of the client used.
function Redirect($url, $permanent = false)
{
if (headers_sent() === false)
{
header('Location: ' . $url, true, ($permanent === true) ? 301 : 302);
}
exit();
}
Redirect('http://www.google.com/', false);
Don't forget to die() / exit() !
Output JavaScript from PHP using echo, which will do the job.
echo '<script type="text/javascript">
window.location = "http://www.google.com/"
</script>';
You can't really do it in PHP unless you buffer the page output and then later check for redirect condition. That might be too much of a hassle. Remember that headers are the first thing that is sent from the page. Most of the redirect is usually required later in the page. For that you have to buffer all the output of the page and check for redirect condition later. At that point you can either redirect page user header() or simply echo the buffered output.
For more about buffering (advantages)
What is output buffering?
1. Without header
here you will not face any problem
<?php echo "<script>location.href='target-page.php';</script>"; ?>
2. Using header function with exit()
<?php
header('Location: target-page.php');
exit();
?>
but if you use header function then some times you will get "warning
like header already send" to resolve that do not echo or print before sending headers or you can simply use die() or exit() after header function.
3. Using header function with ob_start() and ob_end_flush()
<?php
ob_start(); //this should be first line of your page
header('Location: target-page.php');
ob_end_flush(); //this should be last line of your page
?>
Most of these answers are forgetting a very important step!
header("Location: myOtherPage.php");
die();
Leaving that vital second line out might see you end up on The Daily WTF. The problem is that browsers do not have to respect the headers which your page return, so with headers being ignored, the rest of the page will be executed without a redirect.
Use:
<?php header('Location: another-php-file.php'); exit(); ?>
Or if you've already opened PHP tags, use this:
header('Location: another-php-file.php'); exit();
You can also redirect to external pages, e.g.:
header('Location: https://www.google.com'); exit();
Make sure you include exit() or include die().
You can use session variables to control access to pages and authorize valid users as well:
<?php
session_start();
if (!isset( $_SESSION["valid_user"]))
{
header("location:../");
exit();
}
// Page goes here
?>
http://php.net/manual/en/reserved.variables.session.php.
Recently, I got cyber attacks and decided, I needed to know the users trying to access the Admin Panel or reserved part of the web Application.
So, I added a log access for the IP address and user sessions in a text file, because I don't want to bother my database.
Many of these answers are correct, but they assume you have an absolute URL, which may not be the case. If you want to use a relative URL and generate the rest, then you can do something like this...
$url = 'http://' . $_SERVER['HTTP_HOST']; // Get the server
$url .= rtrim(dirname($_SERVER['PHP_SELF']), '/\\'); // Get the current directory
$url .= '/your-relative/path-goes/here/'; // <-- Your relative path
header('Location: ' . $url, true, 302); // Use either 301 or 302
header( 'Location: http://www.yoursite.com/new_page.html' );
Use the following code:
header("Location: /index.php");
exit(0);
I've already answered this question, but I'll do it again since in the meanwhile I've learnt that there are special cases if you're running in CLI (redirects cannot happen and thus shouldn't exit()) or if your webserver is running PHP as a (F)CGI (it needs a previously set Status header to properly redirect).
function Redirect($url, $code = 302)
{
if (strncmp('cli', PHP_SAPI, 3) !== 0)
{
if (headers_sent() !== true)
{
if (strlen(session_id()) > 0) // If using sessions
{
session_regenerate_id(true); // Avoids session fixation attacks
session_write_close(); // Avoids having sessions lock other requests
}
if (strncmp('cgi', PHP_SAPI, 3) === 0)
{
header(sprintf('Status: %03u', $code), true, $code);
}
header('Location: ' . $url, true, (preg_match('~^30[1237]$~', $code) > 0) ? $code : 302);
}
exit();
}
}
I've also handled the issue of supporting the different HTTP redirection codes (301, 302, 303 and 307), as it was addressed in the comments of my previous answer. Here are the descriptions:
301 - Moved Permanently
302 - Found
303 - See Other
307 - Temporary Redirect (HTTP/1.1)
To redirect the visitor to another page (particularly useful in a conditional loop), simply use the following code:
<?php
header('Location: mypage.php');
?>
In this case, mypage.php is the address of the page to which you would like to redirect the visitors. This address can be absolute and may also include the parameters in this format: mypage.php?param1=val1&m2=val2)
Relative/Absolute Path
When dealing with relative or absolute paths, it is ideal to choose an absolute path from the root of the server (DOCUMENT_ROOT). Use the following format:
<?php
header('Location: /directory/mypage.php');
?>
If ever the target page is on another server, you include the full URL:
<?php
header('Location: http://www.ccm.net/forum/');
?>
HTTP Headers
According to HTTP protocol, HTTP headers must be sent before any type of content. This means that no characters should ever be sent before the header — not even an empty space!
Temporary/Permanent Redirections
By default, the type of redirection presented above is a temporary one. This means that search engines, such as Google Search, will not take the redirection into account when indexing.
If you would like to notify search engines that a page has been permanently moved to another location, use the following code:
<?
header('Status: 301 Moved Permanently', false, 301);
header('Location: new_address');
?>
For example, this page has the following code:
<?
header('Status: 301 Moved Permanently', false, 301);
header('Location: /pc/imprimante.php3');
exit();
?>
When you click on the link above, you are automatically redirected to this page. Moreover, it is a permanent redirection (Status: 301 Moved Permanently). So, if you type the first URL into Google, you will automatically be redirected to the second, redirected link.
Interpretation of PHP Code
The PHP code located after the header() will be interpreted by the server, even if the visitor moves to the address specified in the redirection. In most cases, this means that you need a method to follow the header() function of the exit() function in order to decrease the load of the server:
<?
header('Status: 301 Moved Permanently', false, 301);
header('Location: address');
exit();
?>
header("Location: https://www.example.com/redirect.php");
Direct redirect to this link https://www.example.com/redirect.php
$redirect = "https://www.example.com/redirect.php";
header("Location: $redirect");
First get $redirect value and than redirect to [value] like: https://www.example.com/redirect.php
Use:
<?php
header('Location: redirectpage.php');
header('Location: redirectpage.php');
exit();
echo "<script>location.href='redirectpage.php';</script>";
?>
This is a regular and normal PHP redirect, but you can make a redirecting page with a few seconds wait by the below code:
<?php
header('refresh:5;url=redirectpage.php '); // Note: here 5 means 5 seconds wait for redirect.
?>
Yes, it's possible to use PHP. We will redirect to another page.
Try following code:
<?php
header("Location:./"); // Redirect to index file
header("Location:index.php"); // Redirect to index file
header("Location:example.php");
?>
To redirect in PHP use:
<?php header('Location: URL'); exit; ?>
In the eve of the semantic web, correctness is something to consider. Unfortunately, PHP's "Location"-header still uses the HTTP 302-redirect code, which, strictly, isn't the best one for redirection. The one it should use instead, is the 303 one.
W3C is kind enough to mention that the 303-header is incompatible with "many pre-HTTP/1.1 user agents," which would amount to no browser in current use. So, the 302 is a relic, which shouldn't be used.
...or you could just ignore it, as everyone else...
You can use some JavaScript methods like below
self.location="http://www.example.com/index.php";
window.location.href="http://www.example.com/index.php";
document.location.href = 'http://www.example.com/index.php';
window.location.replace("http://www.example.com/index.php");
Yes, you can use the header() function,
header("Location: http://www.yourwebsite.com/user.php"); /* Redirect browser */
exit();
And also best practice is to call the exit() function right after the header() function to avoid the below code execution.
According to the documentation, header() must be called before any actual output is sent.
Like others here said, sending the location header with:
header( "Location: http://www.mywebsite.com/otherpage.php" );
but you need to do it before you've sent any other output to the browser.
Also, if you're going to use this to block un-authenticated users from certain pages, like you mentioned, keep in mind that some user agents will ignore this and continue on the current page anyway, so you'll need to die() after you send it.
Here are my thoughts:
IMHO, the best way to redirect an incoming request would be by using location headers, which goes
<?php
header("Location: /index.php");
?>
Once this statement is executed, and output sent out, the browser will begin re-directing the user. However, ensure that there hasn't been any output (any echo / var_dump) before sending headers, else it will lead to errors.
Although this is a quick-and-dirty way to achieve what was originally asked, it would eventually turn out to be an SEO disaster, as this kind of redirect is always interpreted as a 301 / 302 redirect, hence search engines will always see your index page as a re-directed page, and not something of a landing page / main page.
Hence it will affect the SEO settings of the website.
The best way to redirect with PHP is the following code...
header("Location: /index.php");
Make sure no code will work after
header("Location: /index.php");
All the code must be executed before the above line.
Suppose,
Case 1:
echo "I am a web developer";
header("Location: /index.php");
It will redirect properly to the location (index.php).
Case 2:
return $something;
header("Location: /index.php");
The above code will not redirect to the location (index.php).
You can try using
header('Location:'.$your_url)
for more info you can refer php official documentation
We can do it in two ways:
When the user comes on https://bskud.com/PINCODE/BIHAR/index.php then redirect to https://bskud.com/PINCODE/BIHAR.php
By the below PHP code
<?php
header("Location: https://bskud.com/PINCODE/BIHAR.php");
exit;
?>
Save the above code in https://bskud.com/PINCODE/BIHAR/index.php
When any condition is true then redirect to another page:
<?php
$myVar = "bskud";
if ($myVar == "bskud") {
?>
<script> window.location.href="https://bskud.com"; </script>
<?php
}
else {
echo "<b>Check the website name again</b>";
}
?>
1. Using header, a built-in PHP function
a) Simple redirect without parameters
<?php
header('Location: index.php');
?>
b) Redirect with GET parameters
<?php
$id = 2;
header("Location: index.php?id=$id&msg=succesfully redirect");
?>
2. Redirect with JavaScript in PHP
a) Simple redirect without parameters
<?php
echo "<script>location.href='index.php';</script>";
?>
b) Redirect with GET parameters
<?php
$id = 2;
echo "<script>location.href='index.php?id=$id&msg=succesfully redirect';</script>";
?>
Using header function for routing
<?php
header('Location: B.php');
exit();
?>
Suppose we want to route from A.php file to B.php than we have to take help of <button> or <a>. Lets see an example
<?php
if(isset($_GET['go_to_page_b'])) {
header('Location: B.php');
exit();
}
?>
<p>I am page A</p>
<button name='go_to_page_b'>Page B</button>
B.php
<p> I am Page B</p>
Use:
<?php
$url = "targetpage"
function redirect$url(){
if (headers_sent()) == false{
echo '<script>window.location.href="' . $url . '";</script>';
}
}
?>
There are multiple ways of doing this, but if you’d prefer php, I’d recommend the use of the header() function.
Basically
$your_target_url = “www.example.com/index.php”;
header(“Location : $your_target_url”);
exit();
If you want to kick it up a notch, it’s best to use it in functions. That way, you are able to add authentications and other checking elemnts in it.
Let’s try with by checking the user’s level.
So, suppose you have stored the user’s authority level in a session called u_auth.
In the function.php
<?php
function authRedirect($get_auth_level,
$required_level,
$if_fail_link = “www.example.com/index.php”){
if ($get_auth_level != $required_level){
header(location : $if_fail_link);
return false;
exit();
}
else{
return true;
}
}
. . .
You’ll then call the function for every page that you want to authenticate.
Like in page.php or any other page.
<?php
// page.php
require “function.php”
// Redirects to www.example.com/index.php if the
// user isn’t authentication level 5
authRedirect($_SESSION[‘u_auth’], 5);
// Redirects to www.example.com/index.php if the
// user isn’t authentication level 4
authRedirect($_SESSION[‘u_auth’], 4);
// Redirects to www.someotherplace.com/somepage.php if the
// user isn’t authentication level 2
authRedirect($_SESSION[‘u_auth’], 2, “www.someotherplace.com/somepage.php”);
. . .
References;
http://php.net/manual/en/function.header.php
I like the kind of redirection after counting seconds
<?php
header("Refresh: 3;url=https://theweek.com.br/índex.php");
I want to make GET, POST & PUT calls to a 3rd party API and display the response on the client side via AJAX. The API calls require a token, but I need to keep that token secret / not in the client-side JS code.
I've seen a few suggestions like this one to have server-side code in the middle that would be queried by the AJAX, and would handle the actual API call. I'm OK working directly with the API from AJAX, but I'm unsure of how to work with a two-step process in order to hide the token from users. My Googling hasn't turned up any pointers on a best-practice method of achieving this.
In my case the server in the middle would be running PHP, so I assume cURL / Guzzle is the straightforward option to make the API calls with the token. The API responses will be JSON.
Can anyone please give me a rough example of how this would be achieved using jQuery.ajax(), to PHP, to the 3rd party API?
Alternatively if there are any quality resources that cover this method in detail, I'd appreciate a link. Equally, if this is a terrible method to use it'd be great to know why.
Edit
Probably worth noting that I want as much flexibility in deploying this as possible; it would be used on multiple sites with unique configurations, so ideally this would be implemented without altering server or hosting account configuration.
It is bit hard without sample code. But As per I understood you can follow this,
AJAX CALL
$.ajax({
type: "POST",
data: {YOU DATA},
url: "yourUrl/anyFile.php",
success: function(data){
// do what you need to
}
});
In PHP
Collect your posted data and handle API, Something like this
$data = $_POST['data'];
// lets say your data something like this
$data =array("line1" => "line1", "line2"=>"line1", "line3" =>"line1");
$api = new Api();
$api->PostMyData($data );
Example API Class
class Api
{
const apiUrl = "https://YourURL/ ";
const targetEndPoint = self::apiUrl. "someOtherPartOFurl/";
const key = "someKey819f053bb08b795343e0b2ebc75fb66f";
const secret ="someSecretef8725578667351c9048162810c65d17";
private $autho="";
public function PostMyData($data){
$createOrder = $this->callApi("POST", self::targetEndPoint, $data, true);
return $createOrder;
}
private function callApi($method, $url, $data=null, $authoRequire = false){
$curl = curl_init();
switch ($method)
{
case "POST":
curl_setopt($curl, CURLOPT_POST, 1);
if ($data)
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($data));
break;
case "PUT":
curl_setopt($curl, CURLOPT_PUT, 1);
break;
default:
if ($data)
$url = sprintf("%s?%s", $url, http_build_query($data));
}
if($authoRequire){
$this->autho = self::key.":".self::secret;
// Optional Authentication:
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($curl, CURLOPT_USERPWD, $this->autho);
}
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($curl);
curl_close($curl);
return $result;
}
}
Because all you want is to add token to http headers, which i am assuming is Authorization a simple way would be to implement a proxy server that makes calls to your api endpoint after adding up those. A sample file for nginx would be
location /apiProxy {
proxy_pass http://www.apiendPoint.com/;
proxy_set_header Authorization <secret token>;
}
This is a much more smarter approach rather than writing a program and gets you off with 4 lines of code. Make sure to change your parameters accordingly and add other parameters as needed by api client you are using. The only difference on javascript side would be to use the location url rather than one provided by service which acts as a proxy.
Edit
The configuration for apache would be
NameVirtualHost *
<VirtualHost *>
<LocationMatch "/apiProxy">
ProxyPass http://www.apiendPoint.com/
ProxyPassReverse http://www.apiendPoint.com/
Header add Authorization "<secret token>"
RequestHeader set Authorization "<secret token>"
</LocationMatch>
</VirtualHost>
From your requirements it looks like "server-side code in the middle" relay(proxy) script is the best option.
PHP example here. N.B. to handle CURL errors it returns a new "object" comprising ['status'] ('OK' or info on CURL failure) and ['msg'] containing the actual response from the API provider. In your JS the original API "object" would now require extracting one level down under 'msg'.
Basic Relays/Proxies can be circumvented
If you use a relay script then someone looking for an API key will probably try elsewhere. However; the pirate could simply replace his call to the API provider using your API key, with a call to your script (and your API key will still be used).
Running of your AJAX/relay script by search engine bots
Google bots (others?) execute AJAX. I assume (relay or not) if your AJAX does not need user input then bot visits will result in API key usage. Bots are "improving". In future (now?) they might emulate user input e.g. if selecting a city from a dropdown results in API request then Google might cycle thro dropdown options.
If of concern you could include a check in your relay script e.g.
$bots = array('bot','slurp','crawl','spider','curl','facebook','fetch','mediapartners','scan','google'); // add your own
foreach ($bots as $bot) :
if (strpos( strtolower($_SERVER['HTTP_USER_AGENT']), $bot) !== FALSE): // its a BOT
// exit error msg or default content for search indexing (in a format expected by your JS)
exit (json_encode(array('status'=>"bot")));
endif;
endforeach;
Relay script and additional code to cater for above issues
Do not overdo pirate protection; relays should be fast and delay unnoticeable by visitors. Possible solutions (no expert and rusty with sessions):
1: PHP sessions solution
Checks whether relay is called by someone who visited your AJAX page in last 15 mins, has provided a valid token, and has the same User Agent and IP Address.
Your Ajax Pages add the following snippets to your PHP & JS:
ini_set('session.cookie_httponly', 1 );
session_start();
// if expired or a "new" visitor
if (empty($_SESSION['expire']) || $_SESSION['expire'] < time()) $_SESSION['token'] = md5('xyz' . uniqid(microtime())); // create token (fast/sufficient)
$_SESSION['expire'] = time() + 900; // make session valid for next 15 mins
$_SESSION['visitid'] = $_SERVER['REMOTE_ADDR'] . $_SERVER['HTTP_USER_AGENT'];
...
// remove API key from your AJAX and add token value to JS e.g.
$.ajax({type:"POST", url:"/path/relay.php",data: yourQueryParams + "&token=<?php echo $_SESSION['token']; ?>", success: function(data){doResult(data);} });
The relay/proxy Script (session version):
Use an existing example relay script and before the CURL block add:
session_start(); // CHECK REQUEST IS FROM YOU AJAX PAGE
if (empty($_SESSION['token']) || $_SESSION['token'] != $_POST['token'] || $_SESSION['expire'] < time()
|| $_SESSION['visitid'] != $_SERVER['REMOTE_ADDR'] . $_SERVER['HTTP_USER_AGENT'] ) {
session_destroy(); // (invalid) clear session's variables, you could also kill session/cookie
exit (json_encode(array('status'=>'blocked'))); // exit an object that can be understood by your JS
}
Assumes standard session ini settings. Cookies required and page/relay on same domain (workround possible). Sessions might impact performance. If site already uses Sessions, code will need to take this into account.
2: Sessionless/Cookieless option
Uses a token associated with specific IP Address and User Agent, valid for a maximum of 2 hours.
Functions used by both page and relay e.g. "site-functions.inc":
<?php
function getToken($thisHour = TRUE) { // provides token to insert on page or to compare with the one from page
if ($thisHour) $theHour = date("jH"); else $theHour = date("jH", time() -3600); // token for current or previous hour
return hash('sha256', 'salt' . $_SERVER['REMOTE_ADDR'] . $_SERVER['HTTP_USER_AGENT'] . $theHour);
}
function isValidToken($token) { // is token valid for current or previous hour
return (getToken() == $token || getToken(FALSE) == $token);
}
?>
Relay Script Use an existing example and before the CURL block add:
// assign post variable 'token' to $token
include '/pathTo/' . 'site-functions.inc';
$result = array('status'=>'timed out (try reloading) or invalid request');
if ( ! isValidToken($token)) exit(json_encode(array('msg'=>'invalid/timeout'))); // in format for handling by your JS
Pages needing the API (or your javascript include file):
<?php include '/pathTo/' . 'site-functions.inc'; ?>
...
// example Javascript with PHP insertion of token value
var dataString = existingDataString + "&token=" + "<?php echo getToken(); ?>"
jQuery.ajax({type:"POST", url:"/whatever/myrelay.php",data: dataString, success: function(data){myOutput(data);} });
Note: User Agent is spoofable. IP (REMOTE_ADDR) "cannot" be faked but setup on a minority of sites can cause issues e.g. if you are behind NGINX you may find REMOTE_ADDR always contains the NGINX server IP.
If you are using a typical 3rd party API that will provide NON sensitive information until you reach the usage cap for your API Key then (I think) above solutions should be sufficient.
As people pointed out, you want a proxy method on your server to hide the API-key.
To avoid misuse of your method on the server, protect the call with an one time token (like you usually use for forms) - generated from your server (not in javascript..).
I am not a fan of the coded pasted above which checks for known http-user agents... or site tokens ... this is not secure.
If you use cUrl that you must to protect is your server. The way I personally use is the Google reCaptcha that is sure made for arranging problems like yours. Very well explained the integration in client and server sides step by step here.
https://webdesign.tutsplus.com/tutorials/how-to-integrate-no-captcha-recaptcha-in-your-website--cms-23024
With this way you don't need to change anything in your virtualhost files and any apache configurations.
I would use the solutiuon #MMRahman published, if you want to add a security layer between your backend and your frontend what you could do it is when the user make login generate a unique ID, store it in the server session and in a cookie or local/session store of the browser, this way when you call your backend with ajax you can get the value from the place where you stored in in the browser, and check if the values are the same, if yes you call the external api and return the values if not just ignore the request.
So summaring:
User login -> generate unique ID -> store it in server session and browser session -> make call from ajax passing as parameter the value from browser session-> check if it matchs with server session stored value -> if yes call external api using the token stored in your backend / db / file / whatever you want
I have been developing a social network. I have noticed some security issues where the user can change the variables in javascript/jquery to other user_id's, text content, and other information that has been loaded into the scripts. And this is all done via the inspect tool or other software that can edit the languages. They can even rewrite the functions.
I load data onto the page via php and sql after sending the url_id to a php function.
I have javascript and jquery scripts that in return use this data to perform ajax, post, and get requests and to perform functions.
How can I stop the user from changing these variables before they are sent off to the server? For example when a user makes a post they can change the id to make it someone else's post, or when they click delete an image they can delete someone else's and it gets more complicated. This is a huge concern.
These scripts are included in the php pages or in php scripts that are loaded via ajax.
How can I stop this? Can you give me an easy explanation? I have been searching for months on how to stop this. I still don't understand how to stop the user from doing so. If there is another way could to do this? Can you provide me with true 100% examples? What are the other options I have?
Here are some snippets of my code
<? if (login_check($mysqli) == true) : ?>
<script>
$.post("auto/online.php?q=<? echo $id ?>");
function o() {
setTimeout(function() {
$.post("auto/online.php?q=<? echo $id ?>");
o();
}, 6e4);
}
</script>
<? endif; ?>
<?php echo '<div class="post-btn" onclick="ajaxPost(postenter.value,\''.$name.'\',\''.$id.'\');" title="Post">Post</div>'; ?>
function ajaxPost(content,name,id) {
var ip = '<?php echo $ip ?>';
content = content.replace(/<br\s*\/?>/mg,"\n");
var postArray = [content, id, ip];
postArray = JSON.stringify(postArray);
alert(postArray);
if (content.length == 0) {
alert('Oops it looks like your post is empty.');
} else {
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
document.getElementById("postenter").innerHTML = "";
var html = xmlhttp.responseText;
alert(html);
$(html).hide().insertAfter("#wrapper").fadeIn(500);
document.getElementById("postenter").value = "";
}
}
xmlhttp.open("POST", "auto/post.php", true);
xmlhttp.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
xmlhttp.send('data=' + postArray);
}
}
<? if ($id == $user) : ?>
<div class="modalSetPro" onclick="setProImage(<? echo $picID; ?>,<? echo $uid; ?>)">Set Profile</div>
<div class="modalSetBac" onclick="setProCover(<? echo $picID; ?>,<? echo $uid; ?>)">Set Background</div>
<div class="modalDelImg" onclick="delItemPre(<? echo $picID; ?>, 1, <? echo $uid; ?>)">Delete</div>
<? endif; ?>
function delItemPre(itemID, type, user) {
var modArr = [itemID, type, user];
modArr = JSON.stringify(modArr);
$("#LoadMe").load('modals/del_modal.php?p=' + modArr);
}
How can I stop the user from changing these variables before they are sent off to the server? For example when a user makes a post they can change the id to make it someone else's post, or when they click delete an image they can delete someone else's and it gets more complicated. This is a huge concern.
You can't.
Your server side code should evaluate the user's privileges and decide whether or not they can do the action. JavaScript validation is more for the user experience - guiding and preventing mistakes.
You are not able to prevent this, which is why server-side validation is required.
Here is a stackoverflow discussing it: Why do we need both client side and server side validation?
There is some good information here:
http://www.w3schools.com/php/php_form_validation.asp
Basically, you want to put your validations in the PHP page that you are posting your ajax to.
Store and check all insecure data on server side, not client. This way user can't change it.
First of all when you are working on client side you have no control how user interact with you jquery or javascript code. So thumb rule is that never expose sensitive data in html or java script.
More over If you are curious about security you have not required to load User id in hidden field or any other client side code(html). In you case like when user is replying to any post you have to crosscheck at server side whether current logged in user is authorized to perform this task or not. also cross check whether this post is relate to current logged in user.
I have no knowledge about php but in asp.net we can create a session at server side and when user post data get the User id from session not from html content posted by user.
I am creating a website that has users log in and select a pdf document that they want to download. When they open up the document to view and possibly download, I want data to be logged into a database at the same time.
The code to send the data to the database works (Except for: Undefined index: learningMaterial). But when I want to have the pdf document open and at the same time log the user and other data, all that happens is the document opens up.
Any advice would be appreciated, even for overall better methods of going about what I'm trying to achieve here. Still inexperienced with PHP.
See code below.
HTML
<form name="myform" method='post' action="../includes/writeStats.php">
<input type='hidden' name='learningMaterial' id='learningMaterial' value='learningMaterial'>
<a href='../documents/test.pdf' id='mylink' class='courses' name='Driver Training'> Driver Training </a>
</form>
JS - In header
<script type="text/javascript">
function submitform(){
document.myform.submit(); }
var form = document.getElementById("myform");
document.getElementById("mylink").addEventListener("click", function () {
submitform();
});
</script>
PHP
<?php
$con=mysqli_connect("localhost","root","password","qmptest");
// Check connection
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
//Get latest log nr
$result = mysqli_query($con,"SELECT * FROM logbook ORDER BY log DESC LIMIT 1");
while($row = mysqli_fetch_array($result)) {
$log = $row['log'] + 1;
//If statement to check if log is 0(first entry) to go here
}
$date = date("Y/m/d");
session_start(); // Start a new session
$person = $_SESSION['currentUser'];
//Not sure if this is correct along with my HTML input
$material = mysqli_real_escape_string($con, $_POST['learningMaterial']);
//Insert into database
$sql="INSERT INTO logbook (log, date, person, learningMaterial)
VALUES ('$log', '$date', '$person', '$material')";
if (!mysqli_query($con,$sql)) {
die('Error: ' . mysqli_error($con));
}
mysqli_close($con);
?>
Your way, clicking the link will override the form being submitted. This leads to the file opening and the form never going through.
Instead, you could try either opening the file in a new window by adding target="_blank" to the tag, or send the files URL through to the PHP, executing the database code then adding to the end:
header("Location: http://yourdomain.com/yourfile.pdf");
Your file is just a normal file being returned by your web server:
<a href='../documents/test.pdf' ...
So while you may be able to suggest to users or browsers that they should invoke some code before downloading this file, you can't actually require it. Any user can just request the file directly. And since PDF files don't execute PHP code (thankfully), your server-side PHP code has no way of knowing that the file has been requested.
What you can do is obscure the file itself behind a PHP request. You can create something like a download.php page which accepts the name of a file (test.pdf) and returns that file.
Be very careful when doing this. Don't just allow users to request any file and blindly return whatever they request. A user can request something like "../../../../../../../../../../etc/passwd" and if your code just builds a path and returns the file then you've just given users a sensitive file. It's best practice to keep a finite known list of identified files (perhaps in a database table) and let users request by the identifier rather than by the file path itself. That way the actual path is only ever known server-side in data that you control.
The main point here, however, is that by using such a page you inject some PHP code in between the user and the file. In order to get the file, the user needs to make a request to a PHP page. On that page you can record the act of the user having requested the file. (As well as perform authorization checks to validate that the user is allowed to view the file, etc.)
Never assume client-side code is going to do what you expect it to do. If you want to ensure something happens for anything approaching security or auditing purposes, it needs to happen in server-side code.