i try to automate a download from a HTML-datasheet to generate a customized reporting. The following i was doing with CURL:
// init cURL HTTP Client
$header = array();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_COOKIEFILE, '/.cookies');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/.cookies');
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 600);
curl_setopt($ch, CURLOPT_URL, 'https:// ... /signin.html');
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, "username=".$login."&password=".$pass);
$response = curl_exec($ch);
The login works fine and i can get many pages without any problems. Now i try to get the datasheet by the following:
curl_setopt($ch, CURLOPT_URL, 'https:// ... /data.html');
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_POSTFIELDS, '');
$response = curl_exec($ch);
But now i get the following answer:
<html>
<head>
<script language='javascript'>function autoNavigate() {window.location="/data.html";}</script>
</head>
<body onload='autoNavigate()'></body>
</html>
The javaScript call refresh the same page as i loaded before. In a browser it works fine, but if i load the same page again with "curl_exec($ch)" i've got a 302-error?
Is there a possibilty the refresh the page with curl without a full reload? Or any other idea to get the content of the page?
Thanks
try:
$postfields = '';
curl_setopt($ch, CURLOPT_URL, 'https:// ... /data.html');
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
$response = curl_exec($ch);
It creates problem when you set false the CURLOPT_POSTFIELDS value but earlier you set it as True bacause it holds the previous details in Cookie.
I hope this will helpful for you.
Did you check the link of data.html?
If the data.html in window.location="data.html"; is the same location of data.html in curl_setopt($ch, CURLOPT_URL, 'https:// ... /data.html'); try to double curl_exec($ch) so may be it needs to access two times. Or if it different, just simple change your link.
Related
I want a make an html page that can auto login to a site and then navigate to another page of that site and get some elements and display them on my html page using java script the site is based on MOODLE-API is there any way to do this.
I've tried using this curl an php script but it didnt helped since moodle api uses cookies to login that go away as soon as the user closes the browser.
the website is https://edumate.raoiit.com/login/index.php
<?php
//Upload a blank cookie.txt to the same directory as this file with a CHMOD/Permission to 777
function login($url,$data){
$fp = fopen("cookie.txt", "w");
fclose($fp);
$login = curl_init();
curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($login, CURLOPT_TIMEOUT, 40000);
curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($login, CURLOPT_URL, $url);
curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($login, CURLOPT_POST, TRUE);
curl_setopt($login, CURLOPT_POSTFIELDS, $data);
ob_start();
return curl_exec ($login);
ob_end_clean();
curl_close ($login);
unset($login);
}
function grab_page($site){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_URL, $site);
ob_start();
return curl_exec ($ch);
ob_end_clean();
curl_close ($ch);
}
function post_data($site,$data){
$datapost = curl_init();
$headers = array("Expect:");
curl_setopt($datapost, CURLOPT_URL, $site);
curl_setopt($datapost, CURLOPT_TIMEOUT, 40000);
curl_setopt($datapost, CURLOPT_HEADER, TRUE);
curl_setopt($datapost, CURLOPT_HTTPHEADER, $headers);
curl_setopt($datapost, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($datapost, CURLOPT_POST, TRUE);
curl_setopt($datapost, CURLOPT_POSTFIELDS, $data);
curl_setopt($datapost, CURLOPT_COOKIEFILE, "cookie.txt");
ob_start();
return curl_exec ($datapost);
ob_end_clean();
curl_close ($datapost);
unset($datapost);
}
?>
Well, your code cannot work, since CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR both require an absolute path to the file. the easiest way is to use realpath('cookie.txt').
And if the website really has cookies which are removed on exit, just create a global $ch = curl_init(); object, and use it in the methods by writing global $ch; at the very beginning of a method, an not calling curl_init() or curl_close() in any method.
I am sending multiple file of form to server but it show error like this:-
$post_params['files'][] =
'#' . $this->fields['files']['tmp_name'][$index]
. ';filename=' . $this->fields['files']['name'][$index]
. ';type=' . $this->fields['files']['type'][$index];
}
//print_r($post_params); die;
// We send XML via CURL using POST with a http header of text/xml.
$url = "http://54.221.28.145:8080/kv_image_rest_service/rest/files/upload";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: multipart/form-data'));
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_params);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$ch_result = curl_exec($ch);
curl_close($ch);
// Print CURL result.
echo $ch_result;
I'm trying to show a Javascript function, which contains a alert/notification after my PHP Curl post went through.
Any idea how to get this working?
My Javascript:
<script>
$('body').pgNotification({
style:'bar',
message: 'added',
position:'top',
type:'success',
timeout:'6000'
}).show();
</script>
My PHP:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'www.site.com');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'parameters to pass');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookiefile';
$response = curl_exec($ch);
curl_close($ch);
?>
Now I want to show the Javascript Notification after the PHP curl post is done.
Sadly I can't simply use a echo 'HTMLCODE'; to show a notification. I need the timeout function in it.
I appreciate every kind of help.
function test() {
$('body').pgNotification({
style:'bar',
message: 'added',
position:'top',
type:'success',
timeout:'6000'
}).show();
}
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'www.site.com');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'parameters to pass');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookiefile';
$response = curl_exec($ch);
curl_close($ch);
echo '<script>test();</script>';
?>
Expecting that you are not calling CURL through AJAX. Just echo your javascript function if CURL response is what you want it to be.
Javascript
function showNotification(){
$('body').pgNotification({
style:'bar',
message: 'added',
position:'top',
type:'success',
timeout:'6000'
}).show();
}
PHP
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'www.site.com');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'parameters to pass');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookiefile';
$response = curl_exec($ch);
curl_close($ch);
if($response == 'whatever it should return'){
echo "
<script>
// Run this if page is loaded
// Probably using jQuery?
$(document).ready(function(){
showNotification();
});
</script>";
}
?>
I know that curl does not execute javascript, it only grabs static html, so this is why a simple curl will not work for me.
I do not know much about php, I'm new to this, but what I understand so for is that if I did not have to first login to grab the content I can simple use file_get_contents witch will first execute the dynamic content and then grab the html content, witch in return give me what I need, but I first have to login and then get the page.
I tried to login using curl
$user = "myuser";
$pass = "mypassword";
//create cookie file
$random = rand(0,9999999);
$cookie = $random."cookie.txt";
$fp = fopen("$cookie","w") or die("<BR><B>Unable to open cookie file $cookie_file_path for write!<BR>");
fclose($fp);
//do login using curl
$LOGINURL = "https://controlpanel.example.com/index.html";
$agent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0";
$v2 = array( 'userName'=>$user, 'password'=>$pass);
$reffer = "https://www.google.com";
//this first call is to set the cookie
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
ob_start(); // Prevent output
curl_exec ($ch);
ob_end_clean(); // End preventing output
curl_close ($ch);
unset($ch);
//now that the cookie is set, do login
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,$v2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_REFERER, $reffer);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$result = curl_exec($ch);
//now we are logged-in
//now grab the page you need
$profileurl = 'https://controlpanel.example.com/information.html';
curl_setopt($ch, CURLOPT_URL, $profileurl);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$result = curl_exec ($ch);
But this will only get the static html, not the dynamic content too.
Let me explain better.
The code I get, at this point using above curl method, in $result is:
.....
<div id="DisplayAccountInfo"><span class="loading">Loading info</span></div>
.....
If I do this manually using firefox and inspect element with firebug the source is:
.....
<div id="DisplayAccountInfo">
<div class="formModule" id="formContainer">
......
<legend>Your code for this hour is 8T5D9LO</legend>
.....
</div>
</div>
.....
What I notice in firebug console is:
GET https://controlpanel.example.com/async/information.html
200 OK
669ms
jquery-....min.js (line 19)
What I, as a noob, understand from this is that the content is dinamicly loaded using jquery, and curl does not know how to do that.
I tried to put instead of
$profileurl = 'https://controlpanel.example.com/information.html';
curl_setopt($ch, CURLOPT_URL, $profileurl);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$result = curl_exec ($ch);
//replaced the above with this
$result = file_get_contents($profileurl);
but I get the html from login page because I think it does not recognize anymore that I'm logged in.
So how can I solve this? Can you please help me?
I think I got what you're doing.
The key point here is, most website handle login with cookie. In https://controlpanel.example.com/information.html, if the website set a cookie after you login in your browser, then the good news is you can solve this problem.
The problem in your code is, PHP won't set cookie for you.
You need 2 steps:
Step 1. You need to obtain the cookie when your php curl the login
Here's how you get cookie header returned from the login page.
$ch = curl_init('https://controlpanel.example.com/index.html');
....
$result = curl_exec($ch);
preg_match('/^Set-Cookie:\s*([^;]*)/mi', $result, $m);
parse_str($m[1], $cookies);
echo $cookies;//See if you've successfully obtained the return cookie
Step 2. You access https://controlpanel.example.com/information.html with the cookie you obtained in step 1. (like you've already did in your own code)
haha, so easy it did not cross my mind.
For me it is simple, I did not have to call
https://controlpanel.example.com/information.html
but
https://controlpanel.example.com/async/information.html
to get the div I wanted :)
Lucky for me I noticed the get function in firebug :)
So the cod now is :
$user = "myuser";
$pass = "mypassword";
//create cookie file
$random = rand(0,9999999);
$cookie = $random."cookie.txt";
$fp = fopen("$cookie","w") or die("<BR><B>Unable to open cookie file $cookie for write!<BR>");
fclose($fp);
//do login using curl
$LOGINURL = "https://controlpanel.example.com/index.html";
$agent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0";
$v2 = array( 'userName'=>$user, 'password'=>$pass);
$reffer = "https://www.google.com";
//this first call is to set the cookie
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
ob_start(); // Prevent output
curl_exec ($ch);
ob_end_clean(); // End preventing output
curl_close ($ch);
unset($ch);
//now that the cookie is set, do login
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,$v2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_REFERER, $reffer);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$result = curl_exec($ch);
//now we are logged-in
//now grab the page you need
$profileurl = 'https://controlpanel.example.com/async/information.html';
curl_setopt($ch, CURLOPT_URL, $profileurl);
curl_setopt($ch, CURLOPT_POST, 0);
$result = curl_exec ($ch);
I am trying to request the Appcelerator Cloud for creating the user with photo, the user is getting created without photo , but when i attach the photo with request i get this error
This Code Gives me error "{ "meta": { "status": "fail", "code": 400, "message": "Failed to upload photo: Failed to indentify photo file" } }"
Please Help me with this , i m new to PHP
<?php
$agent = "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7";
$data = json_encode(array(
'securitycode' => $_POST['code'],
'designation' => $_POST['desi'],
'comapanyname' => $_POST['cname'],
'companylogoid' => $_POST['clogoid'],
'companyurl' => $_POST['curl'],
'location' => $_POST['location'],
'emailprivateflag' => $_POST['emailp'],
'linkedinurl' => $_POST['linkedin'],
'twitterhandle' => $_POST['twitter'],
'isspeaker' => $_POST['isspeaker'],
'allowaccess' => $_POST['access']
));
//$photo = array('photo' => '#' . $_FILES['file']['name'][0]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://api.cloud.appcelerator.com/v1/users/login.json?key=LJOVi9A95Y3r6TqlFmxS314v6ox4xaPf");
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt ($ch, CURLOPT_HTTPHEADER, Array("Content-Type: application/x-www-form-urlencoded","Accept: */*"));
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie1.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie1.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,"login=test#test.com&password=test");
// $html=curl_exec($ch);
$ch1 = curl_init();
curl_setopt($ch1, CURLOPT_URL,"https://api.cloud.appcelerator.com/v1/users/create.json?key=LJOVi9A95Y3r6TqlFmxS314v6ox4xaPf");
curl_setopt($ch1, CURLOPT_USERAGENT, $agent);
curl_setopt ($ch1, CURLOPT_HTTPHEADER, Array('Content-Type: multipart/form-data',"Accept: */*"));
// curl_setopt($ch1, CURLOPT_COOKIEFILE,'cookie2.txt');
// curl_setopt($ch1, CURLOPT_COOKIEJAR, 'cookie2.txt');
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch1, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch1, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch1, CURLOPT_POST, true);
curl_setopt($ch1, CURLOPT_POSTFIELDS, 'email='.$_POST['email'].'&first_name='.$_POST['fname'].'&last_name='.$_POST['lname'].'&password=ciosummit&password_confirmation=ciosummit&photo=#'. $_FILES['file']['tmp_name'][0].'&custom_fields='.$data);
$html=curl_exec($ch1);
header('Location:index.php?status='. rawurlencode($html));
//echo $html;
?>
You need to send the data in an array just like this:
$post = array('photo' => '#YOURIMAGENAME',
'email'=>$_POST['email'],
'first_name'=>$_POST['fname'],
'last_name'=>$_POST['lame'],
'password'=>'ciosummit',
'password_confirmation'=>'ciosummit',
'custom_fields'=>$data
);
And then send the curl values like:
curl_setopt($ch1, CURLOPT_POSTFIELDS,$post);