I have a xml file containing information i want to store or update in database. my server redirects me to previous page if in 30 seconds script doesn't finish executing (changed max execution time, didn't help)
I want to split the file into multiple arrays and send them over ajax to be processed in more instances thus trying to shorten the execution time.
the file contains 38k rows and in 30 seconds i can add 6700 new objects in db or update 3800 existing ones.
so is there a way to do this? i'm very new to ajax so i don't even know where to start looking for a solution.
EDIT1:
<?php
$time = microtime(TRUE);
$xml = simplexml_load_string(file_get_contents($feed));
$json = json_encode($xml);
$array = json_decode($json,TRUE);
$array= $array['Row'];
set_time_limit(0);
ini_set('memory_limit','4000M');
//echo ini_get('max_execution_time');
//die();
$new = 0;
$existent = 0;
foreach($array as $produs)
{
$prod = Products::model()->findbyattributes(array('cod'=>$produs['ProductId']));
if(!$prod)
{
$prod = new Products;
$prod->cod = $produs['ProductId'];
$prod->price = $produs['PriceSRP'];
$prod->name = $produs['Name'];
$prod->furnizor= 'ABCData';
$prod->brand = $produs['HierarchyNameLevel1'];
//$prod->stock = $produs['Available'];
if($produs['Available'] == "+")
$prod->stock = 'Da';
else
{$prod->stock = 'Nu';}
$prod->category = $prod->getCategory($produs['MinorGroup'], 'ABC');
if(!$prod->category)
continue;
if(!$prod->save())
{
echo '<pre>';
var_dump($prod->geterrors());
echo '</pre>';
}
else{$new++;}
}
elseif($prod)
{
$prod->brand = $produs['HierarchyNameLevel1'];
$prod->price = $produs['PriceSRP'];
$prod->last_edit = date('Y-m-d H:i:s');
if($produs['Available'] == "+")
$prod->stock = 'Da';
else
{$prod->stock = 'Nu';}
if(!$prod->save())
{
echo '<pre>';
var_dump($prod->geterrors());
echo '</pre>';
}
else {$existent++;}
}
}
echo 'adaugat '.$new.' si updatat '.$existent.' produse in ';
print (microtime(TRUE)-$time). ' secunde!';
?>
it appears i may have been unclear in my initial post.
so this is my existing code. the $feed file has 38k items in it that i need to process and add or update existing db entries.
if i run the full 38k file after 30 sec the browser performs a history.back() called by the apache server. i would have liked to process the file from crond and process for example 1 entry every second but that is imposible since i have no access to crond on that specific server. i've tried to split up the file manually and it works perfectly fine for ~6700 new entries or 3500 - 4000 existing ones (since it has to find them, load them, update them and save )
so my initial problem, and what i was asking if it is posible to do it over ajax so the server won't stop the script from executing if its longer them 30 seconds(as in i don't even know if the server will interpret the ajax as a new request and existing script won't wait for it to respond).
I would save the xml file in a temp folder, then do an ajax get that runs the file from a specific off-set for (e.g.) 100 records:
function processScript(offset) {
$.ajax({
type: "POST",
url: "some.php",
data: { offset:offset },
dataType:'json',
success: function(data) {
var o = parseJSON(data);
if(o.offset > 0) {
processScript(o.offset);
}
}
})
}
processScript(0);
In some.php you would want to return a json object with a property 'offset' containing the next block of elements you want to process. When the xml file is complete, set offset to 0.
The above code is enough to get you started. You will also want to do some sort of error cheching in the success function, as well as give a progress notification to the user (e.g. "3,600 of 38,000 lines process"?).
array = $('.def-mask :checkbox:checked').serialize();
$.ajax({
url: 'ajax/battle.php',
type: 'post',
data: { playerReady: 1, attack: attack, defence: array },
success: function(data) {
alert(data);
}
});
More info
Related
I'm sorry but I'm having a hard time setting up something simple but that doesn't work for me. I'm trying to put a code that counts the number of clicks on a phone number ( the last 4 hidden digits that appear ) then record this data in my DB. I set up the JAVASCRIPT at the bottom of my PHP page where I will listen if there is a click ( Addeventlistener.... ) on the phone number.
I understood that we can not execute PHP code in a JS script, OK, so I execute an Ajax code to send to a PHP file the values to insert in a new entry to my DB. Except that during the execution the functions that open a connection to the DB are not recognized while in the same way I use others functions in the same PHP file that selects and returns me data from the DB.
Is the difference that they are two different types of request SELECT and INSERT or it is because I send the data through Ajax that the PHP files that load the function of DB connection are not loaded?
AJAX Script
<script>
var phoneclick = document.querySelector(".data-phone");
var baseUrl = "public_html/oc-content/themes/delta/"
phoneclick.addEventListener("click", function() {
var item_id = 8111;
var ajaxPhoneClick = 1;
$.ajax({
url: '<?php echo osc_current_web_theme_url('model/sql_projet.php'); ?>',
type: "GET",
data: {
id: '1111'
},
success: function(data) {
console.log(data);
}
});
});
</script>
PHP FIle
$itemId = $_GET['id'];
$conn = DBConnectionClass::newInstance();
$data = $conn->getDb();
$comm = new DBCommandClass($data);
$db_prefix = DB_TABLE_PREFIX;
$query = "INSERT INTO {$db_prefix}t_item_stats (item_id,phone_clicks) VALUES ($itemId,1) ";
$result = $comm->query($query);
The error i get is this Fatal error: Uncaught Error: Class 'DBConnectionClass' not found in /Applications/XAMPP/
I want to know the reason why this error is throwing and what should i do to bypass this
I have some results from a fetch.php using json and I successfully brought all results to my bootstrap modal HTML screen.
When the Modal is being shown, I would like to run a MYSQL query using a value coming from the same json I used for the modal, however I can't put this value into a PHP variable to run the SQL query.
How can I get this?
I am trying to bring the same value I input into the HTML textbox (modal), but it is not working. I also tried to use the value from json '$('#PCR').val(data.PCRNo);)', but nothing happen.
This is the script to collect information from database using fetch.php file:
<script>
$(document).ready(function(){
$('#table').on('click', '.fetch_data', function(){
var pcr_number = $(this).attr('id');
$.ajax({
url:'fetch.php',
method:'post',
data:{pcr_number:pcr_number},
dataType:"json",
success:function(data){
$('#PCR').val(data.PCRNo);
$('#PCC').val(data.PCC);
$('#PCR_Creation').val(data.Creation_Date);
$('#PCR_Status').val(data.Stage);
$('#Required_Completion').val(data.Required_Completion);
}
});
});
});
</script>
This is the PHP code
<?php
//trying to get the value I have included on #PCR (textbox) which has ID='PCR' and name ='PCR' **
$PCR= $_POST['PCR'];
//running now the code to check if the database has the value and return the desired response to be shown **
$sql1 = mysqli_query($dbConnected,"SELECT * FROM change_management.tPCN");
while ($row1 = mysqli_fetch_array($sql1)) {
if ($row1['PCRNo']==$PCR){
echo $row1['PCNNo'];
echo "<br/>";
}else{
}
}
?>
I would like include value from this val(data.PCRNo) json return into the $PCR variable, so the MYSQL query is going to work
There are a number of quite basic logical issues with your code which are preventing it from working.
1) data: { pcr_number: pcr_number}- the name pcr_number doesn't match the value PCR which the server is searching for using $_POST['PCR'];. The names must match up. When making an AJAX request, the name you gave to the form field in the HTML does not matter (unless you use .serialize()) because you are specifying new names in the data parameter.
2) Your SQL query doesn't make sense. You seem to be wanting to read a single row relating to a PCR number, yet your query makes no usage of the input PCR value to try and restrict the results to that row. You need to use a SQL WHERE clause to get it to select only the row with that ID, otherwise you'll fetch all the rows and won't know which one is correct. (Fetching them all and then using an if in a PHP loop to check the correct one is very inefficient.) I wrote you a version which uses the WHERE clause properly, and passes the PCR value to the query securely using prepared statements and parameters (to project against SQL injection attacks).
3) Your output from the PHP also makes no sense. You've told jQuery (via dataType: "json" to expect a JSON response, and then your code inside the "success" function is based on the assumption you'll receive a single object containing all the fields from the table. But echo $row1['PCNNo']; echo "<br/>"; only outputs one field, and it outputs it with HTML next to it. This is not JSON, it's not even close to being JSON. You need to output the whole row, and then use json_encode() function to turn the object into a JSON string which jQuery can parse when it receives it.
Here's a version of the code containing all the above changes:
JavaScript:
$(document).ready(function(){
$('#table').on('click', '.fetch_data', function(){
$.ajax({
url: 'fetch.php',
method: 'post',
data: { pcr: $(this).attr('id'); },
dataType: "json",
success: function(data){
$('#PCR').val(data.PCRNo);
$('#PCC').val(data.PCC);
$('#PCR_Creation').val(data.Creation_Date);
$('#PCR_Status').val(data.Stage);
$('#Required_Completion').val(data.Required_Completion);
}
});
});
});
PHP:
<?php
$PCR = $_POST['pcr'];
$stmt = $dbConnected->prepare("SELECT * FROM change_management.tPCN WHERE PCRNo = ?");
$stmt->bind_param('s', $PCR);
$stmt->execute();
$result = $stmt->get_result();
//an "if" here will cause a single row to be read
if ($row = $result->fetch_assoc()) {
$output = $row;
}
else
{
$output = new StdClass();
}
$stmt->free_result();
$stmt->close();
//output the result
echo json_encode($output);
?>
N.B. I would potentially suggest studying some tutorials on this kind of subject, since this is a fairly standard use case for AJAX/JSON, and you should be able to find samples which would improve your understanding of all the different parts.
P.S. Currently the PHP code above will return an empty object if there is no matching row in the database. However, this is probably an error condition (and will cause your JavaScript code to crash due to trying to read nonexistent properties), so you should consider how you want to handle such an error and what response to return (e.g. 400, or 404, and a suitable message).
You need to first return json from php by using json_encode.
Inside this loop
while ($row1 = mysqli_fetch_array($sql1)) {
$data = array('PCRNo' => 'itsvalue', 'PCC' => 'itsvalue', 'Creation_Date' => 'itsvalue')
}
print json_encode($data)
store all the data in an associative array and then convert it into json using json_encode and return the json.
Use json data in you ajax file
$.ajax({
url:'fetch.php',
method:'post',
data:{pcr_number:pcr_number},
dataType:"json",
success:function(data){
var data = JSON.parse(data);
$('#PCR').val(data.PCRNo);
$('#PCC').val(data.PCC);
$('#PCR_Creation').val(data.Creation_Date);
$('#PCR_Status').val(data.Stage);
$('#Required_Completion').val(data.Required_Completion);
}
});
Below is the changed script to store different values in $PCR variable
<script>
$(document).ready(function(){
var i = 1;
$('#table').on('click', '.fetch_data', function(){
if(i == 1) {
var pcr_number = $(this).attr('id');
} else {
var pcr_number = $('#PCR').val();
}
$.ajax({
url:'fetch.php',
method:'post',
data:{pcr_number:pcr_number},
dataType:"json",
success:function(data){
$('#PCR').val(data.PCRNo);
$('#PCC').val(data.PCC);
$('#PCR_Creation').val(data.Creation_Date);
$('#PCR_Status').val(data.Stage);
$('#Required_Completion').val(data.Required_Completion);
i++;
}
});
});
});
</script>
I am running a longpolling script to grab data from the database. It was working fine until moving my script to an MVC.
I have viewed the chrome developer tool and it's showing nothing in there, but the page just carries on loading, and when I go to refresh it won't load, I have to shut down my xampp server or close my browser... Here's my script:
class SystemController extends Controller
{
public function lastbid()
{
set_time_limit(0);
// main loop
while (true) {
//get the product info
$getbidresult = ProductModel::bidprice(Request::get('item'));
// if ajax request has send a timestamp, then $last_ajax_call = timestamp, else $last_ajax_call = null
$last_ajax_call = Request::get('timestamp');
// get timestamp of when file has been changed the last time
$lastbid = isset($getbidresult->timestamp) ? $getbidresult->timestamp : 0;
// if no timestamp delivered via ajax or data.txt has been changed SINCE last ajax timestamp
if ($last_ajax_call == null || $lastbid > $last_ajax_call) {
// put last bid info into an array
$result = array(
'bidamount' => isset($getbidresult->amount) ? System::escape($getbidresult->amount): 0,
'timestamp' => System::escape($lastbid)
);
// encode to JSON, render the result (for AJAX)
$json = json_encode($result);
echo $json;
// leave this loop step
break;
} else {
// wait for 1 sec (not very sexy as this blocks the PHP/Apache process, but that's how it goes)
sleep(10);
continue;
}
}
}
}
This is how I am grabbing the data with JS.
function getContent(timestamp)
{
var queryString = {
'timestamp': timestamp
};
$.ajax(
{
type: 'GET',
url: '<?php echo Config::get('URL'); ?>system/lastbid?item=<?php echo System::escape($recentitem->id); ?>',
data: queryString,
success: function(data)
{
var obj = jQuery.parseJSON(data);
$('#bidprice-<?php echo System::escape($recentitem->id); ?>').html(obj.bidamount);
getContent(obj.timestamp);
}
});
}
$(function()
{
getContent();
});
$(document).ready(function() {
});
I've looked in apache logs with no avail unless I am looking in the wrong place. Does anything in the code look out of place, It doesn't to my knowledge but I may be overlooking something.
I have the script in a foreach, so I can initiate the div, for each product.
Edit, viewed apache and mysql logs and it showing nothing. Could it be a memory leak?
I think I have fixed it with the help of someone from an external website. It was to do with the sleep()
I have fixed it using:
session_write_close();
I will do more testing to see how it hold up before reporting back. With the reason why etc.
I am trying to read from a database some data but without waiting for the page to load, so I created an ajax post that starts sending data when page is ready, now after ajax completes I need to read the values from another file. The problem is that after the ajax completes, json that is reading the data is running indefinite.
JQUERY
<?php $url = $_GET['url']; ?>
var jQuery_1_11_0 = $.noConflict(true);
jQuery_1_11_0(document).ready(function () {
var domain = '<?php echo $url; ?>';
$.ajax({
type: 'POST',
url: 'lib/ajax.php',
data: {
action: 'get_all_seo_details', // function that collects data
domain: domain // the domain that is being send to the function in order to get data
},
success: function (data) {
// doesn't need to echo anything only to insert the data, which it done properly
}
});
// Below is the second part of the script that starts when ajax stops
$(document).ajaxStop(function () {
$.getJSON('lib/get-details.php', function(data) {
var twitter_followers = data.twitter_followers;
$('#twitter-followers').html(twitter_followers);
});
// data is being read correctly but it loops repeatedly in the console without finishing
});
});
PHP - get-details.php, reading the data from database after getting inserted with ajax
if (!isset($_SESSION)) {
sec_start();
}
global $db;
$domain = isset($_SESSION['domain']) ? $_SESSION['domain'] : '';
if ($domain == '') {
$domain = $db->query("SELECT * FROM seo_data");
} else {
$domain = $db->query("SELECT * FROM seo_data WHERE domain = '$domain'");
}
$domain_now = $domain->fetch_assoc();
$twitter_followers = (int) $domain_now['twitter_followers'];
echo json_encode(array('twitter_followers' => $twitter_followers));
I'm not sure but when the first AJAX request stops
$(document).ajaxStop(function (){....
starts a new one with
$.getJSON('lib/get-details.php', function(data) { ...
When this second one ends, maybe
$(document).ajaxStop(function (){....
is called again which starts again the 2nd request and so on
I amc creating A Login script with php and javascript.
What I want to do is log the user in without the page refresh which I have archived so far, With some help from Stack Flow users, I am fairly good with PHP but new to the Javascript client side.
Anyway, When the user enters the correct data and the session gets started how do I get it to call the fade out function?
Heres the PHP Side
<?php
require "../core/database.php";
//lets create some veriables to use, This way is shorter
$username = strip_tags(trim($_POST['user_login']));
$password = strip_tags(trim($_POST['pass_login']));
$md5_pass = md5($_POST['pass_login']);
$user_login = mysql_real_escape_string($username);
$pass_login = mysql_real_escape_string($md5_pass);
if (($user_login) && ($password)) {
//Connect to the database to fetch the users username and password
$select_user = mysql_query("SELECT username,password FROM users WHERE username='$user_login' AND password='$pass_login'");
$user_rows = mysql_fetch_array($select_user);
$username_row = $user_rows['username'];
$password_row = $user_rows['password'];
if(($username_row==$user_login) && ($md5_pass==$password_row)) {
//All user information is correct, Now start the session
//I HAVE CALLED IT HERE HOPING THERE,S A BETTER WAY OF DOING THIS. IT WILL CAL
echo "
Yes, Now we can start the session right here, when your ready.
<script>
$('#field').fadeOut();
</script>";
} else {
echo "The username or password you entered is incorrect";
}
} else {
echo "<b>Blank Fields</b> <br>
You must enter A Username/Password Combination";
}
?>
Incase yous need it, there is the client side aswill (modified by some users to make the functionality better)
$(document).ready(function() {
// Make a function that returns the data, then call it whenever you
// need the current values
function getData() {
return {
user_login: $('#user_login').val(),
pass_login: $('#pass_login').val()
}
}
function loading(e) {
$('#content').html('Loading Data');
}
function check(e) {
e.preventDefault();
$.ajax({
url: 'ajax/check.php',
type: 'post',
data: getData(), // get current values
success: function (data) {
$('#content').html(data);
}
});
}
// Don't repeat so much; use the same function for both handlers
$('#field').keyup(function(e) {
if (e.keyCode == 13) {
var username = $('#user_login').val();
loading(e);
check(e);
}
});
$('#submit').click(function(e) {
if (e.keyCode != 13) {
loading(e);
check(e);
}
});
});
Since PHP is Server Side and Java Script controls the Client side, Probably the best way to do or call it is this way, But its worth A ask anyway.
Besides this everything is working out well.
If you want you can help change the way loading data is coded/works, But the functionality is working perfectly so theres not much need.
The ajax success method needs to check the response from the server to see if login was successful and then take the appropriate action:
// php
if(($username_row==$user_login) && ($md5_pass==$password_row)) {
//All user information is correct, Now start the session
echo 'correct';
} else {
echo 'The username or password you entered is incorrect';
}
// js
function check(e) {
e.preventDefault();
$.ajax({
url: 'ajax/check.php',
type: 'post',
data: getData(), // get current values
success: function (data) {
if (data === 'correct') {
$('#field').fadeOut();
} else {
$('#content').html(data);
}
}
});
}
Returning JSON instead of raw HTML is much more flexible. Quick example:
PHP Side
<?php
require "../core/database.php";
$json = array('success' => false, 'error' => null);
$username = strip_tags(trim($_POST['user_login']));
$password = strip_tags(trim($_POST['pass_login']));
$md5_pass = md5($_POST['pass_login']);
$user_login = mysql_real_escape_string($username);
$pass_login = mysql_real_escape_string($md5_pass);
if (($user_login) && ($password)) {
$select_user = mysql_query("SELECT username,password FROM users WHERE username='$user_login' AND password='$pass_login'");
$user_rows = mysql_fetch_array($select_user);
$username_row = $user_rows['username'];
$password_row = $user_rows['password'];
if(($username_row==$user_login) && ($md5_pass==$password_row)) {
$json['success'] = true;
}
else {
$json['error'] = "The username or password you entered is incorrect";
}
} else {
$json['error'] = "<b>Blank Fields</b> <br>You must enter A Username/Password Combination";
}
echo json_encode($json);
Your AJAX function:
function check(e) {
e.preventDefault();
$.ajax({
url: 'ajax/check.php',
type: 'post',
data: getData(), // get current values
success: function (data) {
var loginResult = JSON.parse(data);
if(loginResult.success){
//Login successful - fade out whatever form or fields
//that you want to
$('#field').fadeOut();
} else{
//Add error message to an error div or whatever
$('#error').html(loginResult.error);
}
}
});
}
I'll start by saying that your PHP should be using the newer mysqli_* functions or the PDO object for all of your database queries. Further, you should be using prepared statements which will safeguard you against SQL injection attacks.
Another thing to note is that in a PHP file that is not going to output anything to the browser, or in other words, is just going to run some code, you don't need a closing tag. In fact, you don't want a closing tag. That is because anything after the closing tag will get sent to the browser, which will get included in the response of your AJAX success function. That includes things like spaces and new lines.
Now, on to your PHP. You are going to want to output some JSON so that you can check for success or failure in your AJAX.
PHP
<?php
require "../core/database.php";
//lets create some veriables to use, This way is shorter
$username = strip_tags(trim($_POST['user_login']));
$password = strip_tags(trim($_POST['pass_login']));
$md5_pass = md5($_POST['pass_login']);
$user_login = mysql_real_escape_string($username);
$pass_login = mysql_real_escape_string($md5_pass);
//Create an array to represent our JSON data.
$json = array(
"successCode" => 0
);
if (($user_login) && ($password)) {
//Connect to the database to fetch the users username and password
$select_user = mysql_query("SELECT username,password FROM users WHERE username='$user_login' AND password='$pass_login'");
$user_rows = mysql_fetch_array($select_user);
$username_row = $user_rows['username'];
$password_row = $user_rows['password'];
if(($username_row==$user_login) && ($md5_pass==$password_row)) {
//All user information is correct, Now start the session
//echo "Yes, Now we can start the session right here, when your ready."
$json['successCode'] = 0;
} else {
//echo "The username or password you entered is incorrect";
$json['successCode'] = 1;
}
} else {
//echo "<b>Blank Fields</b> <br>
//You must enter A Username/Password Combination";
$json['successCode'] = 2;
}
//Set that our content type is JSON
header("Content-type: application/json");
echo json_encode($json); //Convert the PHP array to JSON and echo it as the response.
In our PHP, we have created a $json array which will story the successCode that we will be responding to the client. This will tell the client if the login was a success or failure, and even what type of failure occurred. It will then be up to the client to decide how to display that success or failure to the user. This allows multiple applications to use the same server side source, but display the errors differently if desired.
At the end of the PHP, we have set the header Content-type to specify that we are sending back application/json to the client. Then, we encode the PHP array as JSON, and output it to the response.
jQuery/Javascript
//Let's define different messages depending on what status code we get on the client.
var errorMessages = [
"Yes, Now we can start the session right here, when your ready.",
"The username or password you entered is incorrect",
"<b>Blank Fields</b><br />You must enter A Username/Password Combination"
];
function check(e) {
e.preventDefault();
$.ajax({
url: 'ajax/check.php',
type: 'post',
data: getData(), // get current values
success: function (data) {
//First, make sure that data and data.successCode are defined.
if (data && data.successCode) {
//Here, you are getting back the JSON data from the login call.
$('#content').html(errorMessages[data.successCode]);
//If the successCode is 0, which means it was successful, then we want to fade out the #field div.
if (data.successCode == 0) {
$('#field').fadeOut();
}
} else {
//There must've been a server error. You'd handle that here.
}
}
});
}
Why put the error messages on the client instead of the server? Because it allows you to easily change how the error messages are displayed, without having to touch the server side code. The server just outputs an error code, and the client decides how to handle that code.
The Javascript array, errorMessages, defines the error messages corresponding to their index in the array. The error message at index 0 would correspond to successCode = 0, and so on. If you weren't going to use sequential successCodes, you could use a javascript object to specify keys corresponding to each error code.