We Keep Coding

Can't use JSON.parse() to import array from PHP [duplicate]

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 8 months ago.
So I've got a form where I can create multiple datalist-text-inputs of the same type that is later (onclick) put into an invisible input before submitted.
These datalist-text-inputs is created onload so that I can add more with an "+" button with the same function.
The options are imported from my PHP-database, and it worked fine. But suddenly it stopped working and I don't know why. I've tried a thousand things but can't figure it out.
I'm quite new to PHP. I think the problem has to do with JSON.parse() since the code breaks on that line.
script.js
var ajax = new XMLHttpRequest();
ajax.open("GET", "fetch data.php", true);
ajax.send();
ajax.onreadystatechange = function() {
if(this.readyState == 4 && this.status == 200) {
var data = JSON.parse(this.responseText);
var html = "";
for (var a = 0; a < data.length; a++) {
var firstName = data[a].name;
html += "<option value='" + firstName + "'></option>";
};
document.getElementById(type+"list"+addnumber).innerHTML = html;
};
};
type+"list"+addnumber it the name of the input-text-box. Type is an argument and addnumber an variable/integer.
fetch data.php
<?php
$host = "localhost"; $user = "root"; $pass = ""; $db = "collection";
$conn = mysqli_connect($host, $user, $pass, $db);
$result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
$data = array();
while ($row = mysqli_fetch_assoc($result)) {
$data[]=$row;
};
echo json_encode($data);
?>
Also, I might add that this function creates objects in three places on the same page but the value is added/moved to three different invisible inputs.

Based on the exception you're seeing, ie:
JsonException: Malformed UTF-8 characters, possibly incorrectly encoded in C:\wamp64\www\collection\fetch data.php on line <i>11</i>
My guess is that the data you are reading is not encoded in a way which json_encode expects.
The simplest (NOT RECOMMENDED) approach is to pass in the JSON_INVALID_UTF8_IGNORE or JSON_INVALID_UTF8_SUBSTITUTE flags, which will cause bad data to be silently skipped over (or replaced with the unicode REPLACEMENT CHARACTER \0xfffd), rather than treating it as an error. See the documentation for json_encode and predefined JSON constants.
If you want to get to the root of the problem so that all data is correctly encoded as JSON, however:
You can try to force the encoding by setting the mysql character set using PHP's mysqli_set_charset function, eg:
<?php
$host = "localhost"; $user = "root"; $pass = ""; $db = "collection";
$conn = mysqli_connect($host, $user, $pass, $db);
if (!mysqli_set_charset($conn, 'utf8')) {
throw new \Exception("failed to set mysql charset");
}
$result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
...
?>
The most common charsets in my experience are utf8, utf8mb4. If you know your data to contain some other specific character set, you may need to translate it into utf8 before trying to encode it into JSON using PHP's mb_convert_encoding function.
Finally, it could be that the issue has occurred earlier in your application, resulting in bad (mixed-encoding) data. If this is the case, you'll need to detect the bad data row-by-row, perhaps outputting where exceptions were raised in a separate error report, and manually correct the encoding of that data. This can be prevented by ensuring the data is validated and correctly encoded as utf8 before it reaches your database. Note that ensuring the mysql character set is correctly set for all connections is also potentially a part of this solution. It may be that you'll want to configure your database to do this automatically.
example of detection and logging:
<?php
$host = "localhost"; $user = "root"; $pass = ""; $db = "collection";
$conn = mysqli_connect($host, $user, $pass, $db);
if (!mysqli_set_charset($conn, 'utf8')) {
throw new \Exception("failed to set mysql charset");
}
$result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
$data = array();
while ($row = mysqli_fetch_assoc($result)) {
try {
// detect rows which cannot be encoded
$discard = json_encode($row, JSON_THROW_ON_ERROR);
} catch (\JsonException $e) {
// keep track of failed rows so we can correct them later
my_custom_logging_function($row);
// and skip the row so we don't try to encode it later
continue;
}
$data[]=$row;
};
echo json_encode($data, JSON_THROW_ON_ERROR);
?>

OK just change "fetch data.php" to "data.php"

Detect specific info from variable [duplicate]

I have an HTML form field $_POST["url"], having some URL strings as the value.
Example values are:
https://example.com/test/1234?email=xyz#test.com
https://example.com/test/1234?basic=2&email=xyz2#test.com
https://example.com/test/1234?email=xyz3#test.com
https://example.com/test/1234?email=xyz4#test.com&testin=123
https://example.com/test/the-page-here/1234?someurl=key&email=xyz5#test.com
etc.
How can I get only the email parameter from these URLs/values?
Please note that I am not getting these strings from the browser address bar.

You can use the parse_url() and parse_str() for that.
$parts = parse_url($url);
parse_str($parts['query'], $query);
echo $query['email'];
If you want to get the $url dynamically with PHP, take a look at this question:
Get the full URL in PHP

All the parameters after ? can be accessed using $_GET array. So,
echo $_GET['email'];
will extract the emails from urls.

Use the parse_url() and parse_str() methods. parse_url() will parse a URL string into an associative array of its parts. Since you only want a single part of the URL, you can use a shortcut to return a string value with just the part you want. Next, parse_str() will create variables for each of the parameters in the query string. I don't like polluting the current context, so providing a second parameter puts all the variables into an associative array.
$url = "https://mysite.com/test/1234?email=xyz4#test.com&testin=123";
$query_str = parse_url($url, PHP_URL_QUERY);
parse_str($query_str, $query_params);
print_r($query_params);
//Output: Array ( [email] => xyz4#test.com [testin] => 123 )

As mentioned in another answer, the best solution is using parse_url().
You need to use a combination of parse_url() and parse_str().
The parse_url() parses the URL and return its components that you can get the query string using the query key. Then you should use parse_str() that parses the query string and returns
values into a variable.
$url = "https://example.com/test/1234?basic=2&email=xyz2#test.com";
parse_str(parse_url($url)['query'], $params);
echo $params['email']; // xyz2#test.com
Also you can do this work using regex: preg_match()
You can use preg_match() to get a specific value of the query string from a URL.
preg_match("/&?email=([^&]+)/", $url, $matches);
echo $matches[1]; // xyz2#test.com
preg_replace()
Also you can use preg_replace() to do this work in one line!
$email = preg_replace("/^https?:\/\/.*\?.*email=([^&]+).*$/", "$1", $url);
// xyz2#test.com

Use $_GET['email'] for parameters in URL.
Use $_POST['email'] for posted data to script.
Or use _$REQUEST for both.
Also, as mentioned, you can use parse_url() function that returns all parts of URL. Use a part called 'query' - there you can find your email parameter. More info: http://php.net/manual/en/function.parse-url.php

You can use the below code to get the email address after ? in the URL:
<?php
if (isset($_GET['email'])) {
echo $_GET['email'];
}

I a created function from Ruel's answer.
You can use this:
function get_valueFromStringUrl($url , $parameter_name)
{
$parts = parse_url($url);
if(isset($parts['query']))
{
parse_str($parts['query'], $query);
if(isset($query[$parameter_name]))
{
return $query[$parameter_name];
}
else
{
return null;
}
}
else
{
return null;
}
}
Example:
$url = "https://example.com/test/the-page-here/1234?someurl=key&email=xyz5#test.com";
echo get_valueFromStringUrl($url , "email");
Thanks to #Ruel.

$web_url = 'http://www.writephponline.com?name=shubham&email=singh#gmail.com';
$query = parse_url($web_url, PHP_URL_QUERY);
parse_str($query, $queryArray);
echo "Name: " . $queryArray['name']; // Result: shubham
echo "EMail: " . $queryArray['email']; // Result:singh#gmail.com

A much more secure answer that I'm surprised is not mentioned here yet:
filter_input
So in the case of the question you can use this to get an email value from the URL get parameters:
$email = filter_input( INPUT_GET, 'email', FILTER_SANITIZE_EMAIL );
For other types of variables, you would want to choose a different/appropriate filter such as FILTER_SANITIZE_STRING.
I suppose this answer does more than exactly what the question asks for - getting the raw data from the URL parameter. But this is a one-line shortcut that is the same result as this:
$email = $_GET['email'];
$email = filter_var( $email, FILTER_SANITIZE_EMAIL );
Might as well get into the habit of grabbing variables this way.

$uri = $_SERVER["REQUEST_URI"];
$uriArray = explode('/', $uri);
$page_url = $uriArray[1];
$page_url2 = $uriArray[2];
echo $page_url; <- See the value
This is working great for me using PHP.

In Laravel, I'm using:
private function getValueFromString(string $string, string $key)
{
parse_str(parse_url($string, PHP_URL_QUERY), $result);
return isset($result[$key]) ? $result[$key] : null;
}

A dynamic function which parses string URL and gets the value of the query parameter passed in the URL:
function getParamFromUrl($url, $paramName){
parse_str(parse_url($url, PHP_URL_QUERY), $op); // Fetch query parameters from a string and convert to an associative array
return array_key_exists($paramName, $op) ? $op[$paramName] : "Not Found"; // Check if the key exists in this array
}
Call the function to get a result:
echo getParamFromUrl('https://google.co.in?name=james&surname=bond', 'surname'); // "bond" will be output here

parse into another array contain xml child node using php

currently, im having problem to parse xml node in array using condition where parse with <mo> as separator
this is my array(0)
Array([0] => <mi>x</mi><mo>+</mo><mn>2</mn><mo>=</mo><mn>3</mn>);
i want to parse like this
Array[0] => <mi>x</mi>
Array[1] =><mo>+</mo><mn>2</mn>
Array[2]=><mo>=</mo><mn>3</mn>
this is my coding
<?
$result(0)="<mi>x</mi><mo>+</mo><mn>2</mn><mo>=</mo><mn>3</mn>";
$result1= new simplexml_load_string($result);
$arr_result=[];
foreach($result1 as $key => $value){
$exp_key = explode('<', $key);
if($key[0] == 'mo'){
$arr_result[] = $value;
}
print_r($arr_result);
}
if(isset($arr_result)){
print_r($arr_result);
}
?>
thanks in advance !

The approach with XML seems excessive since what you really want is to pull out substrings of a string based on a delimiter.
Here is a working example. It works by finding the position of <mo> and cutting off that section, then searching for the next <mo> in the remain string.
<?php
$result(0)="<mi>x</mi><mo>+</mo><mn>2</mn><mo>=</mo><mn>3</mn>";
$res = $result(0);
$arr_result=[];
while($pos = strpos($res, "<mo>", 1)) {
$arr_result[] = substr($res, 0, $pos); // grab first match
$res = substr($res, $pos); // grab the remaining string
}
$arr_result[] = $res; // add last chunk of string
print_r($arr_result);
?>
Your code above has several issues.
First:
$result1= new simplexml_load_string($result); // simplexml_load_string() is a function not a class
Second:
$key and $value do not contain the '<' and '>' so, this part:
$exp_key = explode('<', $key); will never do anything and isn't needed.
Third:
If your code did work it would only return array('+', '=') because you are appending the data inside the mo element to the result array.

AJAX returning a white space from PHP echo

This is a problem I been having several times, and I always ignore because I cant find a solution.
Basically every time I echoed a value back to Javascript using AJAX, the first value will contain a space (extra character) this is annoying because if I want to check if that value exist sometimes I cant due to the space.
Normally the values returned is something like this
Value1
Value2
Value3
Here is the code below. I suggest to ignore the function seperate_ajx_data as I dont think the problem is located there.
results[0] is giving a white space plus the value!
What could be the issue of having a space in the first value?
PHP code:
$machine = null;
$sql = mysqli_query($connection, "SELECT......");
while($row = mysqli_fetch_array($sql)){
$machine .= $row['MACHINE']."+";
$machine .= $row['COUNT(MACHINE)']."/";
}
echo $machine;
JavaScript code:
function get_machine_vals(id){
var ht = new XMLHttpRequest();
var url = "....";
var val = "....="+id;
ht.open("POST", url, true);
ht.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
ht.onreadystatechange = function(){
if(ht.readyState == 4 && ht.status == 200){
var val = ht.responseText;
var results = seperate_ajx_data(["/","+"],val);
var el = document.getElementById("machine_dropdown");
var opts = el.options;
for(var i =0;i<results.length;i++){
var tmp = results[i];
alert(tmp + " " + tmp.length);
switch (results[i]){
case "Test":
opts[0].innerHTML = "Seko "+results[i+1];
break;
}
}
}
};
ht.send(val);
}
function seperate_ajx_data(symbols, val){
var tmp_storage = [];
var tmp_spliter = val.split(symbols[0]);
var results = [];
for(var x = 0;x<tmp_spliter.length;x++){
tmp_storage = tmp_spliter[x].split(symbols[1]);
for(var y = 0;y<tmp_storage.length;y++){
results.push(tmp_storage[y]);
}
}
return results;
}

This is unfortunately one of those "gotchas" when working with PHP. The errant character might come from a space after a closing PHP tag. A contrived example:
<?php
... do some work ...
?> <?php
... do some work, but note errant space between the tags
Possible fixes?
Don't close the PHP tag
Use a different transfer mechanism that is space-tolerant, for example JSON or XML.
Strip (or "trim") your values client side. In context of your code:
var val = (ht.responseText || '').trim();

Just check your opening <?php brackets, there might be a space (i've seen this before).

It happened with me today. And unlike other answers related to opening bracket, for me it was the space after the closing bracket.
<?php
.....
?>[space][space][space]
I removed the spaces. And solved the issue.

Chunking MySQL array with json encode

I know there are existing some Questions about Chunking a mysql array in php, but my problem is, that I want to keep the output in JSON.
Scenario:
I want to get data from mysql, do some stuff with it ( like time formatting ) and output it in JSON.
The JSON data is parsed in the browser and visualized over a javascript chart.
Problem:
All of the above is working, but because of the huge amount of data, I'm getting an out of memory error, when I select bigger date ranges to output.
The Idea of directly sending out each x-lines of data is not working because of the JSON format it needs to be. Several JSON chunks won't work, it needs to be one for the chart.
So in the end I need to chunk the data but keep it as one big JSON.
(And setting up the memory limit is not really a solution.)
Ideas:
One Idea would be, to let the browser chunk the date range and ask the data as chunks & then put them together.
Of course this would work, but if there is a way to do this server side, it would be better.
Code:
private function getDB($date1, $date2){
$query = 'SELECT * FROM `db1`.`'.$table.'` WHERE `date` BETWEEN "'.$date1.'" AND "'.$date2.'" order by `date`;';
// date = datetime !
$result = $this->db->query($query);
$array = array();
while ( $row = $result->fetch_assoc () ) {
$array[] = array( strtotime($row[ 'date' ])*1000 , (float)$row[ 'var' ] );
// the formatting needs to be done, so the chart accepts it..
}
$result->close();
return json_encode($array);
}

Since this is not an option,
ini_set("memory_limit","32M")
perhaps you can add LIMIT to the function paramaters and query:
private function getDB($date1, $date2, $start, $pageSize){
$query = 'SELECT * FROM `db1`.`'.$table.'` WHERE `date` BETWEEN "'.$date1.'" AND "'.$date2.'" order by `date` LIMIT $start, $pageSize;';
// date = datetime !
$result = $this->db->query($query);
$array = array();
while ( $row = $result->fetch_assoc () ) {
$array[] = array( strtotime($row[ 'date' ])*1000 , (float)$row[ 'var' ] );
// the formatting needs to be done, so the chart accepts it..
}
$result->close();
return json_encode($array);
}
Then setup a for loop in javascript, call this with Ajax, incrementing the $start variable each time.
Store each responseText.substr(1).substr(-1) in an array.
When the responseText is "", all of the records have been returned.
.join the array with a comma, then add a new opening and closing "{ }", and you should have a JSON equivalent to all records.
Minimal parsing, and you'll be using built-in functions for most of it.
var startRec=0;
var pageSize=50;
var xmlhttp=new XMLHttpRequest();
var aryJSON=[];
var JSON;
xmlhttp.onreadystatechange=function(){
if (xmlhttp.readyState==4 && xmlhttp.status==200){
if(xmlhttp.responseText==""){ //Might need to check for "{}" here instead of ""
//All records are received
JSON="{" + aryJSON.join(",") + "}";
aryJSON=[];
startRec=0
}else{
aryJSON.push(xmlhttp.responseText.substr(1).substr(-1));
startRec+=pageSize;
getNextPage();
}
}
}
function getNextPage(){
xmlhttp.open("GET","your.php?start=" + startRec + "&pageSize=" + pageSize,true);
xmlhttp.send();
}

I would recommend that you have the server send the browser exactly what it needs to create the table. Parsing can be a heavy task, so why have the client do that lifting?
I would have your backend send the browser some kind of data structure that represents the table (i.e. list of lists), with all the formatting already done. Rendering the table should be faster and less memory-intensive.

One way of answer would be, to do the chunking on the server, by giving out the JSON, removing the leading [ & ].
#apache_setenv('no-gzip', 1);
#ini_set('zlib.output_compression', 0);
#ini_set('implicit_flush', 1);
$array = array();
echo '[';
$started = false;
while ( $row = $result->fetch_assoc () ) {
$array[] = [ strtotime($row[ 'datetime' ])*1000 , (float)$row[ 'var' ] ];
if(sizeof($array) == 1000){
if($started){
echo ',';
}else{
$started = true;
}
echo substr(substr(json_encode($array),1), 0, -1);
// converting [[datetime1, value1],[datetime2, value2]]
// to [datetime1, value1],[datetime2, value2]
ob_flush();
$array = array();
}
}
if($started)echo ',';
$this->flushJSON($array);
echo ']';
flush();
$result->close();
This is working and reducing the ram usage to 40%.
Still it seems that Apache is buffering something, so the ram usage increases over the time, the script is running. (Yeah, the flush is working, I debugged that, that's not the problem.)
But because of the remaining increase, the fastest way to achieve a clean chunking is to do this like alfadog67 pointed it out.
Also, to mention it, I had to disable the output compression, otherwise apache wouldn't flush it directly..