We Keep Coding

Why doesn't diffbot see the price here?

I'm using diffbot to scrape products. It gets things right on most sites, and if it doesn't the custom API usually allows me to easily tweak until correct. However there are a few cases that are baffling me.
I know diffbot doesn't execute javascript in the custom API preview window, but for the product endpoint, it should always execute it when a request is made to the API (e.g. from the diffbot client in a Python shell).
Foot asylum
For products on this website, e.g. https://www.footasylum.com/hugo-boss-three-pack-tshirt-103678/, the offerPrice field is empty. I can see the price is in a div#priceFrm, so I try to edit and add a custom selector on that field to this effect. However even when making a new API call from the Python shell, the response is 'offerPrice': ''.
This price is obviously being added by Javascript, but why can't diffbot deal with that? What can I do about it?
I can also see the price I want can be found in some JSON data inside a <script>. Normally I could just scrape it from there, with //script[contains(text(), "dataLayer")]/text() followed by a regex. However in another diffbot custom field I defined a selector script:contains(dataLayer) and even this is blank.
Any ideas on getting the price from this product with diffbot?
Nike
I'm also trying to get the price from https://www.nike.com/gb/t/flyknit-trainer-shoe-GBXjsV/AH8396-600
The first problem is the preview window of custom API just gives a 500 error weirdly.
Next I edit the offerPrice field with a custom selector of div[data-test=product-price], however this field doesn't hit anything - even when called from client in Python shell.
Footlocker
Finally on this site https://www.footlocker.co.uk/en/p/jordan-1-flight-2-men-shoes-6671?v=314100340604#!searchCategory=all diffbot cannot seem to get product image.
The images are loaded by "scene7", and with XPATH can be found with //div[#class="s7thumb"][#data-namespace="s7classic"]/#style and then parsing out the "background-url".
I tried to at least get the style attribute with diffbot using the selector div.s7thumb div[data-namespace=s7classic] and then adding the Attribute filter "style", but again nothing at all is returned.

In some cases, specific rendering of certain elements will be blocked either by Diffbot's renderer or by a target site's anti-block measures. That's why Diffbot has X-eval functionality which lets you add custom JavaScript into calls which will get executed on a target site, as if running from the console. In this case, something like the following helps:
function() {
start();
setTimeout(function() {
price = document.querySelector("[itemprop="
Offers "] [itemprop="
price "]");
currency = document.querySelector("[itemprop="
Offers "] [itemprop="
priceCurrency "]").getAttribute("content");
price.parentElement.setAttribute("style", "");
price.parentElement.innerHTML += '<h1 class="thePrice">' + price.innerText + " " + currency + '</h1>';
setTimeout(function() {
end();
}, 500);
}, 500);
}
This has been applied as a fix and the price returns now.

Text to image conversion for table not working when running back end php to get the table data

So I have a drop down selector when certain items are ticked, it filters based on SQL WHERE parameter.
I'm doing this the way This user describes it
For most part, it works completely fine, and if I open the back end file directly the conversion of value to image works fine.
This is what it looks like when I open the back end directly
However, from my main page, if I try to call this back end php it does not show the image at all.
Instead shows up like this
To convert the image we are using this javascript
$.expr[':'].textEquals = $.expr.createPseudo(function(arg) {
return function( elem ) {
return $(elem).text().match("^" + arg + "$");
};
}
);
along with something like this
$('.spec:textEquals("1")').html("<img id='id' title='title' src='src'/>");
Any help to resolve this problem would be appreciated

Javascript code running fine until put into a function

I have taken some source code from here to submit a form using AJAX. The form is basically taking some information from a user and putting it into a database via PHP. The code I have works, but given that what I am working on has many forms all doing the same thing, I - obviously - want to make sure my code is lean and mean. So, making sure that each of my form field names have the same as my database with some matching IDs for various parts of the form for user feedback, have changed it to the following:
<script type="text/javascript">
$(document).ready(function() {
// process the form
$('#formId').submit(function(event) { // for the specified form:
// completeForm('#formId'); // WHERE I CALL THE FUNCTION
// FUNCTION STARTS HERE WHEN IT IS ONE
var formData = {}; formId = '#formId';
$(formId + ' input, ' + formId + ' select, ' + formId + ' textarea').each(
function(index){ // for each item (input, select, textarea in the specified form
var input = $(this);
// First, clear any existing formatting that has been added by previous form inputs
$('#' + input.attr('id') + '-group').removeClass('has-info has-error has-success bg-error bg-info bg-success');
// Next, add data to the array of data to pass to the PHP script
Array.prototype.push.call(formData, input.val());
}
); // End of loop through each item.
// Now the data is collected, call the requested PHP script via AJAX.
$.ajax({
type : 'POST', // define the type of HTTP verb we want to use (POST for our form)
url : $(this).attr('action'), // '/processing/general_settings_processing.php', // the url where we want to POST
data : formData, // our data object
dataType : 'html' // 'json' // what type of data do we expect back from the server
})
// using the done promise callback
.done(function(data) {
// log data to the console so we can see
console.log(data);
// Return success and error messages to the user
// Code to come once the basics have been sorted out!
});
// FUNCTION WOULD END HERE WHEN IT IS ONE.
event.preventDefault();
});
});
</script>
The code as above works absolutely fine; the relevant PHP file is called and - although I have no processing done in this particular file yet - does its stuff (I have it echoing the $_POST array to a file and returning to view in the console log atm).
However, when I try and make it a function, it doesn't - the console log simply echoes out the source code for this document instead of the PHP array that it supposed to be doing. The function is placed above the $(document).ready line; specified as function completeForm(formID) { . . . } , contains the section of code as noted in the comments and called in the commented out line as shown. So, logically (to me) it should work.
The ultimate idea is to have the function to do this in a file that can be called by all the forms that call it, while the code to call the function is in the relevant part of the page. Some pages will have more than one form using it. (I mention that should even if what I am doing here works, it wouldn't when I come to reuse the code!)
(I'm relatively new to JS and JQuery, although have a fairly good grasp of some programming techniques, mainly these days just in PHP.)

The issue you are having with making that a function is you forget to include the "thisBinding". As a result, when you tried to use the form's action target in your ajax call with this code:
url : $(this).attr('action')
it does not find an "action" attribute - basically the issue is that this is actually window and as a result there is no action attribute. Simply bind this to your function call and your code will work as written.
completeForm.call(this,'#formId');
.call MDN

jQuery Parse JSON Response

I am successfully Posting to a PHP file, and getting a good response. The part I cannot seem to get is parsing it out then displaying it on my page. Here is my javascript in a validate handler:
submitHandler: function(form) {
var formData = $(form).serialize();
$.post('http://test.php', formData, function(data) {
if(data.success) {
$('#load').show();
var response = i;
$('#load').hide();
//var msg = '';
for(var i = 0; i < x.flights.length; i++) {
msg += '<span>';
msg += '<p>Flight Number: ' + x.flights[i].flight_number + '</p>';
msg += '<p>Cost: ' + x.flights[i].cost + '</p>';
msg += '</span>';
}
//this is were I think it should display. but It's not working
$('#load').html(msg);
Here is my json response:
success
true
message
"Success"
flights
[Object { flight_number="334", cost="983.40", departure_city="Kearney Regional Airpor...arney, Nebraska - (EAR)", more...}]
0
Object { flight_number="334", cost="983.40", departure_city="Kearney Regional Airpor...arney, Nebraska - (EAR)", more...}
flight_number
"334"
cost
"983.40"
departure_city
"Kearney Regional Airport, Kearney, Nebraska - (EAR)"
arrival_city
"Chadron Muni Airport, Chadron, Nebraska - (CDR)"
departs
"2014-03-19 04:33:00"
arrives
"2014-03-19 08:12:00"
duration
"219"
adult_seats_available
"2"
senior_seats_available
"1"
I know you you are not seeing the JSON response, but I can see it in FF firebug. I'm new to jQuery/JSON, and I just want to print the response to my page. Thanks in advance.

Look into JSON.stringify(data) doc I don't know where you want to display your JSON but if you just want to see what it is open up the debugger and console.log(JSON.stringify(data)); should do it.

You are using post method and it may default return xml, json, script, text, html. So you are getting the data from post method.
Try looking at https://api.jquery.com/jQuery.post/
And if you want to access it, then you can use JSON.stringify(data) to show json as a string. And to access data from json you can use dot(.) notation.

There's a couple of things to look at here.
First you're adding your html (ie msg) to the #load element ($('#load').html(msg);) but earlier in the code you're hiding it ($('#load').hide();). So I think this will answer your main question; i.e. remove the the line $('#load').hide(); or add $('#load').show(); after the line $('#load').html(msg);.
Still not seeing anything? Then perhaps the response isn't as corrent as you're claiming so perhaps an easier way to check what's been assigned to msg is to alert the html alert(msg); <- make this call after you've built your html.
So aside from this is the use of the #load element you seem to be using it to hold a loading gif but also assigning it the response via the content of msg. Perhaps you need to separate the use of the elements so that the #load element is for the loading gif and you add another element for the response. so you're code is more like this.
html
<div id="load" class="hide"><img src="loading-animation.gif">...</div>
<div id="post-response" class="hide alert"><div>
where the hide class is display none and alert class represents an style for a response alert.
js
submitHandler: function(form) {
// show the loading gif before we make the
// post data and wait for an async response
$('#load').show();
...
$.post('http://test.php', formData, function(data) {
if(data.success) {
// post-back has completed so lets hide the animation
$('#load').hide();
// your code to build html msg
// assign and show the response element
$('post-response').html(msg);
$('post-response').show();
...

Append string to url using jquery or javascript

Is it possible to append a string to url using the body? Something similar to the JSFiddle below.
http://jsfiddle.net/qXs77/
hi
I want to append this query to the url: ?q1=iPodTouch&x=79&y=20
When the user clicks the link they will be taken to the next page with the attached string above.

Updated :
No need to pass it on another function ,
Use like this:
Navigate

If you wanted to utilize jQuery more you could try this approach.
$('a[data-page]').on('click', function(e){
e.preventDefault();
var page = $(this).data('page');
location.href += ( location.search.length ? '&' : '?' ) + 'page=' + page;
});
and have the HTML look like:
Page 2