I'm helping a small client with a rather simple issue, but it turned out not to be so simple.
byseven.dk (in the footer)
Has a contact formular, which says "Navn (Required)" now the first part is danish the second is english, and I would like to replace the (Required) with a simple *.
However I can't seem to locate the specific place for the (Required)
Is there a simple way to track this down, as I'm sure it goes something like this
frontpage.php -> footer.php -> sidebar(widget).php (functions.php) -> some template file I can't find.
// Update 2
First of, I have no clue what this -3 and +5 system is, however I assume it's something negative, would anyone be so kind as to explain to me, what the problem with the question is?
Second, I found the widget in the backend, which reads the following:
[contact-form to='kontakt#byseven.dk' subject='Kontaktformular'][contact-field label='Navn' type='name' required='1'/][contact-field label='Email' type='email' required='1'/][contact-field label='Besked' type='textarea' required='1'/][/contact-form]
And so the label says 'Navn' but not 'Navn (required)' I'm assuming this could be due to a script inserting this 'Required' after whatever the label is.
Still unable to find the file in which this is inserted. I did a search for the Cmd + F throug all files inside the theme folder of the FTP.
// Update 3
For now I fixed it with a script, but still I would like to know how to track the origin of the code down.
function staticTranslation() {
jQuery( 'label span' ).replaceWith( ' *' );
}
staticTranslation();
I would start with
find . -name ¨*.php¨ -exec grep Navn {} \;
assuming the server is Linux. If not use whatever tool you have for a recursive pattern search.
Then search through *.js, *.html and whatever other next files you have got. There is a chance this might be off the file system and be residing in a database.
Related
I looked up various questions and answers but unfortunately none of the problems I found dealt with a case that is similar to mine. In a typical question, the JavaScript table builds up directly when the website is loaded. In my case, however, I first have to navigate through the JavaScript module and select several criteria before I get the sought-after result.
This is my case: I have to scrape the exchange rates for various currencies from this website www.globocambio.co. To do that, I have (1) to navigate to “I WANT COLOMBIAN PESO”, (2) select the currency (e.g., “Chilean Peso”), (3) and the collection destination (e.g., “El Dorado International Airport”). Only then the respective exchange rate is being loaded. See this screenshot for illustration. I marked the three selection steps red. Green is the data point that I want to scrape for different currencies.
I am not very familiar with JavaScript but I tried to understand what is going on. Here is what I found out:
Using Chrome DevTools, I investigated the Network activity when loading an exchange rate. There is an XHR called “GetPrice” that requests the price using this URL: https://reservations.globocambio.co/DesktopModules/GlobalExchange/API/Widget/GetPrice and using the following Form Data
ISOAOrigen=CLP&cantidadOrigen=9000&ISOADestino=COP&cantidadDestino=0¢erId=27&operationType=OperationTypesBuying
I understand that the Form Data contains the information that I initially selected manually:
operationType=OperationTypesBuying: this is the “I WANT COLOMBIAN PESO” option
ISOAOrigen=CLP: this is the “Chilean Peso”
centerId=27: this is the “El Dorado International Airport”
The server responds to my request with the following information:
{“MonedaOrigen":{"ISOA":"CLP","Nombre":null,"Margen":0.1630000000,"Tramo":0.0,"Fixing":2.9000000000},"CantidadOrigen":9000.00,"MonedaDestino":{"ISOA":"COP","Nombre":null,"Margen":0.0,"Tramo":0.0,"Fixing":0.0},"CantidadDestino":21845.70,"TipoCambio":2.42730000000000000000,"MargenOrigen":0.0,"TramoOrigen":0.0,"FixingOrigen":0.0,"MargenDestino":0.0,"TramoDestino":0.0,"FixingDestino":0.0,"IdCentro":"27","Comision":null,"ComisionTramoSuperior":null,"ComisionAplicada":{"CodigoMoneda":null,"CodigoTipoMoneda":0,"ComisionFija":0.0,"ComisionVariable":0.0,"TramoInicio":0.0,"TramoFin":null,"Orden”:0}}
From this response, "TipoCambio":2.42730000000000000000 is then being written on the website using this line of HTML code: <span id="spTipoCambioCompra">2.427300</span>
This means that "TipoCambio" is the value that I am looking for.
So, I have to communicate somehow via R with the server using the Form Data as input variables. Can anyone tell me how to do this?
I mean, understand that I have to combine the URL https://reservations.globocambio.co/DesktopModules/GlobalExchange/API/Widget/GetPrice with the Form Data “ISOAOrigen=CLP&cantidadOrigen=9000&ISOADestino=COP&cantidadDestino=0¢erId=27&operationType=OperationTypesBuying” somehow but I do not know how it works..
Any help will be appreciated!
Update:
I still have no idea how to solve the above issue, yet. However, I try to approach it with small steps.
Using RSelenium, I am currently trying to find out how to click on the option “I WANT COLOMBIAN PESO”. My idea was to use the following code:
library(RSelenium)
remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
port = 4445L,
browserName = "chrome")
remDr$open()
remDr$navigate("https://www.globocambio.co/en/home")
webElem <- remDr$findElement("id", "tabCompra") #What is wrong here?
webElem$clickElement() # Click on "I WANT COLOMBIAN PESO"
But I get an error message after executing webElem <- remDr$findElement("id", "tabCompra"):
Selenium message:no such element: Unable to locate element: {"method":"css selector","selector":"#tabCompra"}
(Session info: chrome=81.0.4044.113)
For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html
...
Error: Summary: NoSuchElement
Detail: An element could not be located on the page using the given search parameters.
class: org.openqa.selenium.NoSuchElementException
Further Details: run errorDetails method
What am I doing wrong here?
I solved my problem using selenium in Python:
from selenium import webdriver
driver = webdriver.Firefox(executable_path = '/your_path/geckodriver')
driver.get("https://www.globocambio.co/en/")
driver.switch_to.frame("iframeWidget");
elem = driver.find_element_by_id('tabCompra')
elem.click()
elem = driver.find_element_by_id('inputddlMonedaOrigenCompra')
elem.click()
elem.send_keys(Keys.CLEAR)
elem.send_keys("Chilean Peso")
elem.send_keys(Keys.ENTER)
elem.send_keys(Keys.ARROW_DOWN)
elem.send_keys(Keys.RETURN)
elem = driver.find_element_by_id('info-change-compra')
print(elem.text)
I'm using diffbot to scrape products. It gets things right on most sites, and if it doesn't the custom API usually allows me to easily tweak until correct. However there are a few cases that are baffling me.
I know diffbot doesn't execute javascript in the custom API preview window, but for the product endpoint, it should always execute it when a request is made to the API (e.g. from the diffbot client in a Python shell).
Foot asylum
For products on this website, e.g. https://www.footasylum.com/hugo-boss-three-pack-tshirt-103678/, the offerPrice field is empty. I can see the price is in a div#priceFrm, so I try to edit and add a custom selector on that field to this effect. However even when making a new API call from the Python shell, the response is 'offerPrice': ''.
This price is obviously being added by Javascript, but why can't diffbot deal with that? What can I do about it?
I can also see the price I want can be found in some JSON data inside a <script>. Normally I could just scrape it from there, with //script[contains(text(), "dataLayer")]/text() followed by a regex. However in another diffbot custom field I defined a selector script:contains(dataLayer) and even this is blank.
Any ideas on getting the price from this product with diffbot?
Nike
I'm also trying to get the price from https://www.nike.com/gb/t/flyknit-trainer-shoe-GBXjsV/AH8396-600
The first problem is the preview window of custom API just gives a 500 error weirdly.
Next I edit the offerPrice field with a custom selector of div[data-test=product-price], however this field doesn't hit anything - even when called from client in Python shell.
Footlocker
Finally on this site https://www.footlocker.co.uk/en/p/jordan-1-flight-2-men-shoes-6671?v=314100340604#!searchCategory=all diffbot cannot seem to get product image.
The images are loaded by "scene7", and with XPATH can be found with //div[#class="s7thumb"][#data-namespace="s7classic"]/#style and then parsing out the "background-url".
I tried to at least get the style attribute with diffbot using the selector div.s7thumb div[data-namespace=s7classic] and then adding the Attribute filter "style", but again nothing at all is returned.
In some cases, specific rendering of certain elements will be blocked either by Diffbot's renderer or by a target site's anti-block measures. That's why Diffbot has X-eval functionality which lets you add custom JavaScript into calls which will get executed on a target site, as if running from the console. In this case, something like the following helps:
function() {
start();
setTimeout(function() {
price = document.querySelector("[itemprop="
Offers "] [itemprop="
price "]");
currency = document.querySelector("[itemprop="
Offers "] [itemprop="
priceCurrency "]").getAttribute("content");
price.parentElement.setAttribute("style", "");
price.parentElement.innerHTML += '<h1 class="thePrice">' + price.innerText + " " + currency + '</h1>';
setTimeout(function() {
end();
}, 500);
}, 500);
}
This has been applied as a fix and the price returns now.
I'm attempting to get Roxy Fileman to work in my environment, however running into a little hitch. I can see the directories for images (done via a symlink) however when returning from the Roxy UI via the "select" button I'm getting a url like this:
https://images.example.com/path/to/Uploads/symlink/123/logo.png
And what I really want is:
https://images.example.com/symlink/123/logo.png
I've already put in the RETURN_URL_PREFIX, however I'm still getting the extended path whether I use a session_path_key or just the regular root (and then browse to the correct directory via the symlink).
My code for the session_path_key looks like:
<?php $_SESSION['dynamic-user-folder'] = "/path/to/Uploads/symlink/"; ?>
and my config.json:
"FILES_ROOT": "",
"RETURN_URL_PREFIX": "https://images.example.com/",
"SESSION_PATH_KEY": "dynamic-user-folder",
Ok, I found the answer. In the plugin php folder there is a file called filelist.php. In here I simply added a
$fullPath = str_replace('/path/to/Uploads/', '', $fullPath);
And this returns the path variable p with all of the stuff I don't want removed back to the UI, so when you do a select it uses the correct URL with the RETURN_URL_PREFIX in front and then the symlink and file name.
First of all I have to say: be patient with me because I am not familiar with the argument that I am going to illustrate you.
I'd like to download the intraday historical values of some equities on Frankfurt Boerse website. Let me take this equity for example: http://www.boerse-frankfurt.de/en/equities/adidas+ag+DE000A1EWWW0/price+turnover+history/tick+data#page=1
As you can see there are two options: trades on Frankfurt and trades on Xetra. I'd love to download the latters. I tried to scrape the data but my knowledge of python is very poor.
How can I 'select' the desired onclick option?
Thanks in advance for your replies. Regards
Ps: For your information, I noted the following fact inspecting the Xetra element: it changes value when I move on to next page and if I come back the value is again different. Here an example: first time on page 1 I got
a onclick="d39081344_fkt_set_par('6');d39081344_fkt_set_active(this);" class="brs_d39081344_li current last"
, then I moved on to page 2 and I got
a onclick="d51109535_fkt_set_par('6');d51109535_fkt_set_active(this);" class="brs_d51109535_li current last" and coming back to page 1 I got a onclick="d96086211_fkt_set_par('6');d96086211_fkt_set_active(this);" class="brs_d96086211_li current last"
The trick is to look at what calls are made when you navigate through the pages. Your browser's network analysis tool is invaluable for this. When I go from page to page, a POST is made to 'http://www.boerse-frankfurt.de/en/parts/boxes/history/_tickdata_full.m with data about the request.
Then the goal is to replicate and loop the requests using python. Here is code to get you started:
import requests
r = requests.post('http://www.boerse-frankfurt.de/en/parts/boxes/history/_tickdata_full.m', data={'component_id':'PREKOP97077bf9dec39f14320bf9d40b636c7c589', 'page':"3", 'page_size':'50', 'boerse_id':'6', 'titel':'Tick-Data', 'lang':'en', 'text':'LOcbaec84ecad1b94ad2fd257897c87361', 'items_per_page':'50', 'template':'0', 'pages_total':'50', 'use_external_secu':'1', 'item_count':'2473', 'include_url':'/parts/boxes/history/_tickdata_full.m', 'ag':'291', 'secu':'291', })
print r.text #here is your data of interest, it still needs to be parsed
That is the general idea. You would then put that in a loop, adding one to the page parameter each time.
I am writing a gadget for Jira with some configuration options. One of these configuration options is a "project or filter picker".
My problem lies in the part, when I want to reconfigure the gadget's preferences. I have read the code of the timesince-gadget as an example and I think the relevant part is the following:
if (/^jql-/.test(gadget.getPref("projectOrFilterId"))){
projectAndFilterPicker =
{
userpref: "projectOrFilterId",
type: "hidden",
value: gadgets.util.unescapeString(this.getPref("projectOrFilterId"))
};
} else {
projectAndFilterPicker = AJS.gadget.fields.projectOrFilterPicker(gadget, "projectOrFilterId", args.options);
}
Basicly I've copied the code from the timesince-gadget. Unfortunately even if already configured, the javascript always enters the else part.
A problem is, that I ve no experience with jql and don't totally understand the if clause.
But usually (e.g. when calling the rest api and processing the config infos)
gadget.getPref("projectOrFilterId")
returns a string containing the id of the picked project or filter.
Question is now: How can I make my gadget remember the last configuration like it's done with some many other Jira gadgets?
I really hope anyone can help me with that.
It turnes out, the answer is even simplier then I thought.
First: In the descriptor you can totally forget the if part from above. Just
var projectAndFilterPicker = AJS.gadget.fields.projectOrFilterPicker(gadget, "projectOrFilterId", args.options);
is needed.
Second: Retrieve the project's or filter's name in your rest resource, which shouldn't be a problem, since you already want to use the processed id. Then return this name back to the view part of your javascript and type in something like
this.projectOrFilterName = args.myrestclasskey.projectOrFilterName;
And tada: reconfiguration will display the old configured name!
I had this problem once when I forgot to specify the option in the Gadget XML file. I solved it by adding this to the XML:
<UserPref name="projectOrFilterId" datatype="hidden"/>