How to scrape Javascript template in Htmlunit? - javascript

How to scrape 'handlerbars.js' javascript template in htmlunit 2.21? I try to execute this, but this message shown:
'text/x-javascript-template' is not javascript.
and script does not execute. I execute command page.asXml().
Output text is nothing. Do you know dependency for execute javascript template in Htmlunit? Please help.

This Code
HtmlUnit Config
webClient = new WebClient(BrowserVersion.CHROME);
webClient.getBrowserVersion().setBrowserLanguage("ko-kr");
ConfirmHandler okHandler = new ConfirmHandler() {
#Override
public boolean handleConfirm(Page page, String message) {
logger.info("[Confirm] " + message);
return true;
}
};
AlertHandler alertHandler = new AlertHandler() {
#Override
public void handleAlert(Page page, String message) {
logger.info("[Alert] " + message);
}
};
webClient.setRefreshHandler(new ImmediateRefreshHandler());
webClient.getOptions().setGeolocationEnabled(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getOptions().setUseInsecureSSL(true);
webClient.getOptions().setCssEnabled(false);
webClient.getOptions().setJavaScriptEnabled(script);
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setTimeout(0);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.setConfirmHandler(okHandler);
webClient.setAlertHandler(alertHandler);
page move code
page = webClient.getPage("https://promotions.expediapartnercentral.com/promotions/manageroomandrate.html?htid=" + siteIdx);
webClient.waitForBackgroundJavaScript(1000);
page.asXml() repsonse is empty(script is not execute)

Related

Passing value from java to Javascript

I'm making an Android app using webview.
The app can print out receipts. What I want to do is when the printer is not working, alert box shows up to tell the printer isn't working, and return false to the form's onsubmit event to prevent form from being submitted.
Java code:
public class JSKicker {
#JavascriptInterface
public void callPrint(final String argumet) {
Thread thread = new Thread(new Runnable() {
public void run() {
int nRtn;
connectionNum = myPrinter.Connect("000.000.0.000");
if(connectionNum < 0){ //Printer not working
webview.post(new Runnable() {
#Override
public void run() {
String script = "alert('Printer Error'); return printer_connection = false;";
webview.evaluateJavascript(script, new ValueCallback<String>() {
#Override
// I can't figure out what to do here...
});
}
});
}else{ //Printer is working properly
connectionNum = myPrinter.SetLocale(8);
strText = argument;
nRtn = myPrinter.PrintText(strText, "SJIS");
nRtn = myPrinter.PaperFeed(64);
nRtn = myPrinter.CutPaper(1);
myPrinter.Disconnect();
}
}
});
thread.start();
}
JavaScript in header:
<script type="text/javascript">
function gate(){
jQuery.ajax({
url:'/cart_info.php',
type:'GET'
})
.done( (data) => {
window.JSKicker.callPrint(data);
})
if (printer_connection = false) {
return false;
}else{
return true;
}
}
</script>
HTML form tag:
<form method="post" id="order_form" onsubmit="return gate();">
How can I get this work?
Could you do it thru WebView.evaluateJavascript()?
https://developer.android.com/reference/android/webkit/WebView.html#evaluateJavascript(java.lang.String,%20android.webkit.ValueCallback%3Cjava.lang.String%3E)
So with that you could send simple CustomEvent to document in WebView
webView.evaluateJavascript("document.dispatchEvent(new Event('printer_error', { details: "No printer found!" }));");
and in JavaScript you can hook listener for your custom event to react.
document.addEventListener('printer_error', e => alert(e.details));
Didn't test this so might be that at least evaluateJavascript() needs callback function.
WebSocket can solve your problem.
WebSockets provide a persistent connection between a client and server that both parties can use to start sending data at any time. The client establishes a WebSocket connection through a process known as the WebSocket handshake.
Its very straight forward and easy to implement.
You can follow referrer links for more details:-
JAVA WebSocket:- WebSocket using Spring Boot, WebSocket using Simple JEE
Browser WebSocket(JavaScript):- WebSocket API
In Android ,if you want webview pass value to JavaScript.
First,you need to set the webview enable the JavaScript,
private WebView mWebView;
void onCreate(){
mWebView = findViewById(R.id.webview);
WebSettings webSettings = mWebView.getSettings();
webSettings.setJavaScriptEnabled(true);
webSettings.setSupportZoom(false);
}
And ,in you want do some thing code
if(connectionNum < 0){ //Printer not working
// I need to do something here to send a message that the printer isn't working to JS.
//In thread can not use mWebView,should send message to MainThread to do
// mWebView.loadUrl("javascript:javaCall()");
Message msg = new Message();
msg.what = 1;
myHandler.sendMessage(msg);
//myHandler can be your MainThread send to here
}
And where the mWebView created in your code, be in main thread ,you can use the
Handler to deal with the message sended to here.
private Handler myHandler = new Handler() {
#Override
public void handleMessage(Message msg) {
// this 1 is just thread send to here
if (msg.what == 1) {
//and here can do the webview UI method
mWebView.loadUrl("javascript:javaCall()");
}
}
};
the javaCall() is where you JacaScript invoke method,in javaScript you can writre like this:
<script language="javascript">
function javaCall(){
alert("Printer Error");
//other thing you can do
}
</script>
if you have problem ,you can refer to the official document.
public void loadUrl (String url)
Loads the given URL.
Also see compatibility note on evaluateJavascript(String, ValueCallback).
webview use link

Clicking a button with HtmlUnit not working

I am trying to click a login button using HtmlUnit 2.23, using either DomElement or executeJavaScript but none of it works. Is there any other way to get it working? This is the code:
#Test
public void noLogin() throws Exception {
final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED);
//see http://stackoverflow.com/questions/3600557/turning-htmlunit-warnings-off/18563576#18563576
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
//see http://stackoverflow.com/questions/18556200/htmlunit-button-click
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
final HtmlPage loginPage = webClient.getPage("https://sandbox.paydirekt.de/account/#/login");
loginPage.getElementById("username").setAttribute("value", "no");
loginPage.getElementById("password").setAttribute("value", "login");
//final HtmlPage welcomePage = ((DomElement)loginPage.getFirstByXPath("//button[#type='submit']")).click();
//see http://stackoverflow.com/questions/14474580/use-htmlunit-doesnt-work-button-click
//ScriptResult result = loginPage.executeJavaScript("javascript:document.getElementsByName('loginForm')[0].submit();");
ScriptResult result = loginPage.executeJavaScript("javascript:document.getElementsByClassName('loginBox__loginButton button button--full')[0].click()");
Page page = result.getNewPage();
if (page.isHtmlPage()) {
HtmlPage welcomePage = ((HtmlPage)page);
System.out.println(welcomePage.asText());
Assert.assertTrue(welcomePage.asText().contains("Benutzername oder Passwort nicht korrekt."));
}
webClient.close();
}
You can also use the connection wrapper that HTMLUnit offers to check for each request and response manually. Here is an example
new WebConnectionWrapper(webClient) {
public WebResponse getResponse(WebRequest request) throws IOException {
WebResponse response = super.getResponse(request);
if (request.getUrl().toExternalForm().contains("THE URL YOU WANT TO CHECK")) {
String content = response.getContentAsString("UTF-8");
//CHECK CONTENT HERE
WebResponseData data = new WebResponseData(content.getBytes("UTF-8"), response.getStatusCode(),
response.getStatusMessage(), response.getResponseHeaders());
response = new WebResponse(data, request, response.getLoadTime());
}
return response;
}
};

Android - Error loading local js in application

I have an application which is basically a webview app. The website is based on angular js. Problem is the website takes a lot of time to start rendering. So I tried loading js from the asset folder. I used WebResourceResponse in my webviewclient
#Override
public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
if(url.startsWith("https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.4.1/angular.min.js")){
return loadFromAssets(url, "angular.min.js", "application/javascript", "");
}
return null;
}
private WebResourceResponse loadFromAssets( String url,
String assetPath, String mimeType, String encoding){
AssetManager assetManager = this.activity.getAssets();
InputStream input = null;
try {
Log.d(LOG_TAG, "Loading from assets: " + assetPath);
input = assetManager.open("/" + assetPath);
WebResourceResponse response =
new WebResourceResponse(mimeType, encoding, input);
return response;
} catch (IOException e) {
Log.e("WEB-APP", "Error loading " + assetPath + " from assets: " +
e.getMessage(), e);
}
return null;
}
But I keep getting the filenotfound exception
07-26 17:19:21.662 20883-21536/com.exun.brinjal D/﹕ Loading from assets: angular.min.js
07-26 17:19:21.663 20883-21536/com.exun.brinjal E/WEB-APP﹕ Error loading angular.min.js from assets: /angular.min.js
java.io.FileNotFoundException: /angular.min.js
at android.content.res.AssetManager.openAsset(Native Method)
at android.content.res.AssetManager.open(AssetManager.java:335)
at android.content.res.AssetManager.open(AssetManager.java:309)
at com.exun.brinjal.HelloWebViewClient.loadFromAssets(HelloWebViewClient.java:85)
at com.exun.brinjal.HelloWebViewClient.shouldInterceptRequest(HelloWebViewClient.java:37)
at com.android.webview.chromium.WebViewContentsClientAdapter.shouldInterceptRequest(WebViewContentsClientAdapter.java:283)
at com.android.org.chromium.android_webview.AwContents$IoThreadClientImpl.shouldInterceptRequest(AwContents.java:244)
at dalvik.system.NativeStart.run(Native Method)
07-26 17:19:21.666 20883-21536/com.exun.brinjal D/﹕ Loading from assets: angular-route.min.js
07-26 17:19:21.669 20883-21536/com.exun.brinjal E/WEB-APP﹕ Error loading angular-route.min.js from assets: /angular-route.min.js
java.io.FileNotFoundException: /angular-route.min.js
at android.content.res.AssetManager.openAsset(Native Method)
at android.content.res.AssetManager.open(AssetManager.java:335)
at android.content.res.AssetManager.open(AssetManager.java:309)
at com.exun.brinjal.HelloWebViewClient.loadFromAssets(HelloWebViewClient.java:85)
at com.exun.brinjal.HelloWebViewClient.shouldInterceptRequest(HelloWebViewClient.java:40)
at com.android.webview.chromium.WebViewContentsClientAdapter.shouldInterceptRequest(WebViewContentsClientAdapter.java:283)
at com.android.org.chromium.android_webview.AwContents$IoThreadClientImpl.shouldInterceptRequest(AwContents.java:244)
at dalvik.system.NativeStart.run(Native Method)
I have tried to change assetManager.open("/" + assetPath); to file:/// and /assets but it doesn't help. All the js and css files I need are in the assets folder. I also want to know how will I get to load all those files also.
After some brainstorming, the answer was to remove "/" from assetManager.open("/" + assetPath);
So finally just put assetManager.open(assetPath); in the code and voila!

How to load a javascript redirect into android webview?

I am trying to load a page into webview. Normally, when i call
webview.loadUrl(url);
it works fine. The url has a javascript code that redirects the page. Here is the jacascript code:
<script type="text/javascript">
if (!document.cookie || document.cookie.indexOf('AVPDCAP=') == -1)
{ document.write('<scr'+'ipt src="http://some_link_here+Math.floor(89999999*Math.random()+10000000)+'&millis='+new Date().getTime()+'&referrer='+encodeURIComponent(document.location)+'" type="text/javascript"></scr'+'ipt>'); }
</script>
When i try to load that javascript code into my webview like the following:
String data = javascript_code_above;
topBannerWV.loadDataWithBaseURL("", data, "text/html", "UTF-8", null);
where data is this javascript code, it does not load the page. I also tried following:
topBannerWV.loadDataWithBaseURL("", data, "text/javascript", "UTF-8", null);
but this time that text is loaded into webview. Can anyone help me with this?
Thanks.
I am doing something like this and it works perfectly:
webView.loadUrl("file:///android_asset/html.html");
webView.setWebViewClient(new WebViewClient() {
public void onPageFinished(WebView view, String url) {
try {
webView.loadUrl("javascript:init('" + some parameter + "')");
} catch (JSONException e) {
e.printStackTrace();
}
}
});
script in html file is:
<script type="text/javascript">
function init(val){
document.getElementById('content').innerHTML=val;
}
window.addEventListener('load',function(){
console.log("Window Loader");
init();
});
</script>
I found the solution.
Instead of directly giving the string to webview to load, i first wrote it into a file and then gave that file to webview to load.
getContext().getApplicationContext().deleteFile("data.html");
Writer output = null;
File dir = getContext().getApplicationContext().getFilesDir();
File file = new File(dir.getAbsolutePath() + File.separator + "data.html");
file.createNewFile();
output = new BufferedWriter(new FileWriter(file));
output.write(WVdata);
output.close();
topBannerWV.loadUrl("file:///" + getContext().getApplicationContext().getFilesDir() + "/data.html");
I do not know why this happens, actually i do not change the data string, just put it into a file.

android webview javascript not working with loadDataWithBaseUrl

I am trying to load data into android webview using
webview.loadDataWithBaseURL("", htmlcontent, "text/html", null, "");
a method returns htmlContent from a StringBuilder which populates html data.
I have enabled javascript and set webChromeClient as follows
webview.getSettings().setJavaScriptEnabled(true);
webview.setWebChromeClient(new WebChromeClient());
webview.addJavascriptInterface(new JSClass(), "Android");
my interface to javascript:
class JSClass {
public void getHTMLContent(String html)
{
Log.i(Global.TAG, "HTMLContentReceived: "+html);
}
}
and my javascript in html page:
<script type="text/javascript">
var ele = document.getElementsByClassName('test');
for(var i=0;i<ele.length;i++){
ele[i].onclick = function(){
window.Android.getHTMLContent(this.innerHTML);
}
}
</script>
but somehow the javascript is not returning any value.
It works fine with loadData(url) where url is a simple webpage in assets folder
Please help
Thanks in advance
You don't have any baseURL to use, since you're loading a dynamical generated HTML.
For this reason webview.loadData(htmlcontent, "text/html", null); should be more than enough.
Javascripts don't throw any exceptions in Java code. Remember that JS is not that type-safe/strict as Java code ... My way of doing is to put logs between sensitive Javascript calls to see if that line passes and to check values. Since you didn't provide the HTML, I would setup the WebChomeClient and override the onConsoleMessage:
webview.setWebChromeClient(new MyChromeClient());
private class MyChromeClient extends WebChromeClient {
#Override
public boolean onConsoleMessage(ConsoleMessage consoleMessage) {
String message = consoleMessage.message() + " -- line " + consoleMessage.lineNumber();
switch (consoleMessage.messageLevel()) {
case ERROR:
logErrorMessage(message);
break;
default:
logInfoMessage(message);
break;
}
return true;
}
private void logInfoMessage(String message) {
Log.i("JSTag", message);
}
private void logErrorMessage(String message) {
Log.e("JSTag", message);
}
}
From your JavaScript you would then call for example: console.log('check my value:' + (ele != null)). More info on this here.
Looking at your JavaScript code, I can't understand to what points this.innerHTML.

Categories