I'm trying to build a web crawler based on the requirements that were described here, and I figured WebView would be the most suitable way to implement this.
The problem seems to emerge when the next URL I need to visit is based on the HTML contents of the current page.
I am using view.evaluateJavascript to get the current page HTML and parse the URL part inside onReceiveValue, but then there is no way for me to navigate to the URL because onReceiveValue cannot access the view.
Also, using loadUrl in onPageFinished does not work as well, because it is done even before I retrieve the HTML content, thus navigating to the page with a null value
WebView myWebView = new WebView(this);
setContentView(myWebView);
myWebView.getSettings().setJavaScriptEnabled(true);
MyJavaScriptInterface jInterface = new MyJavaScriptInterface(this);
myWebView.addJavascriptInterface(jInterface, "HTMLOUT");
myWebView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
if (url.equals("http://url.com")) {
final String[] versionString = {
null
};
view.evaluateJavascript("(function(){return window.document.body.outerHTML})();",
new ValueCallback < String > () {
#Override
public void onReceiveValue(String html) {
String result = removeUTFCharacters(html).toString();
Matcher m = r.matcher(result);
versionString[0] = m.group(1);
}
});
String getFullUrl = String.format("https://url.com/getData?v=%s", versionString[0]);
view.loadUrl(getFullUrl);
}
}
});
myWebView.loadUrl("http://url.com");
Call your url from onReceiveValue
myWebView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
if (url.contains("https://www.google.com")) {
final String[] versionString = {
null
};
view.evaluateJavascript("(function(){return window.document.body.outerHTML})();",
new ValueCallback< String >() {
#Override
public void onReceiveValue(String html) {
String getFullUrl = String.format("https://cchat.in", versionString[0]);
view.loadUrl(getFullUrl);
}
});
}
}
});
myWebView.loadUrl("https://www.google.com");
I used 2 website to demonstrate. I am able to call 2nd URL from onReceiveValue.
You can try this.
Related
I'm loading data from website to a webview in my application.And i need to make the video plays inside the webview.my problem is when i don't use setJavaScriptAsEnable the images and the video loads just fine but i can't play the video inside the webview it gives me the following error
An error occurred. try watching this video on youtube or enabe JavaScript.
however when i use setJavaScriptAsEnable(true) all the images and videos inside the webview disappears like the following image.
here's my code for the webview
void setupWebView(final WebView webView) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
webView.getSettings().setLayoutAlgorithm(WebSettings.LayoutAlgorithm.TEXT_AUTOSIZING);
} else {
webView.getSettings().setLayoutAlgorithm(WebSettings.LayoutAlgorithm.NORMAL);
}
webView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
// add padding to the html page
webView.loadUrl("javascript:(function(){ document.body.style.paddingBottom = '30px'})();");
}
#Override
public boolean shouldOverrideUrlLoading(WebView view, String request) {
if (request.contains("http")) {// TODO: 14/08/2017 use regs
Log.d("url", request);
new CheckIfImageAsyncTask().execute(request);
return true;
}
return false;
}
#RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
#Override
public boolean shouldOverrideUrlLoading(WebView view, WebResourceRequest request) {
String url = request.getUrl().toString();
if (url.contains("http")) {
new CheckIfImageAsyncTask().execute(url);
return true;
}
return false;
}
});
// settings
webView.setWebChromeClient(new WebChromeClient());
webView.getSettings().setPluginState(WebSettings.PluginState.ON);
webView.getSettings().setPluginState(WebSettings.PluginState.ON_DEMAND);
webView.getSettings().setAllowFileAccess(true);
webView.getSettings().setJavaScriptEnabled(true);
webView.getSettings().setDomStorageEnabled(true);
String data = "<div>" + post.getContent() + " </div>";
webView.loadDataWithBaseURL("file:///android_asset/", getHtmlData(data), "text/html", "utf-8", null);
}
private class CheckIfImageAsyncTask extends AsyncTask<String, Void, Boolean> {
private String url;
#Override
protected Boolean doInBackground(String... strings) {
url = strings[0];
URLConnection connection = null;
boolean isImage = false;
try {
connection = new URL(url).openConnection();
String contentType = connection.getHeaderField("Content-Type");
isImage = contentType.startsWith("image/");
} catch (IOException e) {
e.printStackTrace();
}
return isImage;
}
#Override
protected void onPostExecute(Boolean isImage) {
try {
if (isImage) {
Intent intent = new Intent(PostDetailsActivity.this, FullScreenImageActivity.class);
intent.putExtra("coverImage", url);
startActivity(intent);
} else {
Intent intent = new Intent(Intent.ACTION_VIEW, Uri.parse(url));
startActivity(intent);
}
} catch (Exception e) {
Log.e("Error", e.getMessage());
FirebaseCrash.report(new Exception(e.getMessage()));
}
}
}
i searched a lot and i can't figure it out. any help will be appreciated. thanks in advance.
I have a code here where I want to retrieve an element from the webview of android and I can't seems to figure out what's wrong. It might be some silly mistakes but I just can't see why and what's wrong. Thanks.
private void retrieveContentWebView(String address){
Log.d("Web Download"," Inside ");
Log.d("Web Download ","address : " + address);
WebView webView;
webView = new WebView(context);
webView.loadUrl(address);
webView.getSettings().setJavaScriptEnabled(true);
webView.addJavascriptInterface(this, "myjava");
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP)
webView.getSettings().setMixedContentMode(WebSettings.MIXED_CONTENT_ALWAYS_ALLOW);
webView.setScrollBarStyle(View.SCROLLBARS_INSIDE_OVERLAY);
webView.getSettings().setDomStorageEnabled(true);
webView.setWebViewClient(new WebViewClient() {
#Override
public void onPageStarted(WebView view, String url, Bitmap favicon) {
super.onPageStarted(view, url, favicon);
Log.d("Web Download"," Inside Page Started");
}
#Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
Log.d("Web Download"," Inside Page Finished");
view.loadUrl("javascript: {" +
"var titleTexts = document.getElementsByTagName('title')[0]" +
"myjava.onData(titleTexts.text);}");
}
});
}
//This method will be registered as a JavaScript interface
#JavascriptInterface
#SuppressWarnings("unused")
public void onData(final String value){
Log.d("Web Download"," Inside Handler");
final String getValue = value;
final Handler handler = new Handler();
handler.postDelayed(new Runnable() {
#Override
public void run() {
// Do something after 5s = 5000ms
Log.d("Web Download HTML :",getValue);
}
}, 5000);
}
Try adding this to your WebView:
webView.setWebChromeClient(new MyCustomChromeClient(this));
Create the MyCustomChromeClient for yourself.
I am using authorization headers to access the web page, but when using WebViewClient with authorization headers the webview not rendering the css and also the js not loading.
public class TableViewTest extends AppCompatActivity {
WebView webView;
SharedPreferences pref;
#SuppressLint("SetJavaScriptEnabled")
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_table_view_test);
String url = "http://myurl";
pref = getSharedPreferences("app", Context.MODE_PRIVATE);
webView = (WebView) findViewById(R.id.webView1Id);
webView.setWebViewClient(wvc);
//webView.setVerticalScrollBarEnabled(true);
//webView.setHorizontalScrollBarEnabled(true);
webView.getSettings().setJavaScriptEnabled(true);
webView.getSettings().setBuiltInZoomControls(true);
webView.getSettings().setDomStorageEnabled(true);
webView.getSettings().setUseWideViewPort(true);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
webView.getSettings().setMixedContentMode(WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE);
webView.getSettings().setMixedContentMode(WebSettings.MIXED_CONTENT_ALWAYS_ALLOW);
}
//webView.setLayerType(View.LAYER_TYPE_HARDWARE, null);
webView.loadUrl("http://myurl");
}
#Override
protected void onSaveInstanceState(Bundle outState) {
super.onSaveInstanceState(outState);
webView.saveState(outState);
}
#Override
protected void onRestoreInstanceState(Bundle savedInstanceState) {
super.onRestoreInstanceState(savedInstanceState);
webView.restoreState(savedInstanceState);
}
WebViewClient wvc = new WebViewClient() {
#SuppressWarnings("deprecation")
public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
try {
final String acToken = pref.getString("token", "DEFAULT");
OkHttpClient okHttpClient = new OkHttpClient();
Request request = new Request.Builder().url(url).addHeader("Authorization", "Bearer " + acToken)
.build();
Response response = okHttpClient.newCall(request).execute();
return new WebResourceResponse(response.header("text/html", response.body().contentType().type()), // You can set something other as default content-type
response.header("content-encoding", "utf-8"), // Again, you can set another encoding as default
response.body().byteStream());
} catch (ClientProtocolException e) {
//return null to tell WebView we failed to fetch it WebView should try again.
return null;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
};
}
Please help me. Any help is appreciated.
Used AdvancedWebView library to overcome the issues. The default webview in android can not handle the css issues sometimes.
And also both Android webview and AdvancedWebView Can not handle the headers except in initial request in the webview. It can not pass the headers across all the links in particular webview request. Please refer this to rectify your doubts Github Question.
My SO Question for the headers issue.
If anybody finds the solution for the above header passing issue, feel free to add your solution. Thank you.
This works for me.
(btw is deprecated now)
WebViewClient wvc = new WebViewClient() {
#Override
public WebResourceResponse shouldInterceptRequest (WebView view, String url){
try {
OkHttpClient httpClient = new OkHttpClient();
com.squareup.okhttp.Request request = new com.squareup.okhttp.Request.Builder()
.url(url.trim())
.addHeader("PIPPO", "PIPPO")
.build();
com.squareup.okhttp.Response response = httpClient.newCall(request).execute();
Log.d("TAG",response.header("content-type", "text/html"));
Log.d("TAG",response.header("content-encoding", "UTF-8"));
return new WebResourceResponse(
null,
null,
response.body().byteStream()
);
} catch (Exception e) {
//return null to tell WebView we failed to fetch it WebView should try again.
return null;
}
}
My only fear is that on smartphones with "problematic" default browser, the result may be broken.
I'm trying to fill Webforms from a Webview in Android.
I've already found this piece of code here: Fill fields in webview automatically
String username = "cristian";
webview.loadUrl("javascript:document.getElementById('username').value = '"+username+"';");
Unfortunatly I dont understand where I have to open the page I want to fill in.
setContentView(R.layout.web);
final WebView mWebView = (WebView) findViewById(R.id.webview);
mWebView.getSettings().setJavaScriptEnabled(true);
mWebView.loadUrl(url);
String user="u";
String pwd="p";
mWebView.loadUrl("javascript:document.getElementById('username').value = '"+user+"';document.getElementById('password').value='"+pwd+"';");
When I try it this way, the site gets displayed but without any values in the forms.
Thanks in advance for helping
Android 4.4 new WebView have the issue with use loadUrl("javascript:") method, the paramter string will be url-decode before execution.
You can try use evaluateJavascript() for API >= 19 and loadUrl() for API < 19.
The code below is work for me.
mWebView.loadUrl(url)
WebSettings settings = mWebView.getSettings();
settings.setJavaScriptEnabled(true);
String js = "javascript:document.getElementById('username').value = '"+user+"';document.getElementById('password').value='"+pwd+"';";
mWebView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
if (Build.VERSION.SDK_INT >= 19) {
view.evaluateJavascript(js, new ValueCallback<String>() {
#Override
public void onReceiveValue(String s) {
}
});
} else {
view.loadUrl(js);
}
});
}
please refer: loadUrl("javascript:....") behaviour changed incompatibly in Android 4.4
You should fill the values after the page has been loaded. This is an example using your code:
mWebView.loadUrl(url);
mWebView.getSettings().setJavaScriptEnabled(true);
mWebView.setWebViewClient(new WebViewClient() {
public void onPageFinished(WebView view, String url) {
String user="u";
String pwd="p";
view.loadUrl("javascript:document.getElementById('username').value = '"+user+"';document.getElementById('password').value='"+pwd+"';");
}
});
Just Enable DomStorage and write "var x=" to string:
webview.getSettings().setJavaScriptEnabled(true);
web.getSettings().setDomStorageEnabled(true);
webview.loadUrl(urlString);
webview.setWebViewClient(new WebViewClient(){
public void onPageFinished(WebView view, String url){
super.onPageFinished(view, url);
String js = "javascript:var x=document.getElementById('username').value = '"+user+"';var y=document.getElementById('password').value='"+pass+"';";
if (Build.VERSION.SDK_INT >= 19) {
view.evaluateJavascript(js, new ValueCallback<String>() {
#Override
public void onReceiveValue(String s) {
}
});
} else {
view.loadUrl(js);
}
view.loadUrl(js);
}
});
I had the same issue, and after following different explanations on stackOverflow, I succeed to make mine working.
here is my approach
webView.loadUrl(url);
webView.getSettings().setDomStorageEnabled(true);
webView.setWebViewClient(new WebViewClient(){
public void onPageFinished(WebView view, String url) {
String email="email#email.jp";
//view.loadUrl("javascript:document.getElementById('email').value = '"+email+"'");
view.loadUrl("javascript:document.forms[0].email.value = '"+email+"';");
Log.d("email", "can not add email");
}
});
2 things :
1) you need to add this line webView.getSettings().setDomStorageEnabled(true); (reference: Android WebView always returns null for javascript getElementById on loadUrl)
2) you need to access the variable in your php code by using this view.loadUrl("javascript:document.forms[0].email.value = '"+email+"';");
as you can see in my code I used the solution proposed by #gnpaolo, but it didn't work for me so I commented it and use this one. (reference: How to inject a String into Android WebView)
Finally, just want to add that you do not need to create a special javascript.
one more thing the forms[0] is the position of the variable in the php form, and in my case, I have the email of the user, so I wrote view.loadUrl("javascript:document.forms[0].email.value = '"+email+"';");
Hope, this can help others.
Short question that i cannot get my finger on, i'd like to launch a new browser from my WebView after people click a hyperlink. But how can i set that target of that link to escape the WebViewClient??
Here's my code, any help is greatly appreciated:
WebView site = (WebView)findViewById(R.id.WebView);
site.setWebViewClient(new WebViewClient());
site.setScrollBarStyle(View.SCROLLBARS_INSIDE_OVERLAY);
site.getSettings().setJavaScriptEnabled(true);
button1.setOnClickListener(button1OnClickListener);
button2.setOnClickListener(button2OnClickListener);
button3.setOnClickListener(button3OnClickListener);
button4.setOnClickListener(button4OnClickListener);
final AlertDialog alertDialog = new AlertDialog.Builder(this).create();
progressBar = ProgressDialog.show(FlitsersActivity.this, "Thingy1", "Load...", false, true);
site.setWebViewClient(new WebViewClient() {
public boolean shouldOverrideUrlLoading(WebView view, String url) {
Log.i(TAG, "Loading...");
view.loadUrl(url);
return true;
}
public void onPageFinished(WebView view, String url) {
Log.i(TAG, "Done: " +url);
if (progressBar.isShowing()) {
progressBar.dismiss();
}
}
public void onReceivedError(WebView view, int errorCode, String description, String failingUrl) {
Log.e(TAG, "Error: " + description);
Toast.makeText(MyActivity.this, "Oh no! " + description, Toast.LENGTH_SHORT).show();
alertDialog.setTitle("Error");
alertDialog.setMessage(description);
alertDialog.setButton("OK", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int which) {
return;
}
});
alertDialog.show();
}
});
site.loadUrl("http://www.etcetera.com");
}
Exactly, thanks! I think i found out around the time you wrote the second response ;)
Right now i am using the code below: it works like a charm :) Thank you very much for responding!
site.setWebViewClient(new WebViewClient() {
public boolean shouldOverrideUrlLoading(WebView view, String url) {
String myAlternativeURL = "http://newURL";
if (!url.equals(myAlternativeURL)) {
{
Intent i = new Intent(Intent.ACTION_VIEW);
i.setData(Uri.parse("http://newURL"));
startActivity(i);
}
return true;
} else {
Log.i(TAG, "Loading...");
view.loadUrl(url);
return true;
}
}
You are overriding the loading of all URLs by forcing the WebView to load them with this code:
public boolean shouldOverrideUrlLoading(WebView view, String url) {
Log.i(TAG, "Loading...");
view.loadUrl(url);
return true;
}
What you need to do is create a browser intent with the URL if you want it to open in the default browser instead of your WebView.
http://developer.android.com/guide/appendix/g-app-intents.html
The code would be something like this:
Intent i = new Intent(Intent.ACTION_VIEW);
i.setData(Uri.parse(url));
startActivity(i);
More info:
if (url.equals("http://www.etcetera.com")
// load url normaly
else
// Load in new window via intent
Static Problem
So I found out, that in my case, I have to add a Context before startActivity()
context.startActivity(browserIntent);
From: https://stackoverflow.com/a/6651535/13413319
In order to do that:
I have this variable:
Context context;
and of cource assigning its value by Constructor
Context = context;
After that calling the methode, wenn you need to perform the task.
openInBrowser(url);
methode:
public static void openInBrowser(String url) {
Intent browserIntent = new Intent(Intent.ACTION_VIEW);
browserIntent.setData(Uri.parse(url));
context.startActivity(browserIntent);