We Keep Coding

htmlUnit after longer Javascript-Check of Bandwidth the Text-Input is not updated

On this page: https://www.check24.de/dsl/vergleich/
I tried to implement a crawler for bandwidth checks of different providers via httpUnit, version 2.31.
If you manually fill in the Address field on the page, you will see a pop-up that shows the progress of the bandwidth check, then you have the available Internet bandwidths for the requested address on the same page.
The requested address is in labels (the place where first the text input fields were.
In my attempt to write a crawler using htmlUnit, although I returned the same page after (a longer waiting time) the input field was not replaced by some label in a fieldset (id="tko-vcheck-done-wrapper") that show the address.
Here is my Code:
public Map<String, Integer> checkProviderBandWidthsByAddress(String zip, String city, String street, String hno){
WebClient webClient = null;
try{
webClient = getWebCient();
HtmlPage page = webClient.getPage("https://www.check24.de/dsl/vergleich/");
HtmlTextInput inputZipCity = (HtmlTextInput) page.getElementById("c24api_ac_widget_zipcity");
HtmlHiddenInput inputZip = (HtmlHiddenInput) page.getElementById("c24api_ac_widget_zipcode");
HtmlHiddenInput inputCity = (HtmlHiddenInput) page.getElementById("c24api_ac_widget_city");
HtmlTextInput inputStreet = (HtmlTextInput) page.getElementById("c24api_ac_widget_street");
HtmlTextInput inputStreetNumber = (HtmlTextInput) page.getElementById("c24api_ac_widget_streetnumber");
HtmlButton buttonCheck = (HtmlButton) page.getElementById("tko-filter-vcheck-submit");
inputZipCity.setValueAttribute(zip + " " + city);
inputZipCity.fireEvent(Event.TYPE_INPUT);
page.getWebClient().waitForBackgroundJavaScriptStartingBefore(1000);
inputZip.setValueAttribute(zip);
inputCity.setValueAttribute(city);
inputStreet.setValueAttribute(street);
inputStreetNumber.setValueAttribute(hno);
page = buttonCheck.click();
page.getWebClient().waitForBackgroundJavaScriptStartingBefore(30000);
DomElement done = page.getElementById("tko-vcheck-done-wrapper"); // <-- Probleme here: NULL
List<DomElement> providers = page.getByXPath("//div[contains(#class, 'tko-result-row tko-clearfix')]");
Map<String, Integer> bandWidths = findMaxSpeed(providers); // works fine to read the download BandWith for general tarif - but this dont contains the address-specific bandwith
return bandWidths;
}catch(Exception e){
e.printStackTrace();
return Collections.emptyMap();
}finally {
webClient.close();
}
}
public static WebClient getWebCient(){
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_52); // also tried with Other
webClient.setRefreshHandler(new WaitingRefreshHandler());
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setCssEnabled(false);
webClient.setCssErrorHandler(new SilentCssErrorHandler());
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setUseInsecureSSL(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getCookieManager().setCookiesEnabled(true);
webClient.getOptions().setPopupBlockerEnabled(false);
return webClient;
}
If someone has an idea to solve the problem, I would be very happy about it

Pages like this horrible monster are a challenge for HtmlUnit.
But if you are a bit patient, then it will work.
(I'm using HtmlUnit release 2.32)
Have added some comments to the sample code; hope that helps.
And please take the code as proof of concept, there was not enough time for writing good code.
public static void main(String[] args) throws Exception {
String url = "https://www.check24.de/dsl/vergleich/";
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_60)) {
HtmlPage page = webClient.getPage(url);
// this page has starts a lot of javascript
// we have to wait until this is finished to get a page
// that can respond to our typing
wait(webClient, 60);
HtmlTextInput inputZipCity = (HtmlTextInput) page.getElementById("c24api_ac_widget_zipcity");
inputZipCity.type("50126");
wait(webClient, 30);
// System.out.println(page.getElementById("tko-result-filter-form-acsuggest").asXml());
HtmlTextInput inputStreet = (HtmlTextInput) page.getElementById("c24api_ac_widget_street");
HtmlTextInput inputStreetNumber = (HtmlTextInput) page.getElementById("c24api_ac_widget_streetnumber");
inputStreet.type("Hauptstr.");
wait(webClient, 10);
inputStreetNumber.type("10");
wait(webClient, 10);
HtmlButton buttonCheck = (HtmlButton) page.getElementById("tko-filter-vcheck-submit");
buttonCheck.click();
wait(webClient, 4 * 60);
HtmlPage refreshedPage = ((HtmlPage) page.getEnclosingWindow().getEnclosedPage());
// System.out.println("----------------");
// System.out.println(refreshedPage.asText());
System.out.println(refreshedPage.getElementById("tko-result-sorting-text").getTextContent());
}
}
private static void wait(WebClient webClient, int seconds) {
long timeLimit = System.currentTimeMillis() + seconds * 1000;
int scriptCount = webClient.waitForBackgroundJavaScript(1000);
while (scriptCount > 1 && timeLimit > System.currentTimeMillis()) {
scriptCount = webClient.waitForBackgroundJavaScript(1000);
}
// seems like there is always one job in the queue (maybe some kind of heartbeat)
if (scriptCount > 1) {
System.out.println("Still some js is running " + scriptCount);
}
}
At least this produces something like
68 Tarife verfügbar von 12,91 € bis 107,47 € (Durchschnitt pro Monat)
Same text is shown on the web site when running with a real browser.

How to upload photo from library in webview in android app

I am using webview component on my android app. Users can load images from android photo library and show these images on a web page in the webview. How can I upload these image to my backend server from javascript?
Below is my java code to handle image chooser behavior:
setWebChromeClient(new WebChromeClient() {
#Override
public boolean onShowFileChooser(WebView webView, ValueCallback<Uri[]> filePathCallback, FileChooserParams fileChooserParams) {
Intent intent = new Intent(Intent.ACTION_PICK);
intent.setType("image/*");
intent.setAction(Intent.ACTION_GET_CONTENT);
Intent chooser = Intent.createChooser(intent, "Select Image");
activity.startActivityForResult(chooser, ResultCode.CHOOSE_PHOTO_REQUEST);
return false;
}
});
the above code will show image picker and when a user select an image, the onActivityResult will pass the selected image path to javascript as below:
if (resultCode == Activity.RESULT_OK) {
Uri imageUri = imageReturnedIntent.getData();
Log.d("IMAGE", "choose image uri " + imageUri);
String path = getRealPathFromURI(imageUri);
Log.d("IMAGE", "choose image real path " + path);
webView.loadUrl("javascript:choosePhotos('" + path + "')");
}
public String getRealPathFromURI(Uri contentUri) {
Cursor cursor = null;
try {
String[] proj = {MediaStore.Images.Media.DATA};
cursor = mainActivity.getContentResolver().query(contentUri, proj, null, null, null);
int column_index = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATA);
cursor.moveToFirst();
return cursor.getString(column_index);
} finally {
if (cursor != null) {
cursor.close();
}
}
}
in javascript, I can put the image path on a <img src=''/> tag to show the selected images. The path is something like this: '/storage/emulated/0/DCIM/Camera/IMG_20160808_200837.jpg'
It works fine here. But now I want to upload this image to my backend server. How can javascript handle this path: /storage/emulated/0/DCIM/Camera/IMG_20160808_200837.jpg.

You Can use filePathCallback to pass the selected file to webview.
Just create a global variable
ValueCallback filePathCallback;
and assign the parameter from onShowFileChooser() method to it.
then you can use this callback in onActivityResult() to pass the selected file to webview as :
Uri results[] = new Uri[]{imageUri};
filePathCallback.onReceiveValue(results);
then on html you will get file at

amcharts not showing theme background in Android webview

I am using a webview to host amcharts in an android application. The basic setup for the code is that I form an HTML document programatically and load it hosted in my assets folder. The amcharts libraries are located in a subdirectory of the assets folder. So far, this setup has allowed me to generate charts in a webview with locally provided data. In order to change the chart's theme, I have been following a tutorial from the amcharts websight http://www.amcharts.com/tutorials/working-with-themes/. The tutorial allowed me to change the way the bars in the chart look, but the background remains unchanged. I have also looked at some posts reporting that the themes of other users were not working. My problem differs in the sense that the way the graph is being drawn is correct according to the theme. It is only the background that is not being set.
This is the class I use to build the test chart data.
public class AmChartsUtils {
public static String getTestHtml(String chartData) {
chartData = chartData.replace("\n", "");
return "<HTML><HEAD>" + getHeadContent(chartData)+ "<BODY>"+ getBodyContent() + "</BODY></HTML>";
}
private static String getHeadContent(String chartData) {
return "<script src=\"js/amcharts.js\" type=\"text/javascript\"></script>"+
"<script src=\"js/serial.js\" type=\"text/javascript\"></script>" +
"<script src=\"js/themes/dark.js\" type=\"text/javascript\"></script>" +
"<script src=\"js/themes/chalk.js\" type=\"text/javascript\"></script>" +
// "<link rel=\"stylesheet\" href=\"style.css\" type=\"text/css\">"
getChartData(chartData);
}
private static String getChartData(String chartData) {
StringBuilder sb = new StringBuilder();
sb.append("<script type=\"text/javascript\">");
sb.append("var chartData = [");
sb.append(chartData);
sb.append("];");
sb.append(buildStartCommand());
sb.append("</script>");
return sb.toString();
}
private static String buildStartCommand() {
StringBuilder sb = new StringBuilder();
sb.append("if(document.body){");
sb.append("document.body.style.backgroundColor = \"#282828\";");
// sb.append("document.body.style.backgroundImage = \"url(\" + bgImage + \")\";");
sb.append("}");
sb.append("AmCharts.ready(function() {");
sb.append("var chart = new AmCharts.AmSerialChart(AmCharts.themes.chalk);");
sb.append("chart.dataProvider = chartData;");
sb.append("chart.categoryField = \"country\";");
sb.append("chart.angle = 30;");
sb.append("chart.depth3D = 15;");
sb.append("chart.backgroundColor = \"#282828\";");
sb.append("chart.backgroundAloha = \"1.0\";");
sb.append("var graph = new AmCharts.AmGraph();");
sb.append("graph.valueField = \"visits\";");
sb.append("graph.type = \"column\";");
sb.append("chart.addGraph(graph);");
sb.append("chart.write('chartdiv');");
sb.append("});");
return sb.toString();
}
private static String getBodyContent() {
return "<div id=\"chartdiv\" style=\"width: 400px; height: 400px;\"></div>";
}
}
Here is how the chart apears on the screen.
The expected result is the same graph with a dark chalkboard background.
Does anyone know why the background is not coming in? Thanks in advance for any help recieved.

There is a typo in your code:
sb.append("chart.backgroundAloha = \"1.0\";");
It should read this instead:
sb.append("chart.backgroundAlpha = \"1.0\";");
Also, alpha is a numeric parameter. I strongly suggest you supply it as number.
I know you won't be displaying it in some old browsers that might be thrown off by this, but it's a good idea to keep your JavaScript apps as strongly typed as possible.

How to access image file from HTML or Javascript for Windows Phone

I have a requirement in wp8, where the picture selected by the user needs to be shown in the browser. To browse and select the photo, I am using photo chooser task.
I am able to get the physical location of the selected image, but on passing the same to JavaScript from c# its not displaying the image.
On googling came across the following link How to access isolated storage file from HTML or Javascript for Windows Phone and PhoneGap Application But it did not solve my issue.
For reference, the location of the image I am using was:
C:\Data\Users\DefApps\AppData{FA586990-6E21-0130-BF9E-3C075409010C}\Local\sample_photo_00.jpg
This is my Javascript code:
function myPicture(data) {
document.getElementById("capturedImage").src = data.imageUri;
alert("data.imageUri " + document.getElementById("capturedImage").src );
var width = data.imageWidth;
var height = data.imageHeight;
alert("image width" + width );
alert("image height" + height );
}
And this is my C# code:
StorageFolder localFolder = ApplicationData.Current.LocalFolder;
string[] picList = Directory.GetFiles(localFolder.Path, "*.jpg");
foreach (string DeleteFile in picList) {
File.Delete(DeleteFile);
}
StorageFile storageFile = await localFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
using (Stream outputStream = await storageFile.OpenStreamForWriteAsync())
{
await file.CopyToAsync(outputStream);
}
send (storageFile.Path);
Now send function should add MyHTML in picture.

You can call JavaScript function from C# by WebBrowser.InvokeScript and send image in args parameter. But args is string(s), so you will have to encode your image to string using some algorithm... Base64 for example:
string ImageToBase64String(Image image)
{
using (MemoryStream stream = new MemoryStream())
{
image.Save(stream, image.RawFormat);
return Convert.ToBase64String(stream.ToArray());
}
}
You will get some long string like this iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==
On other side - in the JavaScript function you calling you will get that Base64 string and use it like this as src attribute of img element:
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==" />
More info about data uri scheme.
UPDATE: The easer solution. I think you can send your image path, width and height:
ImageProperties properties = await storageFile.Properties.GetImagePropertiesAsync();
webBrowser.InvokeScript("myPicture", storageFile.Path, (string)properties.Width, (string)properties.Height);
function myPicture(src, width, height) {
document.getElementById("capturedImage").src = src;
alert("data.imageUri " + document.getElementById("capturedImage").src );
alert("image width" + width );
alert("image height" + height );
}

Enlarge image on click of link

#foreach (var item in Model)
{
<img src='ShowShowcaseImage/#Html.Encode(item.ProductID)' id='#item.ProductID' />
<b>#Html.DisplayFor(m => item.ProductName)</b>
Enlarge
}
<div id="EnlargeContent" class="content">
<span class="button bClose"><span>X</span></span>
<div style="margin: 10px;" id="imageContent">
</div>
<p align="center"></p>
</div>
//Popup javascript
$('.enlargeImg').bind('click', function (e) {
$.post('/Home/EnlargeShowcaseImage/' + $(this).attr('id'), null, function (data) {
document.getElementById("imageContent").innerHTML += data;
});
$('#EnlargeContent').bPopup();
});
});
//
C# method
public ActionResult EnlargeShowcaseImage(string id)
{
var imageData = //linq query for retrive bytes from database;
StringBuilder builder = new StringBuilder();
if (imageData != null)
builder.Append("<img src='" + imageData.ImageBytes + "' />");
return Json(builder);
}
I want to show pop up of enlarged image on click of enlarge link. Image is stored in bytes in database. Two images are stored in database for each product - one is thumbnail and the other is enlarged. I am showing thumbnail image and I want to show enlarged image on click of enlarge link. I can't retrieve it from database.

I can't retrieve it from database
So your question is about retrieving an image from a database, right? It has strictly nothing to do with ASP.NET MVC?
Unfortunately you haven't told us whether you are using some ORM framework to access to your database or using plain ADO.NET. Let's assume that you are using plain ADO.NET:
public byte[] GetImage(string id)
{
using (var conn = new SqlConnection("YOUR CONNECTION STRING COMES HERE"))
using (var cmd = conn.CreateCommand())
{
conn.Open();
// TODO: replace the imageData and id columns and tableName with your actual
// database table names
cmd.CommandText = "SELECT imageData FROM tableName WHERE id = #id";
cmd.Parameters.AddWithValue("#id", id);
using (var reader = cmd.ExecuteReader())
{
if (!reader.Read())
{
// there was no corresponding record found in the database
return null;
}
const int CHUNK_SIZE = 2 * 1024;
byte[] buffer = new byte[CHUNK_SIZE];
long bytesRead;
long fieldOffset = 0;
using (var stream = new MemoryStream())
{
while ((bytesRead = reader.GetBytes(reader.GetOrdinal("imageData"), fieldOffset, buffer, 0, buffer.Length)) > 0)
{
stream.Write(buffer, 0, (int)bytesRead);
fieldOffset += bytesRead;
}
return stream.ToArray();
}
}
}
}
and if you are using some ORM it could be as simple as:
public byte[] GetImage(string id)
{
using (var db = new SomeDataContext())
{
return db.Images.FirstOrDefault(x => x.Id == id).ImageData;
}
}
and then inside your controller action:
public ActionResult EnlargeShowcaseImage(string id)
{
var imageData = GetImage(id);
if (imageData != null)
{
// TODO: adjust the MIME Type of the images
return File(imageData, "image/png");
}
return new HttpNotFoundResult();
}
and it is inside your view that you should create an <img> tag pointing to this controller action upon button click:
$('.enlargeImg').bind('click', function (e) {
$('#imageContent').html(
$('<img/>', {
src: '/Home/EnlargeShowcaseImage/' + $(this).attr('id')
})
);
$('#EnlargeContent').bPopup();
});
But hardcoding the url to your controller action in javascript like this is very bad practice because when you deploy your application it might break. It might also break if you decide to change the pattern of your routes. You should never hardcode urls like this. I would recommend you generating this url on the server.
For example I see that you have subscribed to some .enlargeImage element. Let's suppose that this is an anchor. Here's how to properly generate it:
#Html.ActionLink("Enlarge", "EnlargeShowcaseImage", "Home", new { id = item.Id }, new { #class = "enlargeImage" })
and then adapt the click handler:
$('.enlargeImg').bind('click', function (e) {
// Cancel the default action of the anchor
e.preventDefault();
$('#imageContent').html(
$('<img/>', {
src: this.href
})
);
$('#EnlargeContent').bPopup();
});

Try jQuery,
Here is one
http://superdit.com/2011/06/11/hover-image-zoom-with-jquery/
http://demo.superdit.com/jquery/zoom_hover/