how to get dynamic generated src attribute of web element? - javascript

I want to check if our advertisement(picture or flash) has successfully displayed on website page.But the element can not be found in page source.
I get the element in chrome Developer tools.
<div id="QQcom_all_Rectangle:1" data-loc="QQcom_all_Rectangle" data-index="1" style="height: 250px; display: block; width: 300px; position: relative;" class="l_qq_com" adconfig_lview="l.qq.com" adconfig_charset="gbk" adconfig_lview_template="http://l.qq.com/lview?c=www&loc={loc}" oid="1800716433" display="banner">
<a class="absolute a_cover" href="http://c.l.qq.com/lclick?loc=QQcom_all_Rectangle&click_data=dXNlcl9pbmZvPW9CM2pnVGd4RnhHNyZhZHhfZXh0PSZwY3RyPTUwMSZhdmVyPTUwMTIwMSZwcmk9eHRiQkZXc0ovclloYXdzdWZqRmpSTkhyZWFuL3pQU2omYnRwcmk9R3VBL25heHhnY3JyNTdrdVNCNW4yWis3TlJqM01nTmw=&oid=1800716433&soid=gmLndBibVj/1bQtQRjKVNkVKAV1Q&dtype=0&pctr=501&aver=501201&btoid=100418428&pri=xtbBFWsJ/rYhawsufjFjRNHrean/zPSj&btpri=GuA/naxxgcrr57kuSB5n2Z+7NRj3MgNl&index=1&page_type=2&chl=703&k=%E6%9B%9D%E7%81%AB%E7%AE%AD%E5%B7%B2%E8%A3%81%E6%8E%89%E6%B5%B7%E8%80%B6%E6%96%AF%20%E5%9B%9E%E5%BD%92%E4%BC%91%E5%9F%8E%E4%BB%85%E4%B8%80%E5%91%A8%E5%BE%81%E6%88%982%E5%9C%BA%2C%E6%B5%B7%E8%80%B6%E6%96%AF%2C%E7%81%AB%E7%AE%AD%2CNBA&t=%E6%9B%9D%E7%81%AB%E7%AE%AD%E5%B7%B2%E8%A3%81%E6%8E%89%E6%B5%B7%E8%80%B6%E6%96%AF%20%E5%9B%9E%E5%BD%92%E4%BC%91%E5%9F%8E%E4%BB%85%E4%B8%80%E5%91%A8%E5%BE%81%E6%88%982%E5%9C%BA_&r=&s=" target="_blank" rel="nofollow" style="position:absolute;width:300px;height:250px;left:0px;top:0px;cursor:pointer;z-index:10;background-color:#fff;filter:alpha(opacity=0);opacity:0;"></a>
<div class="absolute" style="position: absolute; width: 24px; height: 16px; left: 26px; bottom: 0px; cursor: pointer; display: none; z-index: 20; background: url(http://ra.gtimg.com/web/res/icon/report_default_new.png) 50% 0% no-repeat;"></div>
<div style="position: absolute; left: 0px; bottom: 0px; width: 26px; height: 16px; z-index: 12; background: url(http://ra.gtimg.com/web/res/icon/leftbottom_new.png) 100% 0% no-repeat;"></div>
<div class="absolute" style="position:absolute;width:18px;height:18px;right:0px;bottom:0px;cursor:pointer;z-index:20;background:url(http://ra.gtimg.com/web/privacy/white_icon.png) no-repeat;"></div>
</div>
but in page source,it only shows
<!--$loc$_div AD begin...."l=$loc$&log=off"--><div id="QQcom_all_Rectangle:1" data-loc="QQcom_all_Rectangle" data-index="1" style="height:0;" class="l_qq_com"></div><!--$loc$ AD end --><!--[if !IE]>|xGv00|c5668531d36ed7899852180841ca2aa2<![endif]-->
how can I get the 300px_250px image url?
anyone knows?

Yeah! That's simply because of the fact that there are DOM manipulation libraries which alter the Document-Object Markup and not the source.
You have to understand the difference between the DOM and the physical page source. The physical page source helps render the DOM, after which it can be modified by using libraries; since you can't change the source on the server side as it needs to be re-rendered for different people, only the DOM is changed. In very simple words, DOM is what the browser has rendered: it's like a copy of the source which has been, then, modified by the libraries.
Using a library like jQuery can do this easily.
var $element = $( "#QQcom_all_Rectangle:1" );
if ( !$element || $element === null ) {
console.log( "The element hasn't been rendered. Not found.");
}
But if it is not YOUR page, and you can't really modify it and/or get the data. You can try doing this with a web-kit emulator like PhantomJS or Selenium web-kit. Since these are just testing frameworks, you can't create a fully fledged JavaScript applications.
Theoretically, you can:
Render this in a WebKit (Browser Rendering Component)
Get the source of the DOM
Use it
But that's just in theory because you need to use some language to create an application like that and since it's outside the scope of your question, you can't.

Using selenium webdriver can handle this.
WebElement QQcom_all_Rectangle=driver.findElement(By.id("QQcom_all_Rectangle:1"));
List<WebElement> links=QQcom_all_Rectangle.findElements(By.tagName("a"));
String style=links.get(0).getAttribute("style");

Related

simulate real user click bypassing a web honeypot

I would like to scrape a web page but I can't be detected as a bot. I am using js to fetch data, fill inputs or click buttons.
I have read that I have to take into account some "diplayed: none" attributes as they seems to be honeypots. In my webe there is a div which shows this:
// When mouse is not used
<div style="position: absolute; top: 0px; left: 0px; display: none; z-index: 10000;"><div class="tip">
// When mouse has moved but not on any clickable point
<div style="position: absolute; top: 264px; left: 272px; z-index: 10000; display: none;"><div class="tip">
// When mouse on a clickabel point
<div style="position: absolute; top: 264px; left: 272px; z-index: 10000;"><div class="tip">
This make me think the web check if the click has been done programmatically or if it is from a real user.
Thereby
Is this something I have to work with or it is insignificant??
If so, how could I bypass it??
Thank you very much
display:none; isn't your only problem, one could also use negative left:-100px; or top:-100px; values, and/or color:white; on a white background, and so forth.
But they are all foolhardy attempts to prevent bots when all they had to do is this...
Luckily, JavaScript already provides a mechanism for determining if an event was user-initiated via the "isTrusted" boolean property. Usage is as follows...
<button id="logon" onclick="if(event.isTrusted){SomeFunction();}">Logon</button>
Any JavaScript attempt to click this button such as logon.click(); would fail this test and the function would never run, and no there's no way you can simulate a human gesture.
Hackers exploit bad programmers who use foolish tricks like the ones you and I mentioned.
Good point raised by Dave, but the version posted was for clarity purposes.
See if you can fool this little modification...
<!DOCTYPE html>
<head>
</head>
<body>
<script type="text/javascript">
function SomeFunction(event){
if(event.isTrusted){
alert('This came from a human');
} else {alert('This came from a bot');}
}
</script>
<button id="logon" onclick="SomeFunction(event);">Logon</button>
<button onclick="logon.click();">BOT clicking button</button>
<button onclick="SomeFunction(true);">BOT running function directly</button>
</body>
</html>
NB: For the last button, we are simulating a BOT here through a button so we have the luxury of passing the human onclick event to get through, but that won't be the case for a BOT and hence the boolean variable I added.

Getting html source changes dynamically from ifram tag

I've been trying to get html source from iframe tag for a while, however, sadly I'm still failing on it. My environment is under chrome browser and with dom-distiller extension that gives me a feature extracting main articles from web pages. This extension worked with the code below.
<iframe id="dom-distiller-result-iframe"
src="chrome-extension://oiembdaoobijmdmeobkalaehgifealpl/external/chromium/src/components/dom_distiller/core/html/dom_distiller_viewer.html"
style="z-index: 2147483647; position: fixed;
left: 0px; right: 0px; top: 0px; bottom: 0px;
margin: auto; width: 100vw; height: 100vh; background: white; border: none;">
</iframe>
It injected some iframe tag in the web page I'm seeing. With this iframe tag, a distilled document was wonderfully rendered without ADs. I thought if I could access iframe tag whose id is "dom-distiller-result-iframe" and get rendered html source code. I've tried many ways however, none of them didn't worked.
Could any one give me some advices?
Did not test it but should do the job:
<?
$url = "yoursite.com";
$html = file_get_html($url);
libxml_use_internal_errors(true);
$dom = new DOMDocument();
$dom->loadHTML($html);
$iFrame = $dom->getElementsByTagName('iframe')->item(0);
$iFrameID = $iFrame->getAttribute('id');
$iframehtml = file_get_html($iFrameID);
echo $iframehtml;
?>

Create sub-tab similar to DevTools from Chrome Extension

Question
Can a Chrome extension add a "sub-tab" to a Chrome tab like the one that is created with Chrome DevTools (Inspect)?
Background
I currently am adding a div to the side of a window from my extension by taking an HTML file, sidebar.html, that contains the following code:
<style>
#comment_container {
float: left;
width: 30%;
position: fixed;
top: 0;
height: 100%;
background-color: #aabcff;
opacity: 0.9;
}
</style>
<div id="comment_container">
<div>
<p>...</p>
</div>
</div>
After a series of functions, sidebar.html is stored in a javascript variable sidebar. I then call
$( "body" ).wrapInner( "<div id='outerDiv'></div>");
$( "#outerDiv" ).width("70%");
$( "body" ).append(sidebar);
This appends a sidebar to the side of the window. What I would like to do is, instead of appending #comment_container to the document, to create a "sub-tab" like Chrome's DevTools (Inspect).

How to validate google map embed code iframe?

I already searched for this topic but unfortunately cannot find the answer.
Is there any way to validate the Google map embed code iframe? Like when user paste the code in a textbox and click a button, system checks if that code is valid. Or is parsing the code and checking for pattern is the only way?
Thank you in advance for any help or idea.
Google is giving you the correct code to use. I use it all the time for my friend's real estate websites. Use a responsive iframe code too so it adapts to all the various screen sizes. Use this code, change the iframe source to fit your needs.
/* Flexible iFrame */
.Flexible-container {
position: relative;
padding-bottom: 56.25%;
padding-top: 30px;
height: 0;
overflow: hidden;
}
.Flexible-container iframe,
.Flexible-container object,
.Flexible-container embed {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}
<!-- Responsive iFrame -->
<div class="flexible-container">
<!-- Responsive iFrame -->
<div class="flexible-container">
<iframe src="https://www.google.com/maps/embed?pb=!1m14!1m8!1m3!1d1457.0935074158538!2d-70.75659948473937!3d43.07956346598421!3m2!1i1024!2i768!4f13.1!3m3!1m2!1s0x0%3A0xb5e8c11925c54ab4!2sSamonas+Realty+Llc!5e0!3m2!1sen!2sus!4v1402966179098" width="600" height="450" frameborder="0" style="border:0"></iframe>
</div>
</div>
For the record I searched for regex that match iframe google embed code.
I found this question: PHP - Regex to check if <iframe> comes from Google Maps
And modified the regex:
<iframe\s*src="https:\/\/www\.google\.com\/maps\/embed\?[^"]+"*\s*[^>]+>*<\/iframe>
Hope that helps someone with same problem. :D Thank you all!

jQuery jScrollPane only works with text

I am trying to use jScrollPane to scroll through my gallery that is inside of an iframe. I am using the jQuery code for iframe scrolling from the official example.
Using this code doesn't work for my images (it just hides the original browser scrollbars, doesn't throw any errors and refuses to scroll), but it works perfectly if I replace my images with multiple paragraphs so they overflow (just like in the example).
EDIT: I forgot to mention that images scroll perfectly using the default browser scrollbar.
EDIT #2: Made a JSFiddle.
Here's how my gallery is structured:
...
</head>
<body>
<div id="content"> // Used the same way as in the example, works with text
<div class="gallery">
<div class='picture'>
<img class='pin' src='something'/>
<div class='wrapper'>
<img class='thumbnail' src='something'/>
<img class='border' src='something' />
</div>
</div>
<div class='picture'>
...
</div>
</div>
</div>
...
And here's the CSS for gallery and pictures:
.gallery {
position: relative;
width: 98%;
}
.picture {
float: left;
position: relative;
display: list-item;
list-style-type: none;
width: 11%;
}
.picture .pin {
position: absolute;
left: 48%;
width: 13%;
}
.picture .thumbnail {
position: absolute;
width: 89%;
margin-top: 19%;
}
.picture .border {
position: absolute;
width: 100%;
}
I ignored margins and some other irrelevant stuff, but you get the idea.
The jQuery code is exactly the same as presented in the example.
I think the problem is that images are loaded after scrollbar is initialized, and scrollbar does not detect container size changes by default. If you read documentation carefully, you can see next:
Demo showing how autoReinitialise can also be used so that content
with images in it displays correctly in jScrollPane
So, try autoReinitialise option. If it won't help - update your question with example of your iframe on jsFiddle.net
Solved it!
Apparently, the script didn't like the absolute position of my .picture .pin. Changing it to relative and restyling a bit solved my issue.
Updated JSFiddle

Categories