Load and parse URL via JS in the background - javascript

Currently I am trying to develop a little Firefox extension.
In detail: i want to display users from the site dota2lounge.com the current prize of their steam items on the steam community market. My idea was to do this via a Firefox extension which reads the item names from the HTML code on dota2lounge.com . Via JS i would like to search the steam community market for the item names and parse the current prize. This should happen without any further action from the user and without opening extra tabs/windows.
In java i would just load the site into a variable and work with it. How could i do this with JS (or Jquery)? Or maybe there is an even better way in the addon-sdk from firefox which could solve this issue.
Any thoughts and hints are welcome.

This should be pretty simple to do using the Add-on SDK. Here is a list of modules you should look at:
the request module will allow you to make requests to other sites: https://developer.mozilla.org/en-US/Add-ons/SDK/High-Level_APIs/request
while the request module is fine, what you may want to do instead to get info from the steam site is use the page-worker module to load the site and easily extract info from it using jQuery. This is much nicer than using regex. The code would look something like this gist:
https://gist.github.com/canuckistani/6c299c812bbe582d9efb

Related

tvOS load website URL

I am new in tvOS application. i am trying to integrate custom application with TVML & TVJS. I created a page and add banner in it. My requirement is to load a webpage inside that application. I tried different methods to load a webpage using that application.js file but fails. Please help me how to load a webpage using TVML and TVJS.
If I understood you correctly, you need to use UIWebView class but you cannot find it in the tvOS SDK (because it was removed). But you can use this class as a "private API class" (you can create an instance of this class and call methods for it, because you have enough information about UIWebView). It can be hard for you. If so, you can use some libraries (such as https://github.com/jvanakker/tvOSBrowser) that using the private API to make UIWebView available in the tvOS.
I think you might have a hard time on getting a web browser on the TV app store, as I think apple does really see people browsing on the apple tv.
Depends on your use case,
if you are only trying to display a page, you might able to use some of the online url to picture services like https://urlbox.io/docs, or build your own server using existing browser features. https://developer.mozilla.org/en-US/docs/Web/API/HTMLIFrameElement/getScreenshot
If you are getting users to another services, the correct way to do is with deep link, which you can learn more about here. https://developer.apple.com/videos/play/wwdc2017/246/
or if you are trying to transfer the view into sales, displaying a QR code is the way to go. as iOS 11 makes QR Code scanning a standard, it will be very easy for your users to scan the code. https://www.cnet.com/how-to/how-to-use-ios-11s-hidden-qr-code-reader/

How to architect a HUD in a Google Chrome extension?

I am trying to make a Google Chrome extension using content script.
My goal is to have a display at the top of the page (which is already working on my own pages) that can interact with the page.
I need things which are very complicated to put together in an extension, due to security policies :
Using require.js on the extension (that works for now, using this Github repo)
Using a templating engine to describe my display : I need to add a lot of content to the page and I don't think writing HTML in javascript would be a good workflow.
For my current version I use jade with my server, but this is not possible with an extension. I think I need to use something like Angular.js or Backbone.js, but I can't make them work on the content script.
I need a lot of communication between my extension and the page : For example I need to detect almost constantly mouse moves
I need communication with my server using socket.io
Every bit of functionality of my extension have been developed and tried in a standalone web page, but now I need to integrate it in a real extension and I am really stuck
So due to these requirements, I am wondering what would be the right approach for building this : putting it all in an iFrame (would the server-side communication work? And how to communicate with the page ?), or a way to make a templating engine work nicely in there, or a solution I didn't think of?
Try this:
Develop the HUD part as a standalone page that the content script will include in an iframe. You should be able to use Angular.js etc. with this, but you will need local copies of as much as possible and you'll need appropriate entries in the manifest.json to get it working in the extension. See/create other questions for the details.
Have your content script inject the code to monitor mouse-moves, etc. into the target page. Have this code digest and summarize the data, so it's not spamming the system. Maybe message the summaries to the HUD page and/or content script five or six times a second.
After that, it should just be a matter of getting the pieces working, one at a time. Break it down to specific problems and ask a question on one specific problem at a time (If you can't find the answers in previous questions).
I'm pretty sure what you appear to want is do-able, but the details are too broad for a single Stack Overflow question.

Interacting with webpage from C++ Application

i have a sitation where i want to access HTML DOM object from within my application to update certain parts of web page through javascript commands at run time.
It is a local webpage opened in FireFox which would be accessed by my application, so that the final output is always shown at the webpage which is updated by appliation.
It would be great if you could give me some idea about how this can be accomplished.
I have similar requirement like the webmonkey extension of firefox but need to do it outside of browser from my application.
You can try QtWebKit from the Qt framework, it provides an OO set of classes to interact with webpages from basic actions to very complicated and advanced stuff. I believe you may find your answer there, a link is provided below...
Good Luck
see Here

How can I get dynamically web content using Perl?

This is kind of tricky. There is this webpage which, I am guessing, uses some kind of AJAX to pull out content based on the search query. When I fetch the page using get in Perl, it fetches the script code behind the php/html, but not the results which are displayed when the query is searched manually. I need to be able to fetch the content of the results page. Is there anyway to do this in Perl?
Take a look at Selenium RC and the WWW::Selenium module in Perl. With them you can control a real web browser.
Another option is WWW::HtmlUnit which uses the HtmlUnit Java library to execute the JavaScript without a web browser. WWW::HtmlUnit uses Inline::Java to give Perl access to the library. I have found that when installing, it is best to say No to the question "Do you wish to build the JNI extension?".
If you are writing tests that need to check the rendered page, you can have a look at Schwern's javascript-tap-harness, which works with Selenium and handles all the scaffolding.
I also found Using WWW::Selenium To Test Or Automate An Ajax Website pretty useful.

Screen scrape a web page that uses javaScript and frames

I want to scrape data from www.marktplaats.nl . I want to analyze the scraped description, price, date and views in Excel/Access.
I tried to scrape data with Ruby (nokogiri, scrapi) but nothing worked. (on other sites it worked well) The main problem is that for example selectorgadget and the add-on firebug (Firefox) don’t find any css I can use to scrape the page. On other sites I can extract the css with selectorgadget or firebug and use it with nokogiri or scrapi.
Due to lack of experience it is difficult to identify the problem and therefore searching for a solution isn’t easy.
Can you tell me where to start solving this problem and where I maybe can find more info about a similar scraping process?
Thanks in advance!
I used excel web query and it works perfect. You can find a lot about scraping with excel on youtube if you search for mrexcel.
Thanks, Mello
You can try IRobotSoft web scraper. It has good frame support and is free.
Iframes aren't a problem - just access the embedded iframe URL directly. You will find that it redirects in the browser unless you disable JavaScript.
Description and date can be extracted straight from HTML source. However prices are images which will make scraping them more cumbersome.

Categories