I will try to ask my Question as precise as I can. At the moment I cannot post any example Code as I'm not allowed to post code of my work. In my next free time I will lookup that I will code some example to be more precise.
Here is my Question:
I'm developing some project where I built some HTML and Javascript code dynamically with Java and load it into a Webview of JavaFX. The Html has to be generated dynamically because there are lots of Information in it that come up from a central database. Then I try to run it via a ChangeListener that is set on LoadWorker for the WebEngine of the Webview to ensure that the Webpage is loaded. The page is loaded correctly but when I try to use some javascript function the Application crashes.
The whole thing is working with a local written html document. In the Html are lots of Javascript-Libraries like Jquery and others. But my project crashes because the libraries and its functions cannot be used. I think that the http-Request of loading a local written html sets up these javascript when the webpage is loaded.
Am I correct with this?
And is there anything I can fix this Problem with dynamic generated htmls loaded in a Webview.
Thanks for any help!
Related
I am coding a web app in Flask. I need to find a way to load "posts" with text variables (from external Python scripts) into an HTML file, ideally using pure Javascript (Not Jquery). The posts need to be loaded automatically, upon the opening of a page.
As of right now, I use the <body onload="function_name">, but I am not sure how to load more than one post. I know how to use python variables in html, but I also don't know how to use those variables to generate posts.
Any pointers/resources/code snippets would be immensely helpful.
Is there any way we get the full HTML source code of a website with JavaScript code (fetched from 3rd party) are also included in it?
I understand we can get using document.getElementsByTagName('html')[0].innerHTML but it will not include the 3rd party JS in the HTML code?
The reason for this question (based on my knowledge) is when the browser renders and create DOM then it should have all the required files so does anyone know where and how we can get that full HTML source code of any website?
Yes when the browser renders the page it does indeed fetch these resources, you can use the save option provided by the browser, by clicking right click, then save as, then save as a complete webpage.
If the previous does not work, then there are many tools, extensions, applications that provides this functionality and its easy to search for them.
If you want to save the html and its resources in one html file, you can save it as mhtml file, using this extension for example https://chrome.google.com/webstore/detail/save-webpages-offline-as/nfbcfginnecenjncdjhaminfcienmehn.
I'm working on a project that needs to have a header in all pages, I decided to use php for that and I have an external one JavaScript file. My problem is when the changed content on the html page page JavaScript start running function on that page, but it should not. I am interested how using one JavaScript file on multiple html pages, but there is no conflict. There is a solution to get around this, but it's not good. Plus I have light and dark mode on html pages, so I do not think it's good to create more JavaScript files with the same code :) thanks
How can I screen scrape a multi page application? I want to do this using Javascript. Here are the approaches I have considered and the problems I have encountered.
Using the Fetch web API in a Node application to get the web pages
Problem: The web pages won't load properly when being fetched. I guess all javascript on the page does not run when the page is fetched.
Running JavaScript from the console
This is a very simple way to inject JavaScript straight into the document. But one problem is that opening the web page is a browser and pasting into the console is manual work. Another problem is that while this works for single page application it becomes very cumbersome for multi-page applications.
What better approach exists that solves the problems I have encountered?
Depends on what are you doing. If you just want to get some that from some website then injecting JS in the page is the way to go.
But as you said it's manual work from which I deduce you want to scrape the sites and save the data maybe. In this case a service side script is better suited. To fix the problem with the JavaScript not being loaded you can use things like PhantomJs or Horseman.
Take a look at this: https://medium.com/#designman/building-a-performant-web-scraper-in-node-js-5f4449674163
If you want to save website content (html, js, css files, images) to file system you can take a look on website-scraper package for nodejs https://www.npmjs.com/package/website-scraper
It also has plugin for PhantomJS which allows to handle single page applications
I am trying to crawl some websites and while I am using headless chrome browser with selenium to render some HTLM that have embedded JS, I would also like to simply use requests, for the cases where there is no need for JS code rendering.
Is there a way to know if the HTML needs to be rendered by a browser or if a simple requests.get() would give me the complete HTML content?
Any HTML code generated by tags won't be retrieved by requests.
The only way to know if a page would need to be rendered by a browser to generate the whole content is to check if its HTML code has tags.
Still, if the information you are interested on is not generated by JS, requests.get() will serve you well.