is it possible to programmatically download complete web page (like Chrome can when you hit Ctrl+S) from my web extension?
I need the HTML with accompanying resources (images etc), so pageCapture (MHTML) is not an option i guess (also downloading each resource with ajax separately is not an option here).
Not sure I understood your problem, are you writing your own extension or looking for one?
If you're looking for one I'd recommend Save All Resources extension.
Please note: ensure you are aware of the data that is shared by this extension with the developer (privacy).
Save All Resources collects the following:
Website content
For example: text, images, sounds, videos or hyperlinks
yes you can by using HTTrack software.
HTTrack allows you to download a World Wide Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. Simply open a page of the "mirrored" website in your browser, and you can browse the site from link to link, as if you were viewing it online. HTTrack can also update an existing mirrored site, and resume interrupted downloads. HTTrack is fully configurable, and has an integrated help system.
download here.
https://www.httrack.com/page/2/en/index.html
so hope the answers.
Related
I am new to web dev and currently building a very basic page using Html/CSS and JS where users can upload the image and I can download it in my local system. But by default images are getting downloaded in the 'download' section of my desktop and I want to save them to any specific folder. How can I add this functionality?
You can't.
You do not have access to other peoples file managers as that would be a major security risk.
Uhh... You can't manipulate the download folder from frontend. It may be a security risk.
This cannot be done by Javascript (otherwise we could all produce websites which saved files all over a user's local PC).
As the end user in this case you can go into your browser and tell it which folder to save downloads in by default. (And remember to change it back if you want any other downloads to go elsewhere).
Here for example is what I find on Edge on Windows10 when going into Settings>Downloads:
This is an odd use-case and I've tried to steer the users away from this, but I have a request to load locally stored PPT files in an HTML environment.
The idea is to view the PPT in an iframe so that I can "decorate" all around it. Users will be creating playlists, of sorts, so I need to add "next" and "previous" presnetation buttons, etc.
I have an online version of this working using the Office webapp link that many other posts have shared, but there is a concern for poor connectivity so they want it to be available offline.
My instinct was to shape this up as a PWA. I can make them use Chrome, so I have the FileSystem API "caching" the PPT files from the online source. However, I have no way to render the saved files!
Because it's offline, I can't use googledocs or continue to MS web office.
I don't know the AppCache feature well enough, but wondered if I can preload all of the docs in an iframe, will it cache those. My thought is that it won't, because the content is on microsoft (or Google's) site.
All users will have a legal copy of PPT on the computer, if that helps at all.
So far I have tried storing the files using the filesystem API in combination with the Chrome offline viewer extension and alternatively the Native Docs extension. The Chrome one will allow me to open a doc, but directly in edit mode. I need to simulate the behavior of a ppsx file, but neither extension works with ppsx files and I don't see documentation for an API that will allow me to "auto-play" the presentations.
Thanks for any ideas!
Wayne
I'm thinking of writing a Cordova app which downloads websites so they can be read offline (like HTTrack for Windows). The main reason is lack of a good offline RSS reader for Windows 10 tablets.
I know in general what I would have to do but is there some framework which could simplify some of it?
So far I think I would need to do the following:
Download the HTML of a site
Get a list of all assets (CSS, JS, images, videos)
Download those assets
Replace asset URLs with new local ones.
The biggest problem is downloading the assets. It's not as straight forward as parsing the HTML for link, script, and img tags since CSS could have imports and JS could have ajax calls.
Also, how to decide which assets to download? I wouldn't want to waste time downloading ads..
Also there are some specific questions:
How should I display a downloaded page? My first thought is in an iFrame to prevent collisions.
Are there any legal problems? Especially if I were to publish the app?
How could I save the assets so they have a URL for including in the HTML?
Might it be better using a server to do the heavy lifting (parsing, rewriting, getting URLs etc.)? Are there tools for this already?
Does anyone have any pointers? Or do you think it's impractical?
Check out https://archivebox.io, it's an open-source, self-hosted tool that creates a local, static, browsable HTML clone of websites (it saves HTML, JS, media files, PDFs, screenshot, static assets and more).
It does most of what you want, including saving assets and media files with youtube-dl, wget, and chrome headless.
I'm an iOS dev looking into some security aspects of a native iPhone app that loads local html pages into a UIWebView (that's the iOS class that give browsers functionality to a native app).
If the app loads a local html file and that file contains a link to an external html file then is it possible for that external file to download a file to the client (another html file or a javascript file)? My knowledge of html/javascript isn't detailed enough in this area.
E.g. If A.html is file physically present on the phone and A.html contains a href to B.html where B.html is on a server then is it possible for B.html (using whatever means) to move a file (C.html or D.js etc.) from the server onto the device?
What you probably mean is : are HTML pages able to download (and potentially execute or overwrite) content/code on the device? I'm just going to assume that in the following.
First, the browser isn't supposed to be able to download anything without the user consent. That's especially true on iOS, where the browser actually can't download anything at all. (Well, you can actually download images by long-pressing on them, but they are only going to go to the Photo Roll. Any other download will just fail.)
HTML pages (and JavaScript, images, ...) are of course downloaded to the device before the browser or UIWebView displays them, but they can't access anything on the filesystem.
What's more, due to the same-origin policy, a web page cannot access anything that's outside of its domain (cross-origin requests allow this, but they require the server to send a particular header).
Say your page is on http://mywebsite.com : you can't make a request to http://blah.org if it's not setup properly, or even (much more dangerous) to file:///etc/passwd (which you have no way to setup properly).
I'm not sure about local pages, but (at least on Chrome on the desktop) local web pages cannot download anything at all.
Of course, all of these are the expected behavior.
There have been cases of vulnerabilities (this one for instance) where Safari allows unprivileged access to the filesystem, but in most cases they have been patched quickly by Apple and you won't have to (and are not supposed to if you want to avoid unnecessary headaches) worry about them.
I am currently creating a Chrome extension (which uses javascripts mainly) that allows users to scrape the images on a webpage and download them. I have finished the link scraping part, and the code will return an array like:
["http://example.com/image1.jpg","http://example.com/image2.jpg"]
But how do I download all of the links in ONE CLICK? I tried listing all photos on a new tab and let the users to Ctrl+S save the page. But this greatly affects the UI and I do not like it. I do not host webpage so server side script may not be working.. Any other solutions?
As far as I know, Chrome extensions technically can't save files to disk like Firefox.
The only way to do this is using NPAPI
Unfortunately, extensions using npapi will most likely not be accepted by the Web Store due to security problems. Of course it'll be okay if you use it for yourself or host the extension on your website.
You can install and examize the code of the following extensions, maybe you can even use the provided npapi too:
Screen Capture (by Google) https://chrome.google.com/webstore/detail/cpngackimfmofbokmjmljamhdncknpmg
Chrome Toolbox (by Google) https://chrome.google.com/webstore/detail/fjccknnhdnkbanjilpjddjhmkghmachn
Awesome Screenshot: Capture & Annotate https://chrome.google.com/webstore/detail/alelhddbbhepgpmgidjdcjakblofbmce
Download Asisstant (by Google) - got killed I guess.