I want to do some event eg. clicks in a website. I can do it in chrome with javascript (or chrome extension), but is it possible to do without opening chrome but with server side code? No API is provided. It's not scraping but perform some sort of action.
NodeJS uses Google V8 engine to interpret the JavaScript code. It does not run in a browser environment and therefore it lacks DOM and event handling. However, you can actually mock browser in NodeJS environment using mock-browser package.
const MockBrowser = require('mock-browser/lib/MockBrowser')
const mockBrowser = new MockBrowser()
global.window = mockBrowser.getWindow()
global.document = mockBrowser.getDocument()
global.navigator = mockBrowser.getNavigator()
However, you should be careful with this approach, as some methods (e.g. getComputedStyle) still will not work.
Maybe you should reconsider why you want to use DOM and events on the server side.
PhantomJS: Headless browser for NodeJS
PhantomJS is a headless browser for NodeJS that is used for testing, scraping, etc. It provides you with a full-featured browser that can simulate a browser.
Using CasperJS for scraping
If you want to scrape websites, you may use a library called CasperJS that itself uses PhantomJS. An example:
var casper = require('casper').create();
var links;
function getLinks() {
// Scrape the links from top-right nav of the website
var links = document.querySelectorAll('ul.navigation li a');
return Array.prototype.map.call(links, function (e) {
return e.getAttribute('href')
});
}
// Opens casperjs homepage
casper.start('http://casperjs.org/');
casper.then(function () {
links = this.evaluate(getLinks);
});
casper.run(function () {
for(var i in links) {
console.log(links[i]);
}
casper.done();
});
I know that client-side Javascript cannot write data to the local filesystem, for obvious security reasons.
The only way to save data locally with Javascript seems to be with cookies, localStorage, or allow the user to download a file (with a "Save..." dialog box or to the browser's default Download folder).
But is it possible, in the very specific case when the file is accessed locally with an URL like file:///D:/test/index.html (and not through internet) to write data locally ? (without any server language, and even without any server at all: just local browsing of a HTML file)
For example, would it be possible, by clicking on SAVE here:
<div contenteditable="true" style="height:200px;">Content editable - edit me and save!</div>
<button>Save</button>
...that such a HTML file (accessed through file:///D:/test/index.html) is overwritten with its new content ? (i.e. the local HTML file should be updated when SAVE is pressed).
TL;DR: Is this possible to SAVE a file thanks to Javascript, when the HTML page is accessed locally?
Note: I want to be able to silently save, not propose a Download/Save dialog box in which the user has to choose where to download, then "Are you sure to want to overwrite" etc.
EDIT : Why this question? I'm doing an in-browser notepad that I can run locally without any server (no Apache, no PHP). I need to be able to save easily without having to deal with Dialog Box "Where do you want to download the file?" and having to always re-browse to the same folder to overwrite the currently-being-edited file. I would like a simple UX like in any notepad program: CTRL+S done, the current file is saved! (example: MS Word doesn't ask to browse where you want to save the file each time you do "Save": CTRL+S, done!)
You can just use the Blob function:
function save() {
var htmlContent = ["your-content-here"];
var bl = new Blob(htmlContent, {type: "text/html"});
var a = document.createElement("a");
a.href = URL.createObjectURL(bl);
a.download = "your-download-name-here.html";
a.hidden = true;
document.body.appendChild(a);
a.innerHTML = "something random - nobody will see this, it doesn't matter what you put here";
a.click();
}
and your file will save.
The canonical answer, from the W3C File API Standard:
User agents should provide an API exposed to script that exposes the features above. The user is notified by UI anytime interaction with the file system takes place, giving the user full ability to cancel or abort the transaction. The user is notified of any file selections, and can cancel these. No invocations to these APIs occur silently without user intervention.
Basically, because of security settings, any time you download a file, the browser will make sure the user actually wants to save the file. Browsers don't really differentiate JavaScript on your computer and JavaScript from a web server. The only difference is how the browser accesses the file, so storing the page locally will not make a difference.
Workarounds:
However, you could just store the innerHTML of the <div> in a cookie. When the user gets back, you can load it back from the cookie. Although it isn't exactly saving the file to the user's computer, it should have the same effect as overwriting the file. When the user gets back, they will see what they entered the last time. The disadvantage is that, if the user clears their website data, their information will be lost. Since ignoring a user's request to clear local storage is also a security problem, there really is no way around it.
However, you could also do the following:
Use a Java applet
Use some other kind of applet
Create a desktop (non-Web based) application
Just remember to save the file when you clear your website data. You can create an alert that pops up and reminds you, or even opens the save window for you, when you exit the page.
Using cookies: You can use JavaScript cookies on a local page. Just put this in a file and open it in your browser:
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<p id="timesVisited"></p>
<script type="text/javascript">
var timesVisited = parseInt(document.cookie.split("=")[1]);
if (isNaN(timesVisited)) timesVisited = 0;
timesVisited++;
document.cookie = "timesVisited=" + timesVisited;
document.getElementById("timesVisited").innerHTML = "You ran this snippet " + timesVisited + " times.";
</script>
</body>
</html>
Chromium's File System Access API (introduced in 2019)
There's a relatively new, non-standard File System Access API (not to be confused with the earlier File and Directory Entries API or the File System API). It looks like it was introduced in 2019/2020 in Chromium/Chrome, and doesn't have support in Firefox or Safari.
When using this API, a locally opened page can open/save other local files and use the files' data in the page. It does require initial permission to save, but while the user is on the page, subsequent saves of specific files do so 'silently'. A user can also grant permission to a specific directory, in which subsequent reads and writes to that directory don't require approval. Approval is needed again after the user closes all the tabs to the web page and reopens the page.
You can read more about this newish API at https://web.dev/file-system-access/. It's meant to be used to make more powerful web applications.
A few things to note about it:
By default, it requires a secure context to run. Running it on https, localhost, or through file:// should work.
You can get a file handle from dragging and dropping a file by using DataTransferItem.getAsFileSystemHandle
Initially reading or saving a file requires user approval and can only be initiated via a user interaction. After that, subsequent reads and saves don't need approval, until the site is opened again.
Handles to files can be saved in the page (so if you were editing 'path/to/file.html', and reload the page, it would be able to have a reference to the file). They can't seemingly be stringified, so are stored through something like IndexedDB (see this answer for more info). Using stored handles to read/write requires user interaction and user approval.
Here are some simple examples. They don't seem to run in a cross-domain iframe, so you probably need to save them as an html file and open them up in Chrome/Chromium.
Opening and Saving, with Drag and Drop (no external libraries):
<body>
<div><button id="open">Open</button><button id="save">Save</button></div>
<textarea id="editor" rows=10 cols=40></textarea>
<script>
let openButton = document.getElementById('open');
let saveButton = document.getElementById('save');
let editor = document.getElementById('editor');
let fileHandle;
async function openFile() {
try {
[fileHandle] = await window.showOpenFilePicker();
await restoreFromFile(fileHandle);
} catch (e) {
// might be user canceled
}
}
async function restoreFromFile() {
let file = await fileHandle.getFile();
let text = await file.text();
editor.value = text;
}
async function saveFile() {
var saveValue = editor.value;
if (!fileHandle) {
try {
fileHandle = await window.showSaveFilePicker();
} catch (e) {
// might be user canceled
}
}
if (!fileHandle || !await verifyPermissions(fileHandle)) {
return;
}
let writableStream = await fileHandle.createWritable();
await writableStream.write(saveValue);
await writableStream.close();
}
async function verifyPermissions(handle) {
if (await handle.queryPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
if (await handle.requestPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
return false;
}
document.body.addEventListener('dragover', function (e) {
e.preventDefault();
});
document.body.addEventListener('drop', async function (e) {
e.preventDefault();
for (const item of e.dataTransfer.items) {
if (item.kind === 'file') {
let entry = await item.getAsFileSystemHandle();
if (entry.kind === 'file') {
fileHandle = entry;
restoreFromFile();
} else if (entry.kind === 'directory') {
// handle directory
}
}
}
});
openButton.addEventListener('click', openFile);
saveButton.addEventListener('click', saveFile);
</script>
</body>
Storing and Retrieving a File Handle using idb-keyval:
Storing file handles can be tricky, since they can't be unstringified, though apparently they can be used with IndexedDB and mostly with history.state. For this example we'll use idb-keyval to access IndexedDB to store a file handle. To see it work, open or save a file, and then reload the page and press the 'Restore' button. This example uses some code from https://stackoverflow.com/a/65938910/.
<body>
<script src="https://unpkg.com/idb-keyval#6.1.0/dist/umd.js"></script>
<div><button id="restore" style="display:none">Restore</button><button id="open">Open</button><button id="save">Save</button></div>
<textarea id="editor" rows=10 cols=40></textarea>
<script>
let restoreButton = document.getElementById('restore');
let openButton = document.getElementById('open');
let saveButton = document.getElementById('save');
let editor = document.getElementById('editor');
let fileHandle;
async function openFile() {
try {
[fileHandle] = await window.showOpenFilePicker();
await restoreFromFile(fileHandle);
} catch (e) {
// might be user canceled
}
}
async function restoreFromFile() {
let file = await fileHandle.getFile();
let text = await file.text();
await idbKeyval.set('file', fileHandle);
editor.value = text;
restoreButton.style.display = 'none';
}
async function saveFile() {
var saveValue = editor.value;
if (!fileHandle) {
try {
fileHandle = await window.showSaveFilePicker();
await idbKeyval.set('file', fileHandle);
} catch (e) {
// might be user canceled
}
}
if (!fileHandle || !await verifyPermissions(fileHandle)) {
return;
}
let writableStream = await fileHandle.createWritable();
await writableStream.write(saveValue);
await writableStream.close();
restoreButton.style.display = 'none';
}
async function verifyPermissions(handle) {
if (await handle.queryPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
if (await handle.requestPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
return false;
}
async function init() {
var previousFileHandle = await idbKeyval.get('file');
if (previousFileHandle) {
restoreButton.style.display = 'inline-block';
restoreButton.addEventListener('click', async function (e) {
if (await verifyPermissions(previousFileHandle)) {
fileHandle = previousFileHandle;
await restoreFromFile();
}
});
}
document.body.addEventListener('dragover', function (e) {
e.preventDefault();
});
document.body.addEventListener('drop', async function (e) {
e.preventDefault();
for (const item of e.dataTransfer.items) {
console.log(item);
if (item.kind === 'file') {
let entry = await item.getAsFileSystemHandle();
if (entry.kind === 'file') {
fileHandle = entry;
restoreFromFile();
} else if (entry.kind === 'directory') {
// handle directory
}
}
}
});
openButton.addEventListener('click', openFile);
saveButton.addEventListener('click', saveFile);
}
init();
</script>
</body>
Additional Notes
Firefox and Safari support seems to be unlikely, at least in the near term. See https://github.com/mozilla/standards-positions/issues/154 and https://lists.webkit.org/pipermail/webkit-dev/2020-August/031362.html
Yes, it's possible.
In your example, you are already using ContentEditable and most of tutorials for that attribute have some sort of localStrorage example, ie. http://www.html5tuts.co.uk/demos/localstorage/
On page load, script should check localStorage for data and if true, populate element. Any changes in content could be saved in localStorage when clicking save button (or automatically, in linked example, using blur and focus). Additionally you can use this snippet to check weather user is online or offline and based on state modify your logic:
// check if online/offline
// http://www.kirupa.com/html5/check_if_internet_connection_exists_in_javascript.htm
function doesConnectionExist() {
var xhr = new XMLHttpRequest();
var file = "http://www.yoursite.com/somefile.png";
var randomNum = Math.round(Math.random() * 10000);
xhr.open('HEAD', file + "?rand=" + randomNum, false);
try {
xhr.send();
if (xhr.status >= 200 && xhr.status < 304) {
return true;
} else {
return false;
}
} catch (e) {
return false;
}
}
EDIT: More advance version of localStorage is Mozilla localForage which allows storing other types of data besides strings.
You could save files, and make it persistent using the FileSystem-API and webkit. You would have to use a chrome browser and it is not a standards technology but I think it does exactly what you want it to do. Here is a great tutorial to show how to do just that http://www.noupe.com/design/html5-filesystem-api-create-files-store-locally-using-javascript-webkit.html
And to show that its on topic it starts off showing you how to make the file save persistent...
window.webkitRequestFileSystem(window.PERSISTENT , 1024*1024, SaveDatFileBro);
Convert your HTML content to a data uri string, and set as href attribute of an anchor element. Don't forget to specify a filename as download attribute.
Here's a simple example:
<a>click to download</a>
<script>
var anchor = document.querySelector('a');
anchor.setAttribute('download', 'example.html');
anchor.setAttribute('href', 'data:text/html;charset=UTF-8,<p>asdf</p>');
</script>
Just try it in your browser, no server required.
Have a look into this :)
Download File Using Javascript/jQuery
there should be everything you need. If you still need help or it's not the solution you need, tell me ;)
Yes, it is possible. Proof by example:
TiddlyFox: allows modification of local files via an add-on. (source code) (extension page):
TiddlyFox is an extension for Mozilla Firefox that enables TiddlyWiki
to save changes directly to the file system.
Todo.html: An HTML file that saves edits to itself. Currently, it only works in Internet Explorer and you have to confirm some security dialogs when first opening the file. (source code) (functional demo).
Steps to confirm todo.html actually saves changes to itself locally:
Save todo.html to local harddrive
Open with Internet Explorer. Accept all the security dialogs.
Type command todo add TEST (todo.html emulates the command-line interface of todo.txt-CLI)
Inspect todo.html file for addition of 'TEST'
Caveats: there is no cross-platform method. I'm not sure how much longer these methods will exist. When I first started my todo.html project, there was a jQuery plugin called twFile that allowed cross-browser loading/saving of local files using four different methods (ActiveX, Mozilla XUL, Java applet, Java Live Connect). Except for ActiveX, browsers have disallowed all these methods due to security concerns.
If you are fine with your code running outside of the scope of your default browser, and you are fine with windows only support, HTAs meet the silently save without prompts requirement easily.
The below code doesn't use many HTA specific features but it does still use microsoft specific stuff like ActiveXObject("Scripting.FileSystemObject").
<html>
<head>
<title>Simple Notepad</title>
<meta http-equiv="X-UA-Compatible" content="IE=9">
<script>
document.addEventListener('keydown', function (event) {
if (event.ctrlKey) {
if (event.key == 's') {
var FSo = new ActiveXObject("Scripting.FileSystemObject");
//see https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/opentextfile-method
var thisFile = FSo.OpenTextFile(window.location.pathname, 2, true, -1);
thisFile.Write(document.getElementsByTagName("html")[0].outerHTML);
thisFile.Close();
// Comment out the below alert to get truly silent saving.
alert('Saved Successfully');
if (event.preventDefault) event.preventDefault();
return false;
}
}
}, false);
</script>
</head>
<body contentEditable="true">
<h1>Press <kbd>CTRL + S</kbd> To Save</h1>
</body>
</html>
It also isn't a very rich editing experience but that can be fixed with some more buttons or keyboard shortcuts I think. Like CTRL + B to embolden selected text. It doesn't have any safety checks as of yet, but binding an event handler to beforeunload should prevent any data loss caused by accidentally closing the program.
HTA's do have other disadvantages too. They don't support ES6 (though transpiling is an option).
Although it is a bit dated, If you're not trying to use modern web features, I think you'll agree that it is very functional and usable.
Edit
I forgot to mention, but HTAs have to be saved with the .hta file extension for mshta.exe to be registered as their file type handler. Which is needed so that you can double click it in windows explorer to open it easily.
See also
Introduction to HTML Applications on MSDN
HTML Applications reference on MSDN
I think it's important to clarify the difference between server and client in this context.
Client/server is a program relationship in which one program (the client) requests a service or resource from another program (the server).
Source: http://searchnetworking.techtarget.com/definition/client-server
I'm not sure you'll find too many advanced applications that don't have at least one server/client relationship. It is somewhat misleading to ask to achieve this without any server, because any time your program speaks to another program, it is a client/server relationship, with the requester being the client and the response coming from the server. This is even if you are working locally. When you want to do something outside of the scope of the browser, you need a hook in a server.
Now, that does not mean you can't achieve this without a server-side specific language. For example, this solution uses NodeJS for the server. WinJS has WinJS.xhr, which uses XmlHttpRequest to serve data to the server.
AJAX seeks to offer the same sort of functions. The point here is that whether you have a program or there is some sort of hook pre-built, when you issue a command like "save file" and the file actually gets saved, there is a program on the other side that is parsing it, whether it's a server-side language or something else, meaning you can't possibly have something like this function without a server to receive the request.
Just use https://github.com/firebase/firepad — See it in action
This doesn’t need a server on your computer, it will reach out and save the data remotely.
Use jsPDF -> https://github.com/MrRio/jsPDF
<div id="content">
<h3>Hello, this is a H3 tag</h3>
<p>a pararaph</p>
</div>
<div id="editor"></div>
<button id="cmd">generate PDF</button>
Javascript
var doc = new jsPDF();
var specialElementHandlers = {
'#editor': function (element, renderer) {
return true;
}
};
$('#cmd').click(function () {
doc.fromHTML($('#content').html(), 15, 15, {
'width': 170,
'elementHandlers': specialElementHandlers
});
doc.save('sample-file.pdf');
});
This is an example for those who want to know how to use the localStorage.
<div id="divInput" contenteditable="true" style="height:200px;border: 2px solid blue">
Content editable - edit me and save!
</div>
<button onclick="onSave()">Save</button>
<button onclick="onLoad()">Load</button>
<script>
config = {
localStorageItemName: "demo",
datetimeFormat: {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
hour12: false,
minute: '2-digit',
second: '2-digit'
}
}
function Now() {
return new Date().toLocaleString("zh-TW", config.datetimeFormat)
}
const errMap = {
IsEmptyError: new Error('is empty'),
LengthError: new Error('length = 0')
}
/**
* #param {string} itemName
* #return {Object}
* */
function getLocalStorageItem(itemName) {
const dbDataString = localStorage.getItem(itemName)
if (dbDataString === null) {
throw errMap.IsEmptyError
}
const db = JSON.parse(dbDataString)
if (Object.keys(db).length === 0) {
throw errMap.LengthError
}
return db
}
function onSave() {
const inputValue = document.querySelector(`#divInput`).textContent
try {
const db = getLocalStorageItem(config.localStorageItemName)
db.msg = inputValue
db.lastModTime = Now()
localStorage.setItem(config.localStorageItemName, JSON.stringify(db))
console.log("save OK!")
} catch (err) {
switch (err) {
case errMap.IsEmptyError:
console.info("new localStorageItemName")
localStorage.setItem(config.localStorageItemName,
JSON.stringify({msg: inputValue, createTime: Now()})
)
break
/*
case ...
break
*/
default:
console.error(err.message)
}
}
}
function onLoad(e) {
try {
const db = getLocalStorageItem(config.localStorageItemName)
console.log("load")
document.querySelector(`#divInput`).textContent = db.msg
} catch (err) {
return
}
}
(()=>{
window.onload = () => (
onLoad()
)
})()
</script>
It is written in pure javascript with no dependencies.
Can anyone tell me how to write test case for a link to download pdf file using jasmine framework ?
Thanks in advance.
I can currently set download path location
Chrome
capabilities: {
'browserName': 'chrome',
'platform': 'ANY',
'version': 'ANY',
'chromeOptions': {
// Get rid of --ignore-certificate yellow warning
args: ['--no-sandbox', '--test-type=browser'],
// Set download path and avoid prompting for download even though
// this is already the default on Chrome but for completeness
prefs: {
'download': {
'prompt_for_download': false,
'default_directory': '/e2e/downloads/',
}
}
}
}
For remote testing you would need a more complex infrastructure like setting up a Samba share or network shared directory destination.
Firefox
var FirefoxProfile = require('firefox-profile');
var q = require('q');
[...]
getMultiCapabilities: getFirefoxProfile,
framework: 'jasmine2',
[...]
function getFirefoxProfile() {
"use strict";
var deferred = q.defer();
var firefoxProfile = new FirefoxProfile();
firefoxProfile.setPreference("browser.download.folderList", 2);
firefoxProfile.setPreference("browser.download.manager.showWhenStarting", false);
firefoxProfile.setPreference("browser.download.dir", '/tmp');
firefoxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/vnd.openxmlformats-officedocument.wordprocessingml.document");
firefoxProfile.encoded(function(encodedProfile) {
var multiCapabilities = [{
browserName: 'firefox',
firefox_profile : encodedProfile
}];
deferred.resolve(multiCapabilities);
});
return deferred.promise;
}
Finally and maybe obvious, to trigger the download you click on the download link as you know, e.g.
$('a.some-download-link').click();
I needed to check the contents of the downloaded file (a CSV export in my case) against an expected result, and found the following to work:
var filename = '/tmp/export.csv';
var fs = require('fs');
if (fs.existsSync(filename)) {
// Make sure the browser doesn't have to rename the download.
fs.unlinkSync(filename);
}
$('a.download').click();
browser.driver.wait(function() {
// Wait until the file has been downloaded.
// We need to wait thus as otherwise protractor has a nasty habit of
// trying to do any following tests while the file is still being
// downloaded and hasn't been moved to its final location.
return fs.existsSync(filename);
}, 30000).then(function() {
// Do whatever checks you need here. This is a simple comparison;
// for a larger file you might want to do calculate the file's MD5
// hash and see if it matches what you expect.
expect(fs.readFileSync(filename, { encoding: 'utf8' })).toEqual(
"A,B,C\r\n"
);
});
I found Leo's configuration suggestion helpful for allowing the download to be saved somewhere accessible.
The 30000ms timeout is the default, so could be omitted, but I'm leaving it in as a reminder in case someone would like to change it.
it could be the test for checking href attribute like so:
var link = element(by.css("a.pdf"));
expect(link.getAttribute('href')).toEqual('someExactUrl');
The solutions above would not work for remote browser testing, e.g. via BrowserStack. An alternative solution, just for Chrome, could look like this:
if ((await browser.getCapabilities()).get('browserName') === 'chrome') {
await browser.driver.get('chrome://downloads/');
const items =
await browser.executeScript('return downloads.Manager.get().items_') as any[];
expect(items.length).toBe(1);
expect(items[0].file_name).toBe('some.pdf');
}
One thing I've done in the past is to use an HTTP HEAD command. Basically, it's the same as a 'GET', but it only retrieves the headers.
Unfortunately, the web server needs to support 'HEAD' explicitly. If it does, you can actually try the URL and then check for 'application/pdf' in the Content-Type, without having to actually download the file.
If the server isn't set up to support HEAD, you can probably just check the link text like was suggested above.