So I created a desktop electron application with javascript html css etc I have a bot that I want to run when a button is clicked by the user the bot is written in python. what the bot does is web scraping using selenium and chrome driver im just wondering is there a way where I could store the bot and its source code outside the clients computer so the source code is not visible and still give the client the ability to use the bot to webscrape.
sorry if this is a rookie question im coming from c++ & swift mobile development and im a junior CS student so im just teaching myself new stuff.
I agree with Chris G in that it would be considered best practice to create a web app with one of Python's many web frameworks (Django, FastAPI, Flask, etc).
Alternatively, with the python-shell package this can be done quite simply with electron:
const { app, BrowserWindow } = require('electron');
const pyshell = require('python-shell')
function createWindow() {
window = new BrowserWindow({ width: 600, height: 450 });
window.loadFile('index.html');
pyshell.run('your_script.py', function (err, results) {
if (err) {
throw err;
}
});
}
app.on('ready', createWindow);
app.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
app.quit()
}
}
Source
Then, with a simple python script your_script.py:
a = 'Foo'
b = 'Bar'
print(a + b)
This example is quite simple. Creating your own web facing API would be your best bet if you don't want to run into any compatibility issues when shipping your app.
I've tried to use the microsoft teams client SDK to get access to the current context of the tab. As of yet I haven't been able to get the getContext callback to run let alone return a context that I can use in any meaningful way.
I've tried to create a static html page that uses the client SDK by embedding the script and then log the context of the object. The SDK properly returns an object of the expected shape, but using initialize() then getContext() does nothing. I'm making this website as a tab inside of Microsoft teams creating a new tab and making it a website tab.
<p>hello World!</p>
<script src="https://statics.teams.microsoft.com/sdk/v1.4.2/js/MicrosoftTeams.min.js" crossorigin="anonymous"></script>
<script>
console.log(microsoftTeams);
microsoftTeams.initialize();
microsoftTeams.getContext((context) => console.log(context));
</script>
This is the static page I'm using to try and get context logged out to the console.
const http = require("http");
const fs = require("fs");
this.html;
fs.readFile("./src/index.html", (err, html) => {
if (err) {
throw new Error(err);
} else {
this.html = html;
}
})
http.createServer((req, res) => {
res.write(this.html);
res.end();
}).listen(7777);
This is my server I'm using to return the static html
I'm using ngrok to provide a tunnel from the web to my local server on port 7777. I can get the page to load just fine, but have yet to get the getContext() to run my callback function.
I expect to be able to get the context of the teams tab in the callback of the getContext() function, but that function never runs.
Moving answer from Comments section:
Tab created using default Website App doesn't get the Teams context, it needs to be a trusted URL. You need to build custom Tab to get context for your Microsoft Teams tab.
Please take a look at steps to create Teams App manifest using App Studio.
In the default application specified Hello World with Node.js if you put this script in the end of body tag.
console.log(microsoftTeams);
microsoftTeams.initialize();
microsoftTeams.getContext((context) => console.log(context));
Now you have to run this application in Teams environment only then this context method will get a value and not on regular html page.
I have PWA app in which I am rendering js files using Webpack:
{% render_bundle 'app' 'js' %}
After launching the PWA app in mobile Chrome the file is not updated. Most probably Chrome uses cached version.
I tried to delete PWA app and install it again but it did not help.
Afterwards I have cleared the mobile Chrome cache manually and files were refreshed, however, most of the users won't do it so I need another solution which does not require any actions from end users.
Answers on similar question suggest to add parameter or version number to the js file.
<script type="text/javascript" src="myfile.js?REVISION"></script>
However, it is not clear how can I do it using Webpack?
One more popular answer explains that I can use hash or chunkhash to generate file name using Webpack:
output: {
path: '/',
filename: '[hash].js',
chunkFilename: '[chunkhash].js',
},
This solution won't work for me because I cannot change the name of the file every time when there are some chnages in it. The name of the file should stay the same because I use django's collectfast app. It checks the md5sum of static files and updates only those ones which have been changed.
The name of the static js file should stay the same. At the same time, I need mechanism which will force mobile Chrome to update changed file.
if ('serviceWorker' in navigator) {
navigator.serviceWorker.register('service-worker.js')
.then(reg => {
reg.onupdatefound = () => {
const installingWorker = reg.installing;
installingWorker.onstatechange = () => {
switch (installingWorker.state) {
case 'installed':
if (navigator.serviceWorker.controller) {
// new update available
setTimeout(() => document.location.reload(true), 1000);
caches.keys().then(keys => {
keys.forEach(key => caches.delete(key));
})
}
break;
}
};
};
})
}
I was trying to do the same thing with Svelte PWA code here:
https://github.com/kuhlaid/svelte2/releases/tag/v0.1.6
I resorted to running the app build process and then using the 'replace-in-file' plugin (see the rollup.config.js script). If you search the source code for '__cVersion__' you will see where I am adding the file revision string to try and force a file cache update (not localstorage).
The OTHER thing that needs to be done for a PWA is making sure we clear the CacheStorage in the users browser if you are building your service working with something like Workbox to precache files within the service worker. This is something you are NOT ABLE to do during the build process since the code to clear the CacheStorage needs to be run at time of accessing the app in the browser. To clear this cache you can insert something along these lines in the javascript of your app:
const l = console.log
if ('caches' in window) {
l('CacheStorage is present for this app');
caches.keys().then(function(cacheArray) {
l(cacheArray); // just print the list of CacheStorage names to the console
// for each cache, try and delete it
cacheArray.forEach(function(cache) {
caches.delete(cache).then((bool) => {
l('deleted cache: '+cache); // print the successful deletion to console
}).catch((err) => {l(err)});
});
});
}
This is all well and good, BUT this begs the next question of how you only execute this ONCE for a new code build/update? Well, possibly a 'code version' variable could be added to your javascript somewhere like:
const codeVersion = __cVersion__;
Then during the build/rollup of your code, you dynamically replace __cVersion__ with your new version string (eg. v0.112) and store this codeVersion value to localStorage for the app. Then each time you build you would check localStorage first for a change in the version string, and if it has changed then run the code to delete the CacheStorage for the app (as mentioned above). This version of my PWA code handles these cases:
https://github.com/kuhlaid/svelte2/releases/tag/v0.1.7
Automatically clearing the cache for end-users is one of the complexities of PWA (or any app) that should be understood upfront by the developer. No one likes an app where you are told by the developer that YOU need to manually clear your browser cache each time they push a code update.
you could try putting one of these in your html,
that would force cache to expire.
<meta http-equiv="cache-control" content="max-age=0" />
<meta http-equiv="cache-control" content="no-cache" />
<meta http-equiv="expires" content="0" />
<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT" />
<meta http-equiv="pragma" content="no-cache" />
I know that client-side Javascript cannot write data to the local filesystem, for obvious security reasons.
The only way to save data locally with Javascript seems to be with cookies, localStorage, or allow the user to download a file (with a "Save..." dialog box or to the browser's default Download folder).
But is it possible, in the very specific case when the file is accessed locally with an URL like file:///D:/test/index.html (and not through internet) to write data locally ? (without any server language, and even without any server at all: just local browsing of a HTML file)
For example, would it be possible, by clicking on SAVE here:
<div contenteditable="true" style="height:200px;">Content editable - edit me and save!</div>
<button>Save</button>
...that such a HTML file (accessed through file:///D:/test/index.html) is overwritten with its new content ? (i.e. the local HTML file should be updated when SAVE is pressed).
TL;DR: Is this possible to SAVE a file thanks to Javascript, when the HTML page is accessed locally?
Note: I want to be able to silently save, not propose a Download/Save dialog box in which the user has to choose where to download, then "Are you sure to want to overwrite" etc.
EDIT : Why this question? I'm doing an in-browser notepad that I can run locally without any server (no Apache, no PHP). I need to be able to save easily without having to deal with Dialog Box "Where do you want to download the file?" and having to always re-browse to the same folder to overwrite the currently-being-edited file. I would like a simple UX like in any notepad program: CTRL+S done, the current file is saved! (example: MS Word doesn't ask to browse where you want to save the file each time you do "Save": CTRL+S, done!)
You can just use the Blob function:
function save() {
var htmlContent = ["your-content-here"];
var bl = new Blob(htmlContent, {type: "text/html"});
var a = document.createElement("a");
a.href = URL.createObjectURL(bl);
a.download = "your-download-name-here.html";
a.hidden = true;
document.body.appendChild(a);
a.innerHTML = "something random - nobody will see this, it doesn't matter what you put here";
a.click();
}
and your file will save.
The canonical answer, from the W3C File API Standard:
User agents should provide an API exposed to script that exposes the features above. The user is notified by UI anytime interaction with the file system takes place, giving the user full ability to cancel or abort the transaction. The user is notified of any file selections, and can cancel these. No invocations to these APIs occur silently without user intervention.
Basically, because of security settings, any time you download a file, the browser will make sure the user actually wants to save the file. Browsers don't really differentiate JavaScript on your computer and JavaScript from a web server. The only difference is how the browser accesses the file, so storing the page locally will not make a difference.
Workarounds:
However, you could just store the innerHTML of the <div> in a cookie. When the user gets back, you can load it back from the cookie. Although it isn't exactly saving the file to the user's computer, it should have the same effect as overwriting the file. When the user gets back, they will see what they entered the last time. The disadvantage is that, if the user clears their website data, their information will be lost. Since ignoring a user's request to clear local storage is also a security problem, there really is no way around it.
However, you could also do the following:
Use a Java applet
Use some other kind of applet
Create a desktop (non-Web based) application
Just remember to save the file when you clear your website data. You can create an alert that pops up and reminds you, or even opens the save window for you, when you exit the page.
Using cookies: You can use JavaScript cookies on a local page. Just put this in a file and open it in your browser:
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<p id="timesVisited"></p>
<script type="text/javascript">
var timesVisited = parseInt(document.cookie.split("=")[1]);
if (isNaN(timesVisited)) timesVisited = 0;
timesVisited++;
document.cookie = "timesVisited=" + timesVisited;
document.getElementById("timesVisited").innerHTML = "You ran this snippet " + timesVisited + " times.";
</script>
</body>
</html>
Chromium's File System Access API (introduced in 2019)
There's a relatively new, non-standard File System Access API (not to be confused with the earlier File and Directory Entries API or the File System API). It looks like it was introduced in 2019/2020 in Chromium/Chrome, and doesn't have support in Firefox or Safari.
When using this API, a locally opened page can open/save other local files and use the files' data in the page. It does require initial permission to save, but while the user is on the page, subsequent saves of specific files do so 'silently'. A user can also grant permission to a specific directory, in which subsequent reads and writes to that directory don't require approval. Approval is needed again after the user closes all the tabs to the web page and reopens the page.
You can read more about this newish API at https://web.dev/file-system-access/. It's meant to be used to make more powerful web applications.
A few things to note about it:
By default, it requires a secure context to run. Running it on https, localhost, or through file:// should work.
You can get a file handle from dragging and dropping a file by using DataTransferItem.getAsFileSystemHandle
Initially reading or saving a file requires user approval and can only be initiated via a user interaction. After that, subsequent reads and saves don't need approval, until the site is opened again.
Handles to files can be saved in the page (so if you were editing 'path/to/file.html', and reload the page, it would be able to have a reference to the file). They can't seemingly be stringified, so are stored through something like IndexedDB (see this answer for more info). Using stored handles to read/write requires user interaction and user approval.
Here are some simple examples. They don't seem to run in a cross-domain iframe, so you probably need to save them as an html file and open them up in Chrome/Chromium.
Opening and Saving, with Drag and Drop (no external libraries):
<body>
<div><button id="open">Open</button><button id="save">Save</button></div>
<textarea id="editor" rows=10 cols=40></textarea>
<script>
let openButton = document.getElementById('open');
let saveButton = document.getElementById('save');
let editor = document.getElementById('editor');
let fileHandle;
async function openFile() {
try {
[fileHandle] = await window.showOpenFilePicker();
await restoreFromFile(fileHandle);
} catch (e) {
// might be user canceled
}
}
async function restoreFromFile() {
let file = await fileHandle.getFile();
let text = await file.text();
editor.value = text;
}
async function saveFile() {
var saveValue = editor.value;
if (!fileHandle) {
try {
fileHandle = await window.showSaveFilePicker();
} catch (e) {
// might be user canceled
}
}
if (!fileHandle || !await verifyPermissions(fileHandle)) {
return;
}
let writableStream = await fileHandle.createWritable();
await writableStream.write(saveValue);
await writableStream.close();
}
async function verifyPermissions(handle) {
if (await handle.queryPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
if (await handle.requestPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
return false;
}
document.body.addEventListener('dragover', function (e) {
e.preventDefault();
});
document.body.addEventListener('drop', async function (e) {
e.preventDefault();
for (const item of e.dataTransfer.items) {
if (item.kind === 'file') {
let entry = await item.getAsFileSystemHandle();
if (entry.kind === 'file') {
fileHandle = entry;
restoreFromFile();
} else if (entry.kind === 'directory') {
// handle directory
}
}
}
});
openButton.addEventListener('click', openFile);
saveButton.addEventListener('click', saveFile);
</script>
</body>
Storing and Retrieving a File Handle using idb-keyval:
Storing file handles can be tricky, since they can't be unstringified, though apparently they can be used with IndexedDB and mostly with history.state. For this example we'll use idb-keyval to access IndexedDB to store a file handle. To see it work, open or save a file, and then reload the page and press the 'Restore' button. This example uses some code from https://stackoverflow.com/a/65938910/.
<body>
<script src="https://unpkg.com/idb-keyval#6.1.0/dist/umd.js"></script>
<div><button id="restore" style="display:none">Restore</button><button id="open">Open</button><button id="save">Save</button></div>
<textarea id="editor" rows=10 cols=40></textarea>
<script>
let restoreButton = document.getElementById('restore');
let openButton = document.getElementById('open');
let saveButton = document.getElementById('save');
let editor = document.getElementById('editor');
let fileHandle;
async function openFile() {
try {
[fileHandle] = await window.showOpenFilePicker();
await restoreFromFile(fileHandle);
} catch (e) {
// might be user canceled
}
}
async function restoreFromFile() {
let file = await fileHandle.getFile();
let text = await file.text();
await idbKeyval.set('file', fileHandle);
editor.value = text;
restoreButton.style.display = 'none';
}
async function saveFile() {
var saveValue = editor.value;
if (!fileHandle) {
try {
fileHandle = await window.showSaveFilePicker();
await idbKeyval.set('file', fileHandle);
} catch (e) {
// might be user canceled
}
}
if (!fileHandle || !await verifyPermissions(fileHandle)) {
return;
}
let writableStream = await fileHandle.createWritable();
await writableStream.write(saveValue);
await writableStream.close();
restoreButton.style.display = 'none';
}
async function verifyPermissions(handle) {
if (await handle.queryPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
if (await handle.requestPermission({ mode: 'readwrite' }) === 'granted') {
return true;
}
return false;
}
async function init() {
var previousFileHandle = await idbKeyval.get('file');
if (previousFileHandle) {
restoreButton.style.display = 'inline-block';
restoreButton.addEventListener('click', async function (e) {
if (await verifyPermissions(previousFileHandle)) {
fileHandle = previousFileHandle;
await restoreFromFile();
}
});
}
document.body.addEventListener('dragover', function (e) {
e.preventDefault();
});
document.body.addEventListener('drop', async function (e) {
e.preventDefault();
for (const item of e.dataTransfer.items) {
console.log(item);
if (item.kind === 'file') {
let entry = await item.getAsFileSystemHandle();
if (entry.kind === 'file') {
fileHandle = entry;
restoreFromFile();
} else if (entry.kind === 'directory') {
// handle directory
}
}
}
});
openButton.addEventListener('click', openFile);
saveButton.addEventListener('click', saveFile);
}
init();
</script>
</body>
Additional Notes
Firefox and Safari support seems to be unlikely, at least in the near term. See https://github.com/mozilla/standards-positions/issues/154 and https://lists.webkit.org/pipermail/webkit-dev/2020-August/031362.html
Yes, it's possible.
In your example, you are already using ContentEditable and most of tutorials for that attribute have some sort of localStrorage example, ie. http://www.html5tuts.co.uk/demos/localstorage/
On page load, script should check localStorage for data and if true, populate element. Any changes in content could be saved in localStorage when clicking save button (or automatically, in linked example, using blur and focus). Additionally you can use this snippet to check weather user is online or offline and based on state modify your logic:
// check if online/offline
// http://www.kirupa.com/html5/check_if_internet_connection_exists_in_javascript.htm
function doesConnectionExist() {
var xhr = new XMLHttpRequest();
var file = "http://www.yoursite.com/somefile.png";
var randomNum = Math.round(Math.random() * 10000);
xhr.open('HEAD', file + "?rand=" + randomNum, false);
try {
xhr.send();
if (xhr.status >= 200 && xhr.status < 304) {
return true;
} else {
return false;
}
} catch (e) {
return false;
}
}
EDIT: More advance version of localStorage is Mozilla localForage which allows storing other types of data besides strings.
You could save files, and make it persistent using the FileSystem-API and webkit. You would have to use a chrome browser and it is not a standards technology but I think it does exactly what you want it to do. Here is a great tutorial to show how to do just that http://www.noupe.com/design/html5-filesystem-api-create-files-store-locally-using-javascript-webkit.html
And to show that its on topic it starts off showing you how to make the file save persistent...
window.webkitRequestFileSystem(window.PERSISTENT , 1024*1024, SaveDatFileBro);
Convert your HTML content to a data uri string, and set as href attribute of an anchor element. Don't forget to specify a filename as download attribute.
Here's a simple example:
<a>click to download</a>
<script>
var anchor = document.querySelector('a');
anchor.setAttribute('download', 'example.html');
anchor.setAttribute('href', 'data:text/html;charset=UTF-8,<p>asdf</p>');
</script>
Just try it in your browser, no server required.
Have a look into this :)
Download File Using Javascript/jQuery
there should be everything you need. If you still need help or it's not the solution you need, tell me ;)
Yes, it is possible. Proof by example:
TiddlyFox: allows modification of local files via an add-on. (source code) (extension page):
TiddlyFox is an extension for Mozilla Firefox that enables TiddlyWiki
to save changes directly to the file system.
Todo.html: An HTML file that saves edits to itself. Currently, it only works in Internet Explorer and you have to confirm some security dialogs when first opening the file. (source code) (functional demo).
Steps to confirm todo.html actually saves changes to itself locally:
Save todo.html to local harddrive
Open with Internet Explorer. Accept all the security dialogs.
Type command todo add TEST (todo.html emulates the command-line interface of todo.txt-CLI)
Inspect todo.html file for addition of 'TEST'
Caveats: there is no cross-platform method. I'm not sure how much longer these methods will exist. When I first started my todo.html project, there was a jQuery plugin called twFile that allowed cross-browser loading/saving of local files using four different methods (ActiveX, Mozilla XUL, Java applet, Java Live Connect). Except for ActiveX, browsers have disallowed all these methods due to security concerns.
If you are fine with your code running outside of the scope of your default browser, and you are fine with windows only support, HTAs meet the silently save without prompts requirement easily.
The below code doesn't use many HTA specific features but it does still use microsoft specific stuff like ActiveXObject("Scripting.FileSystemObject").
<html>
<head>
<title>Simple Notepad</title>
<meta http-equiv="X-UA-Compatible" content="IE=9">
<script>
document.addEventListener('keydown', function (event) {
if (event.ctrlKey) {
if (event.key == 's') {
var FSo = new ActiveXObject("Scripting.FileSystemObject");
//see https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/opentextfile-method
var thisFile = FSo.OpenTextFile(window.location.pathname, 2, true, -1);
thisFile.Write(document.getElementsByTagName("html")[0].outerHTML);
thisFile.Close();
// Comment out the below alert to get truly silent saving.
alert('Saved Successfully');
if (event.preventDefault) event.preventDefault();
return false;
}
}
}, false);
</script>
</head>
<body contentEditable="true">
<h1>Press <kbd>CTRL + S</kbd> To Save</h1>
</body>
</html>
It also isn't a very rich editing experience but that can be fixed with some more buttons or keyboard shortcuts I think. Like CTRL + B to embolden selected text. It doesn't have any safety checks as of yet, but binding an event handler to beforeunload should prevent any data loss caused by accidentally closing the program.
HTA's do have other disadvantages too. They don't support ES6 (though transpiling is an option).
Although it is a bit dated, If you're not trying to use modern web features, I think you'll agree that it is very functional and usable.
Edit
I forgot to mention, but HTAs have to be saved with the .hta file extension for mshta.exe to be registered as their file type handler. Which is needed so that you can double click it in windows explorer to open it easily.
See also
Introduction to HTML Applications on MSDN
HTML Applications reference on MSDN
I think it's important to clarify the difference between server and client in this context.
Client/server is a program relationship in which one program (the client) requests a service or resource from another program (the server).
Source: http://searchnetworking.techtarget.com/definition/client-server
I'm not sure you'll find too many advanced applications that don't have at least one server/client relationship. It is somewhat misleading to ask to achieve this without any server, because any time your program speaks to another program, it is a client/server relationship, with the requester being the client and the response coming from the server. This is even if you are working locally. When you want to do something outside of the scope of the browser, you need a hook in a server.
Now, that does not mean you can't achieve this without a server-side specific language. For example, this solution uses NodeJS for the server. WinJS has WinJS.xhr, which uses XmlHttpRequest to serve data to the server.
AJAX seeks to offer the same sort of functions. The point here is that whether you have a program or there is some sort of hook pre-built, when you issue a command like "save file" and the file actually gets saved, there is a program on the other side that is parsing it, whether it's a server-side language or something else, meaning you can't possibly have something like this function without a server to receive the request.
Just use https://github.com/firebase/firepad — See it in action
This doesn’t need a server on your computer, it will reach out and save the data remotely.
Use jsPDF -> https://github.com/MrRio/jsPDF
<div id="content">
<h3>Hello, this is a H3 tag</h3>
<p>a pararaph</p>
</div>
<div id="editor"></div>
<button id="cmd">generate PDF</button>
Javascript
var doc = new jsPDF();
var specialElementHandlers = {
'#editor': function (element, renderer) {
return true;
}
};
$('#cmd').click(function () {
doc.fromHTML($('#content').html(), 15, 15, {
'width': 170,
'elementHandlers': specialElementHandlers
});
doc.save('sample-file.pdf');
});
This is an example for those who want to know how to use the localStorage.
<div id="divInput" contenteditable="true" style="height:200px;border: 2px solid blue">
Content editable - edit me and save!
</div>
<button onclick="onSave()">Save</button>
<button onclick="onLoad()">Load</button>
<script>
config = {
localStorageItemName: "demo",
datetimeFormat: {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
hour12: false,
minute: '2-digit',
second: '2-digit'
}
}
function Now() {
return new Date().toLocaleString("zh-TW", config.datetimeFormat)
}
const errMap = {
IsEmptyError: new Error('is empty'),
LengthError: new Error('length = 0')
}
/**
* #param {string} itemName
* #return {Object}
* */
function getLocalStorageItem(itemName) {
const dbDataString = localStorage.getItem(itemName)
if (dbDataString === null) {
throw errMap.IsEmptyError
}
const db = JSON.parse(dbDataString)
if (Object.keys(db).length === 0) {
throw errMap.LengthError
}
return db
}
function onSave() {
const inputValue = document.querySelector(`#divInput`).textContent
try {
const db = getLocalStorageItem(config.localStorageItemName)
db.msg = inputValue
db.lastModTime = Now()
localStorage.setItem(config.localStorageItemName, JSON.stringify(db))
console.log("save OK!")
} catch (err) {
switch (err) {
case errMap.IsEmptyError:
console.info("new localStorageItemName")
localStorage.setItem(config.localStorageItemName,
JSON.stringify({msg: inputValue, createTime: Now()})
)
break
/*
case ...
break
*/
default:
console.error(err.message)
}
}
}
function onLoad(e) {
try {
const db = getLocalStorageItem(config.localStorageItemName)
console.log("load")
document.querySelector(`#divInput`).textContent = db.msg
} catch (err) {
return
}
}
(()=>{
window.onload = () => (
onLoad()
)
})()
</script>
It is written in pure javascript with no dependencies.