node.js flatfile database using a CSV file and an array? - javascript

I have been using SQL for quite a while now, for a node.js project, I wanted to make the package smaller, and easier to manage, I thought by getting rid of MYSQl I can have put the database as part of the server, using a CSV file, or or something similar, and a variable array, how could I archive this?
I am yet to try any code, but this is how I am planning on going about it at the moment:
A basic express web server using the GET/POST requests, import a CSV file to a array, then do a basic variable comparison, like say, if (vararray === "foo"){
return "foo" exists in the database.}
I am still relatively new to javascript.

Related

Polymer 2.0 - Can you write to fs?

I'm working on an MVP for a project, and I'm trying to mock up a "database" quick and dirty. I thought for now I'll just put my "database" into a .json file and work with that. I am able to use iron-ajax to get a file read in to a Polymer property to be manipulated, however, I don't know how I could write it back onto the filesystem once I manipulated it. I tried
let fs = require('fs');
fs.writeFile('./db/db.json', json, 'utf8');
However, this does not work (apparently, require does not work on the client side). I've tried googling around and checking the answers on the linked thread, but the answers are quite vague ("use <script> tag" - okay, but how?) and I haven't been able to figure it out. How would I be able to pass a json object and write it back to the filesystem?
Simple answer is no you can't. You have to write a little backend for that but for that i usually use the localStorage there you can store JSON and read write.

Pushing to an array in a separate JSON file

Lets say I have a JSON file, with an array inside of that JSON file.
{
"users": ["288381238123", "12312123123"]
}
Now I want to push to that array, and I have that file required at the top
const userList = require('../rolecall.json');
Then I have some other code here, which is seemingly supposed to push to that array
const users = userList.users;
users.push('82313123');
Now when I check back to the JSON file, there is no addition to the array. Why is that? If anyone could help me, that would be great.
When you 'push' to the array in Javascript, you are only changing the Javascript object.
Edit: taking Lyon's comment into account
The short answer is, you can't change a file on the client machine using javascript. That would be a huge security issue (though it used to be possible, fun times).
However, you can ask the browser to display a save dialog asking the user whether they want to save the content of your json to a file.
Old answer
If you want to save the changes to a file located on the server, you need to send the informations to said server, preferably via a POST request.
A simple way to do it is to use Ajax requests.
You can use the JSON.stringify method to format your users object.
xhttp.send(JSON.stringify(users));
Then you can handle the request on the server. The way to do that depends on the language used (PHP, Ruby, Node, etc)
Observations :
You are pushing a element into a local array userList. If you want to update the rolecall.json file, you'll need to write back to that file.
Javascript (client side) has not functionality to create, edit etc.. files.You can do it in back-end side.
Workaround :
You got your json in userList constant variable. now you can push that element into the array and then write back to rolecall.json if you're manipulating it in your backend (ie.: NodeJs)
const userList = {
"users": ["288381238123", "12312123123"]
};
userList.users.push('82313123');
console.log(userList);
In node.js you can use the fs library.
fs = require('fs');
fs.writeFile('fileName.json', JSON.stringify(userList));

Create d3.js graph from data on mongodb server

How can I create d3.js graph from data on mongodb server using node.js?
D3.js includes ways to request non-local data either as json or text (csv) via urls and such.
In a setup that is not security sensitive (like local development or a demo environment) you could fairly directly use the mongo rest api if you enable it, which will give you json output for objects.
Or you could write build a simple http server (like in python, perl or go) that execs (python (also subprocess), perl (also backticks and qx{}), go) the mongoexport tool with the right parameters to provide csv output from mongo.
If you already have data in Mongo, and you've got Node already setup, then maybe that's what you want to use:
⇒ ⇒
If so, there's someone out there that's used Node.js® with some npm modules for MongoDB® to specifically drive a D3.js® visualization.

SQLite with JavaScript

I'm working on a small project where I'm going to read some parameters from a SQLite database. The data is generated from a Linux server running a C code. I then want to create a script using JavaScript to fetch the data from the database. I've tried with alasql.js but it takes very long time (~1 minute) before I get the list with the parameters from two tables.
SELECT sensors.id, sensors.sensorname, sensors.sensornumber, information.sensorvalue, information.timestamp FROM sensors INNER JOIN information ON sensors.id=information.sensorid
I've been reading about IndexedDB but seems like it only works with JavaScript but not with C-code. Please, correct me if I'm wrong. The clue here is that I want a database that supports writing to database from C-code and reading from database from JavaScript. The database can be read either from file:// schema or an IP address.
Would appreciate any help regarding this problem. Thanks!
Unfortunately, SQLite internal file format is complicated to fast parsing with JavaScript. One of the reasons, that the only browser side library which can read it is SQL.js and it is relatively slow, because it can not read data by selected pages from the database file, but only the whole file.
One of the options: you can switch from SQLite format to CSV or TSV plain tet formats, which can be eaily and quickly send to the browser and be parsed with jQuery.CSV, PapaParse, AlaSQL or any other CSV parsing libraries, like:
alasql('SELECT * FROM TSV("mydata.txt",{headers:true})',[],function(data){
// data
});
Another alternative: you can write simple server on node.js with native support of SQLite and then provide requested records in JSON format with ajax to your application (like in this article)

Organize Csv file based on filestructure using JavaScript.

Here's the whole problem:
I need to create a plain csv table from a file structure from the server.
Everyline should be organized as follows:
,
Currently, all the files on the server are organized like this:
L:\Dados\rd\20110727000002978\110614.pdf
Where the string 20110727000002978 is the docid and L:\Dados\rd\20110727000002978\110614.pdf is the file path.
So, the CSV table should have the data like this
L:\Dados\rd\20110727000002978\110614.pdf, 20110727000002978
Currently I'm using a BATCH file to do a DIR /S/B > FILELIST.TXT to list all the files, and MANUALLY updating the
Is it possible to automate this using javascript? I guess it would be a simple script, but I had no idea how to start this.
I guess I don't need to use JavaScript, as I long as I don't need any copyrighted copiler - I guess I can use anything.
As far as I know, the code itself is rather simple, but all programming I know is BATCH and a little PASCAL.
If I understand your question, you won't be able to do this entirely in JavaScript. You'll need a sever-side language, like PHP (as an example) to do the file processing.

Categories