Import data from CSV to a table in Oracle with NodeJs - javascript

I have a doubt.
I would like to know if it is possible to insert data (100K data) that I have in a CSV file, directly to a table in Oracle, using NodeJs.
I have looked for several ways, but I have not found a solution, only the use of external tables but the problem is that I must save the CSV file in a specific directory.

I don't know NodeJs, sorry.
But, from Oracle's side of story, you could use an external table (as you've already mentioned). If that "specific directory" is located on the database server, great! as you'd have to create a directory (an Oracle object) which points to that filesystem directory (and grant read and possibly write privileges to user which will be using that directory).
If it isn't located on the server, you'd still be able to do it by using UNC (universal naming convention).
Another option is SQL*Loader, a command-line tool which can be used on your local PC (you don't have to have access to and "directory" which is related to the database server in any way). You do have to have SQL*Loader installed, of course. It comes with every Oracle database; if you don't have it installed on that PC, you'd install Oracle Client.

There are various Node.js modules to read CSV files. Use one, and then when you have the data in Node.js, you can use node-oracledb's executeMany() to insert multiple rows at once into the DB, see the node-oracledb documentation Batch Statement Execution and Bulk Loading.
However, I would probably go with the earlier solution and use SQL*Loader, see https://blogs.oracle.com/opal/oracle-instant-client-122-now-has-sqlloader-and-data-pump

Related

JavaScript, Reading and Writing files for a locally hosted application

I have read several posts on this site that ask similar questions but the key difference is they involve a client and a server. For my use, this is not the case. I am simply pasting a file directory on my computer into my browser in order to view a local HTML file, packed with CSS and some jQuery.
I've been looking around and the answers I've found are "No; a client can not write to a server", and "No; a server can not write to a client". But there is no answer to "can a client write to a client with JavaScript?"
Use case:
I'm building a webapp (website? JS app?) as a college project for a stock management tool that will be locally hosted and never connect to the internet. Sure, I could knock one together in python in a couple hours, but I wouldn't learn anything. I need to create an access a txt file containing an array of the current stock of all the items so that when the application is loaded, the user doesn't have to manually enter anything but the changes to stock levels.
Honestly, I'm a beginner at JS and JQ and I'm only going off of what makes sense based on a mix of HTML and Python that I know.
Maybe PHP would be the better option for this particular option, or maybe JS will work well enough.
You still won't be doing client-to-client, your browser will just act as though the local file system is the server using the file:// protocol which means the same rules about a "client" (the browser) cannot write to a "server" (your local file system) apply.
If you wan't to be able to write an application that can interface directly with the filesystem, then look into something like Electron which is essentially an augmented website that gives you APIs to interact with the actual computer the app is being run on, including filesystem stuff.

MySQL bundled within an NPM app

The idea is to bundle the MySQL database within the npm app. The reason behind this is because the app is intended to run as a stand-alone desktop application built using Electron and, of course, many other modules.
So far, all of my searches for a MySQL database module only leads to the drivers. All other tutorials talk about the typical web app set up wherein the MySQL db is installed in the server. A comment from another post discussed how to set up MySQL db in server and just mentioned that if it's a desktop app, it's a whole other issue. I want to know if there is a way to do this or if it isn't possible at all.
After further research (and a lot of compromise in the initial design), I decided to use Lovefield by Google. It is the closest to what I have been looking for since 1. it is embedded in the browser and 2. structured query is how you get data. Still hoping there would be a solution to an embedded MySQL database in an Electron app.
Sometime in the past, I have managed to embed a full Drupal CMS installation inside of node-webkit (https://nwjs.io/). It was configured to proxy all the known PHP files to the embedded PHP installation. Then we have simply imported the sql dump file into sqlite and the entire thing worked out nicely. You might want to give that a try.
As #jd-hernandes pointed out, sqlite could be the answer you are looking for.

Html5 - Javascript - sqlite for intranet?

I need some database functionality for our intranet at work, but I am severely limited in my options. My only scripting tools are JavaScript/JQuery.
I was wondering if it would be possible to use SQLite with html5 and JS/JQ?
Thanks for all the pointers in the right direction!
Desired usage:
I wish to port in a large excel sheet in the db file, and run queries and return row results in the intranet page.
Yes it's possible, there are some libraries on the web. For example, the library sql.js is a port of SQLite to Javascript.
There are many options to solve this problem:
There are some packages, which can read data from XLSX file directly, for example: js-xlsx library, where you can retrieve data directly from Excel table (so, you do not need to use SQLite). It takes some time to read whole Excel, but you can keep your XLSX alive.
You can install simple node server for your intranet with SQLite package. This is a fastest way, because the client version of SQL.js loads whole file with database, which can be much bigger than original XLSX file.

Parse Microsoft Office files in Node.JS

I'm working on a web application where users can upload Microsoft Office Document files. Right now, our server is running Node.JS with Express.js and we're hosted on Heroku. Because of this, I don't think that I can install programs such as abiword or catdoc. I can handle the file uploads, but can't parse the contents of the document.
How can I read the contents of the doc file? The information will then be put into a database. It'd be nice to preserve basic formatting (bold, italic, underline), but not essential.
While there don't seem to be anything you can get with NPM that will do Word directly, you might be able to use a REST API to request it via another cloud service. For example Saaspose (they of the famous Aspose tools) have public API for Word, Excel, PDF, and others. They list node.js, javascript, and Heroku support on their page.
EDIT:
I see that Saaspose is now called Aspose for Cloud
Another API that claims something similar is Doxument
Office package: npm install office seems to provide at least part of the answer. I use it to read Excel files, so far have not tried any Word docs.
There doesn't seem to be any yet. See below for something that might help.
Can I read PDF or Word Docs with Node.js?
You can use mammoth to parse .docx files https://www.npmjs.com/package/mammoth
and xlsx to parse .xlsx files https://github.com/SheetJS/js-xlsx

Create database in memory from sql/csv files in Javascript

I am creating a product that as end result will/can create e.g. 10 .sql files, each being a table. The tables will contain various pre-calculated data related to each other.
My users will need to upload these to their website (php, asp, whatever) and will need to make something useful. Only problem, the users may have next to zero understanding of databases, server-side code etc. This means it must be very easy to configure.
So I think thinking upload these .sql (or CSV files, whatever) tables to server, so they are publicly available (i.e. can be retrieved like any other public URL). And then find a Javascript in-memory database engine that can load .sql database files. Does this exist?
I imagine a Javascript solution could work well if amount of data could be kept somewhat down... Otherwise I may need to look for a PHP/ASP solution as well. (Any ideas for libraries that can init in-memory databases from .sql or similar files?)
Preferably I should be able to re-distribute this Javascript library. (So users can get a complete "directory" of .sql files + example page + Javascript database engine to upload)
So to make the question clear: Anyone knows a Javascript-based in-memory database engine that can run inside browser?
If you wish to use javascript and need some 'userfriendly' bridge database, you could use json or xml, because the format are simple text files (like csv as well) for wich you can find smart small editors for your users.
More json is made for javascript parsing and has an understanding tree format, but you should load only some part of sql datas in memory, saying data buffers in xml or json, with php requested with some javascript ajax call. Php do the sql database access work and then you can output json, and with javascript, it is for user's interface, you'll be able to display them.
You can use mysql to store a database in memory:
http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html
Here's a pure JS SQL engine that stores everything in memory, https://github.com/moxley/sqittle
It flatly denies being useful for anything though, and has a limited set of supported commands (see readme on above link.
http://diveintohtml5.ep.io/storage.html might be what you are looking for.
That question seems very old. You might want to look at LokiJS now.

Categories