I have a Rails application that has some JavaScript that needs to parse CSVs and make some AJAX calls based on each record.
I'd like to just load the local CSV directly into browser memory and have the JavaScript parse it and make the required AJAX calls but I haven't found a cross-browser, dependable way to accomplish this (I need to support cruddy old IE6).
I could upload the CSV to my rails application but I plan on hosting the application on Heroku and as I understand it, Heroku doesn't allow you to edit the files system(create files). I could also write the CSVs to a database but these CSVs are large 10mb+ and I imagine I will undoubtedly suffer performance costs in doing this.
Is my best option pushing the CSV to Rails and having Rails respond with a JSON or string version of the CSV? This seems somewhat computationally expensive given the size of these CSVs. I'd prefer to keep it on the client-side. If that's the case can someone point me to an example on how to accomplish this or something similar?
Edit: I don't want users to have to copy and paste these CSVs into a textfield manually.
Edit2: Also, I'm aware of the security restrictions on accessing the local filesystem via JS. A solid flash embed is an acceptable option.
Related
I have created an interactive quiz using html and javascript which will be run on a touchscreen at an event and I need to write the results to a local csv file (so no internet connection). It needs to write to an already existing file, so it cannot be done where the data is stored locally and a download link is generated through the browser.
How would I go about doing this? All methods I have found are either unreliable or no longer supported. The browser I am using is Chrome, so it does not need to be cross-browser compatible.
Can anybody help or point me in the right direction please?
Install a web server.
Point the browser at http://localhost.
Send the data to the server using Ajax or a form submission.
Process the data (including storing it in a file) using the server side language of your choice.
When javascript is used only in the client-side cannot write data as you want.
Follow the #Quentin recommendation about install some web server, as apache using php for instance(It is pretty straight forward!). I also recommend you to create restFull methods to do it with jquery calls from the client side, it is easy to find many examples in the internet and quicky...
If you want something more easy you could work with html post using forms in php, the most easy way to do it.
I am trying to write some HTML/JS code which will facilitate uploading large files (multi-GB) to a remote server. Previously we had been using a flash uploader which uploaded a given file in a single network request. The flash would open a network connection, read a chunk of a file into memory then write that chunk to the network connection then grab the next chunk then write to the network etc. etc. until the entire file is uploaded. It was done this way because most web browsers will attempt to read an entire file into memory before attempting to upload. When dealing with multi-GB files, this essentially crashes the client system because it uses all of the client memory. Now we are having issues with using flash, so it needs to go, we want to replace it without needing to modify the existing server-side code.
A few google searches for jquery uploaders reveals that there are plenty of libraries which support "chunking" but they "chunk" over multiple requests. We do not want to chunk a file over multiple network requests, we merely want the JS to read the file in chunks as it writes the file to a single network connection.
Anybody know a library which can do this out of the box?
We are not opposed to modifying an existing library if need be. Anyone have a snippet that resembles the bellow pseudo-code that I may be able to retrofit into a library?
connection = fopen(...);
fputs("123", connection);
... some unrelated code ...
fputs("456", connection);
fclose(connection);
(excuse my use of C functions in pseudo JS code ... I know that is not how you do it in JS, I am merely demonstrating at a low-level the flow for how I want to write to the network connection before closing it)
NOTE: We are not trying to "modernize" or improve this project extensively -- we are not trying to re-do this project. We have some old code that has sat here for years and we want to make as few changes to the server-side code as possible. I have more important projects to modernize and make more efficient -- this one we just need to work. Please don't advise me to impliment "proper" file chunking on the server side -- that was my suggestion, and if my suggestion were taken then that task would have been assigned to a different developer. Out of my control now, this is a client-side-only fix please!
Thanks, sorry for any headache!
You could try binaryjs. I haven't looked into the internals but I know it supports manually setting the chunk size. Maybe you can even set it to Infinity.
Specifically you could try:
var client = new BinaryClient('example.com', { chunkSize: Number.POSITIVE_INFINITY });
client.send('data...');
Note: binaryjs is a NodeJS server library, and a browser-compatible client library.
I would like to implement an in-browser Microsoft Word document merge feature that will convert the merged document into PDF and offer it to the user for download. I would like to this process to be supported in Google Chrome and Firefox. Here is how I would like it to work:
Client-side JavaScript obtains the Word template document in docx format, either from a server, or by asking the user for a file upload (which it can then read using the FileReader API)
The JavaScript uses its local data structures (e.g., data lists it has obtained via Ajax) to expand the template into a document. It can do this either directly, by unzipping the docx file and processing its contents, or using DOCx.js. The template expansion is just a matter of substituting template variables with values obtained from the local data structures.
The JavaScript then converts the expanded template into PDF.
The JavaScript offers the PDF file to the user for download, e.g., using Downloadify.
The difficulty I am having is in step 3. My understanding (based on all the Googling I have done so far) is that I have the following options:
Require that the local machine is a Windows machine, and invoke Word on it, to convert to PDF. This can be done using a little bit of scripting using WScript.shell, and it looks doable with Internet Explorer. But based on what I have read, it doesn't look like I can call WScript.shell from within either Chrome or Firefox, because of their security constraints.
I am open to trying Silverlight to do the conversion, but I have not found enough documentation on how to do this. Ideally, if I used Silverlight, I would like to write the Silverlight code in JavaScript, because (a) I don't know much CSharp, and (b) I think it would be much easier in JavaScript.
Create a web service that will convert a given docx file to a pdf file, and invoke that service via Ajax. I would rather not do this, if possible, for a few reasons: (a) I tried using docx4java (I am a reasonably skilled Java programmer) but the conversion process is far too slow, and it does not preserve document content very well; and (b) I would like to avoid a call out to the network, to avoid security issues. It does seem possible to write a little service on a Windows server for doing the conversion, and if there is no other good option, I might go that route.
If I have been unclear about anything, please let me know. I would appreciate your ideas and feedback.
I love command line tools.
Load the doc to your server and use LibreOffice to convert it to PDF via the command line
soffice.exe --headless --convert-to pdf --outdir E:\Docs\Out E:\Docs\In\a.doc
You can display a progress bar to the user and when complete give them the option to download the doc.
More info on LibreOffice's command line parameters go here
Done.
Old old question now, but for anyone who stumbles across this, web assembly (wasm) now makes this sort of approach possible.
We've just released https://www.npmjs.com/package/#nativedocuments/docx-wasm which can perform the conversion locally.
I am creating a product that as end result will/can create e.g. 10 .sql files, each being a table. The tables will contain various pre-calculated data related to each other.
My users will need to upload these to their website (php, asp, whatever) and will need to make something useful. Only problem, the users may have next to zero understanding of databases, server-side code etc. This means it must be very easy to configure.
So I think thinking upload these .sql (or CSV files, whatever) tables to server, so they are publicly available (i.e. can be retrieved like any other public URL). And then find a Javascript in-memory database engine that can load .sql database files. Does this exist?
I imagine a Javascript solution could work well if amount of data could be kept somewhat down... Otherwise I may need to look for a PHP/ASP solution as well. (Any ideas for libraries that can init in-memory databases from .sql or similar files?)
Preferably I should be able to re-distribute this Javascript library. (So users can get a complete "directory" of .sql files + example page + Javascript database engine to upload)
So to make the question clear: Anyone knows a Javascript-based in-memory database engine that can run inside browser?
If you wish to use javascript and need some 'userfriendly' bridge database, you could use json or xml, because the format are simple text files (like csv as well) for wich you can find smart small editors for your users.
More json is made for javascript parsing and has an understanding tree format, but you should load only some part of sql datas in memory, saying data buffers in xml or json, with php requested with some javascript ajax call. Php do the sql database access work and then you can output json, and with javascript, it is for user's interface, you'll be able to display them.
You can use mysql to store a database in memory:
http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html
Here's a pure JS SQL engine that stores everything in memory, https://github.com/moxley/sqittle
It flatly denies being useful for anything though, and has a limited set of supported commands (see readme on above link.
http://diveintohtml5.ep.io/storage.html might be what you are looking for.
That question seems very old. You might want to look at LokiJS now.
I am trying to write a small web tool which takes an Excel file, parses the contents and then compares the data with another dataset. Can this be easily done in JavaScript? Is there a JavaScript library which does this?
How would you load a file into JavaScript in the first place?
In addition, Excel is a proprietary format and complex enough that server side libraries with years in development (such as Apache POI) haven't yet managed to correctly 100% reverse engineer these Microsoft formats.
So I think that the answer is that you can't.
Update: That is in pure JavaScript.
Update 2: It is now possible to load files in JavaScript: https://developer.mozilla.org/en-US/docs/DOM/FileReader
In the past four years, there have been many advancements. HTML5 File API has been embraced by the major browser vendors and performance enhancements actually make it somewhat possible to parse excel files (both xls and xlsx) in the browser.
My entries in this space:
http://oss.sheetjs.com/js-xls/ (xls)
http://oss.sheetjs.com/js-xlsx/ (xlsx)
Both are pure-JS parsers
To do everything in js, you'll have to use ActiveX and probably the office web components as well. Just a suggestion, but you probably don't want to go this route; it'll be inefficient and IE/Win only. You'll be better off with a server based solution.
You will need to use ActiveX (see W3C Schools on the use of AJAX) and register the file in the hosting computers Dataconnectors (only the computer hosting the file). Unlike mentioned before, this method is not Microsoft platform dependant (for the client anyways) and you do not need to have Office components installed.
This can be done for most datafiles registered in Windows, including MDB's, and allows you as much control as you want, as you can assign different Windows Accounts for different purposes.
Like I said before, this all is serverside and has no impact on the client, apart from maybe retrieving credentials, actions and all that.
This method uses JavaScript, SQL (no, not even MSSQL, just SQL standard) and requires only that the hosting computer is running ANY Microsoft NT platform.
What Windows dataconnectors do is provide a generalised interface for various data components much like DirectX does for videocards and other peripherals. You can also use it to link an MDB (Microsoft Access) to a MySQL server and feed data live that way, which I believe is even simpler than using XLS spreadsheets...especially since you can import XLS into MDB.
Do you really need an Excel file? Why not use Excel to export the data in CSV or XML and load that?
The Excel file format is very specific to Excel's implementation. If you just need the data, use a file format that just contains the data.