I created a web application to clean up CSV/TSV data. The app allows me to upload a CSV file, read it, fix data, and then download a new CSV file with the correct data. One challenge I have run into is downloading files with more than ~ 2500 lines. The browser crashes with the following error message:
"Aw, Snap! Something went wrong while displaying this webpage..."
To work around this I have changed the programming to download multiple CSV files not exceeding 2500 lines until all the data is downloaded. I would then put together the downloaded CSV files into a final file. That's not the solution I am looking for. Working with files of well over 100,000 lines, I need to download all contents in 1 file, and not 40. I also need a front-end solution.
Following is the code for downloading the CSV file. I am creating a hidden link, encoding the contents of data array (each element has 1000 lines) and creating the path for the hidden link. I then trigger a click on the link to start the download.
var startDownload = function (data){
var hiddenElement = document.createElement('a');
var path = 'data:attachment/tsv,';
for (i=0;i<data.length;i++){
path += encodeURI(data[i]);
}
hiddenElement.href = path;
hiddenElement.target = '_blank';
hiddenElement.download = 'result.tsv';
hiddenElement.click();
}
In my case the above process works for ~ 2500 lines at a time. If I attempt to download bigger files, the browser crashes. What am I doing wrong, and how can I download bigger files without crashing the browser? The file that is crashing the browser has (12,000 rows by 48 columns)
p.s. I am doing all of this in Google Chrome, which allows for file upload. So the solution should work in Chrome.
I've experienced this problem before and the solution I found was to use Blobs to download the CSV. Essentially, you turn the csv data into a Blob, then use the URL API to create a URL to use in the link, eg:
var blob = new Blob([data], { type: 'text/csv' });
var hiddenElement = document.createElement('a');
hiddenElement.href = window.URL.createObjectURL(blob);
Blobs aren't supported in IE9, but if you just need Chrome support you should be fine.
I also faced same problem. I used this code,it will works fine. You can also try this.
if (window.navigator.msSaveBlob) {
window.navigator.msSaveBlob(new Blob([base64toBlob($.base64.encode(excelFile), 'text/csv')]),'data.csv');
} else {
var link = document.createElement('a');
link.download = 'data.csv';
// If u use chrome u can use webkitURL in place of URL
link.href = window.URL.createObjectURL(new Blob([base64toBlob($.base64.encode(excelFile), 'text/csv')]));
link.click();
}
Related
I am using FileSaver.js to print an object array on the client side (HTML/Typescript).
var blob = new Blob([JSON.stringify( marray)], {type: "text/plain;charset=utf-8"});
saveAs(blob, "Data.txt");
It works fine. The problem is it downloads in the download folder (by default). I want to add a file path along with its name. Any idea? Or another way to do this job. fs is not working in this case. it is not recognizing fs and gives the error fs.writefilesync is not a function
How do you know what path the user has?
It is not safe. There is no way to "climb" the user's file system
If you open the source code, you will see the simplest implementation scheme there. It's just a click on a link download
var a = document.createElement('a')
// ...
a.href = blob
// ...
a.dispatchEvent(new MouseEvent('click'))
I'm trying to download an image using node.js and puppeteer but I'm running into some issues. I'm using a webscraper to gather the links of the images from the site and then using the https/http package to download the image.
This works for the images using http and https sources but some images have links that look like this (the whole link is very long so I cut the rest):
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAw8AAADGCAYAAACU07w3AAAZuUlEQVR4Ae3df4yU930n8Pcslu1I1PU17okdO1cLrTD+g8rNcvRyti6247K5NG5S5HOl5hA2uZ7du6RJEGYPTFy1Nv4RUJy0cWVkeQ9ErqqriHNrR8niZuVIbntBS886rBZWCGHVsNEFRQ5BloPCzGn2B+yzZMLyaP........
I'm not sure how to handle these links or how to download the image. Any help would be appreciated.
You need to first decode the url from base64 using node.js Buffer.
// the content type image/png has to be removed first
const data = 'iVBORw0KGgoAAAANSUhEUgAAAw8AAADGCAYAAACU07w3AAAZuUlEQVR4Ae3df4yU930n8Pcslu1I1PU17okdO1cLrTD+g8rNcvRyti6247K5NG5S5HOl5hA2uZ7du6RJEGYPTFy1Nv4RUJy0cWVkeQ9ErqqriHNrR8niZuVIbntBS886rBZWCGHVsNEFRQ5BloPCzGn2B+yzZMLyaP';
const buffer = new Buffer(data);
const base64data = buff.toString('base64');
// after this you will get the url string and continue to fetch the image
These are the base64 encoded images (mostly used for icons and small images).
you can ignore it.
if(url.startsWith('data:')){
//base 64 image
} else{
// an image url
}
if you really want to mess with base64 I can give you a workaround.
import { parseDataURI } from 'dauria';
import mimeTypes from 'mime-types';
const fileContent = parseDataURI(file);
// you probably need an extension for that image.
let ext = mimeTypes.extension(fileContent.MIME) || 'bin';
fs.writeFile("a random file"+"."+ext, fileContent.buffer, function (err) {
console.log(err); // writes out file without error, but it's not a valid image
});
I am facing issue to download pdf in SAPUI5 application. Issue is Getting base64 string from backend system but not able to convert it and display as PDF.
I am able to convert the base64 and download also but only small size.
Not able to download for larger PDF file its downloading but shows download failed.
kindly help me out
var data =" JVBERi0xLjQNJeLjz9MNCjc1MDEgMCBvYmogPDwvTGluZWFyaXplZCAxL0wgOTM2NDM1Mi9PIDc1MDMvRSAxMjE3ODgvTiA1MjIvVCA5MjE0MjgzL0ggWyA2..";
var uri = 'data:application/pdf;base64,' + atob(data);
var link = document.createElement("a");
link.href = uri;
link.style = "visibility:hidden";
link.download = object.FileName;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
Saving the data as a blob and setting the download link to get the data from the blog may solve your problem for large files. The most effective way in this mechanism is to get the data from your server as binary instead of Base64. It works with base64 too - but it is just a resource over kill in the blob scenario.
var data = Uint8Array.from(atob(base64_string), c => c.charCodeAt(0));
var blob = new Blob([data], {type: "octet/stream"});
var link = document.createElement("a");
link.href = window.URL.createObjectURL(blob);
...
...
As per you current solution, a hyperlink will be created with href contains data:application/pdf;base64,' + base64Data. When the hyperlink is clicked the complete URL will be opened in the browser new tab, which makes the browser to download the PFD file.
If the base64 data is bulk then the browser will take time to download PDF. Sometimes browser will be crashed OR leads to download failed error as it takes too much of time to download.
Alternative Options
Using GET_STEAM method you can download the pdf from the backend only.
Using download plugins like downloadjs, FileSaver.js, StreamSaver.js.
As per you requirement you can get different available plugins for file downloading using client-side JavaScript
Here is a sap blog entry solving your problem.
TLDR:
var base64EncodedPDF = "JVBERi0xLjcNCiW..."; // the encoded string
var decodedPdfContent = atob(base64EncodedPDF);
var byteArray = new Uint8Array(decodedPdfContent.length)
for(var i=0; i<decodedPdfContent.length; i++){
byteArray[i] = decodedPdfContent.charCodeAt(i);
}
var blob = new Blob([byteArray.buffer], { type: 'application/pdf' });
var _pdfurl = URL.createObjectURL(blob);
this._PDFViewer.setSource(_pdfurl);
It is about exporting extension data from options page.
I have array of objects, with stored page screenshots encoded in base64, and some other minor obj properties. I'm trying to export them with this code:
exp.onclick = expData;
function expData() {
chrome.storage.local.get('extData', function (result) {
var dataToSave = result.extData;
var strSt = JSON.stringify(dataToSave);
downloadFn('extData.txt', strSt);
});
}
function downloadFn(filename, text) {
var fLink = document.createElement('a');
fLink .setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(text));
fLink .setAttribute('download', filename);
fLink .click();
}
On button click, get data from storage, stringify it, create fake link, set attributes and click it.
Code works fine if resulting file is under ~1.7 MB, but everything above that produce option page to crash and extension gets disabled.
I can console.log(strSt) after JSON.stringify and everything works fine no matter of the size, if I don't pass it to download function..
Is there anything I can do to fix the code and avoid crash?...or is there any limitation is size when using this methods?
I solved this, as Xan suggested, switching to chrome.downloads (it's extra permission, but works fine)
What I did is just replacing code in downloadFN function, it's cleaner that way
function downloadFn(filename, text) {
var eucTxt = encodeURIComponent(text);
chrome.downloads.download({'url': 'data:text/plain;charset=utf-8,'+eucTxt, 'saveAs': false, 'filename': filename});
}
note that using URL.createObjectURL(new Blob([ text ])) also produce same crashing of extension
EDIT:
as #dandavis pointed (and RobW confirmed), converting to Blob also works
(I had messed code that was producing crash)
This is a better way of saving data locally, because on browser internal downloads page, dataURL downloads can clutter page and if file is too big (long URL), it crashes browser. They are presented as actual URLs (which is raw saved data) while blob downloads are only with id
function downloadFn(filename, text) {
var vLink = document.createElement('a'),
vBlob = new Blob([text], {type: "octet/stream"}),
vUrl = window.URL.createObjectURL(vBlob);
vLink.setAttribute('href', vUrl);
vLink.setAttribute('download', filename);
vLink.click();
}
As IE url limit is 2083 characters (see here). I have problems creating a csv file to download:
My script fails in this line: link.href = uri;
where var uri = 'data:text/csv;charset=utf-8,' + escape(CSV);
and CSV is actually a string with the content of the file.
So, how could I resolve this issue without limiting the file content to 2083 characters?
Many thanks
Unfortunately, there isn't really a way without creating the file on a server and sending it. If you need to stay client-side, putting the csv data in a textbox is your best hope.
UPDATE: Just had a quick think about this, you might have better luck if you put your data-uri as the href of an a element, aka
var ae = document.createElement 'a';
ae.href = 'data:text/csv;charset=utf-8,' + escape(CSV);
document.appendChild(ae);
var downloadCSV = document.createElement('a');
downloadCSV.setAttribute('href', 'data:text/csv;charset=utf-8,' + encodeURIComponent(finalOutput));
downloadCSV.setAttribute('download', 'Report.csv');
downloadCSV.click();
Try this, I've had some success with it in chrome, I think you can pass up to 2mb with it.
I've also encountered a lot of problems in creating a CSV on the client side...