The following code is inside <script> tag of index.html of my application and it works fine (basically the code is inside a function which gets called when a user clicks on a download button from the UI;the files are already present on the server/location mentioned below) when I access my app via web browser in the following manner:
file:///C:/jack/testing/ui/static/index.html
By works, I mean, I can see the 3 files getting downloaded in csv format with 3KB of size.
const requests = [
'file1_1555077233.csv',
'file2_1555077233.csv',
'file3_1555077233.csv'
].map(file => {
return fetch('file:///C:/jack/file/JACK/' + file)
.then(response => response.text())
.catch(console.error)
})
Promise.all(requests)
.then(contents => {
const zip = new JSZip()
contents.forEach((content, index) => {
zip.file(`file-${index}.csv`, content)
})
return zip.generateAsync({ type: 'blob' })
})
.then(blob => {
saveAs(blob, 'files.zip')
})
Scenario 1:
However, when I have my application deployed as WAR file in tomcat, I access my application in the following manner from the web browser :
http://localhost:8080/testing/index.html
In this scenario, the above javascript doesn't completely work. I mean, it downloads the 3 files but the size of the file is empty. This makes sense since I am accessing the website over HTTP and my files
should be served by the server.
On windows, how should I modify this line of code return fetch('file:///C:/jack/file/JACK/' + file) so that it starts working in this scenario.
Scenario 2:
Similarly, when I deployed the WAR in the tomcat running on RHEL server, I am accessing my application in the following manner:
https://myserver.com/testing/index/html
In this scenario also, I am noticing empty files are getting downloaded when I used the above code with this change:
return fetch('/srv/users/JACK' + file)
Scenario #1 is just used for my local testing and it's okay if it doesn't work for me because I can understand that I may not be able to access the files from my local directory. However, in Scenario 2, on RHEL server, I am trying to access the files from srv/users/JACK directory which isn't local or specific to any user. Why I am still seeing 3 empty csv files getting downloaded in this scenario? Please advise what would be best course of action to overcome this scenario.
Related
I am currently adding new features to a rather big project which has been running for a couple of years. The project uses a Couch DB database, Hapijs in the backend and React in the frontend.
My goal is to add a download button that let's the user (admin) download all the .pdf files from the database sorted by year and bundled in a .zip archive. The server already has a route for similar tasks, which is why I'm expanding the functionality of this route instead of creating a new one from scratch.
The /generate/{query} route handles all the file exports depending on the params in the query string. The query params will always contain the year. The route starts like this:
server.route({
method: 'GET',
path: '/generate/{query*}',
handler: async function(request, h) {
...
}
})
This route starts a long series of asynchronous processes and in the end successfully stores all the .pdf files in a tmp folder, sorted by year and bundled in .zip files. The only thing left to do would be to send the .zip file as a response back to the frontend, where the download is triggered.
This is where I'm having problems:
I am using the inert package to add files to my response with the file() method. If I pass in the path as a hardcoded string (e.g. file('/tmp/2019.zip')), the data is sent to the frontend correctly and the download starts as it should. However, if I try to build the same string dynamically, I get a corrupted .zip file that's only 1MB - 3MB (the original file is 250MB) (once it was 80MB, once 150MB, but both files were corrupted and couldn't be unpacked). I have tried a couple of different approaches to get a dynamic file path (template literals etc.), but they all produced the same result. This is what I'm currently stuck with:
const response = h.file(Path.join(__dirname + '/tmp/' + request.query.year + '.zip'), {
mode: 'attachment',
})
return response
I then created the following test route:
server.route({
method: 'GET',
path: '/test/{query*}',
handler: function(req, res) {
return res.file(Path.join(__dirname + '/tmp/' + req.query.year + '.zip'), {
mode: 'attachment'
})
}
})
...and it works like a charm, no matter which year I pass in as a query param.
This makes it difficult to pinpoint the exact part of my code that causes trouble: there are no issues with dynamically generated file paths in my test route, so that can't be it. The code in the actual route is also executed correctly, but only if the path to the file is hardcoded, so it's obviously not completely broken. For testing purposes I am currently not removing files from the tmp folder, so it can't be a problem with my async functions (the differently sized corrupted .zip files made me wonder if this might have something to do with my problem). But... what else could it be? Why can't I pass dynamic file names in my main route?
Has anybody here ever encountered anything like this? Let me know if you need more details. Any help is appreciated, even the slightest hunch could be of help!
I'm trying to dynamically get the images from the NOAA / National Weather Service here
https://radar.weather.gov/ridge/RadarImg/N0R/MTX
using JavaScript and it's proving to be tricky. Even though the names of the image files are standard/predictable, part of the naming convention includes the time at which the image was captured, and those times vary, so it makes more sense to loop through the directory instead of just making a lot of guesses. I know that scanning directories is generally not allowed because of the obvious security issues, so I tried opening it in an iframe, but there is no "index.html" file per se--all the html appears to be automatically generated by the browser, so there is no DOM to work with.
So has anyone dealt with something like this before? I'm stumped.
The server at that URL actually responds with the HTML you see in your browser.
Here is an example to parse the images from this directory in node.js:
const DOM = require('dom-parser')
const axios = require('axios')
axios.get('https://radar.weather.gov/ridge/RadarImg/N0R/MTX/').then((res) => {
const dom = new DOM().parseFromString(res.data)
const imageUrls = dom.getElementsByTagName('a')
.map(link => link.getAttribute('href'))
.filter(url => url && url.match('gif$'))
console.log(imageUrls)
})
I’m a bit new to javascriipt/nodejs and its packages. Is it possible to download a file using my local browser or network? Whenever I look up scraping html files or downloading them, it is always done through a separate package and their server doing a request to a given url. How do I make my own computer download a html file as if I did right click save as on a google chrome webpage without running into any server/security issues and errors with javascript?
Fetching a document over HTTP(S) in Node is definitely possible, although not as simple as some other languages. Here's the basic structure:
const https = require(`https`); // use http if it's an http url;
https.get(URLString, res => {
const buffers = [];
res.on(`data`, data => buffers.push(data));
res.on(`end`, ()=>{
const data = Buffer.concat(buffers);
/*
from here you can do what you want with the data. You can write it to a file
with fs, you can console.log it using data.toString(), etc.
*/
});
})
Edit: I think I missed the main question you had, give me a sec to add that.
Edit 2: If you're comfortable with doing the above, the way you access a website the same way as your browser is to open up the developer tools (F12 on Chrome) go to the network tab, find the request that the browser has made, and then using http(s).get(url, options, callback), set the exact same headers in the options that you see in your browser. Most of the time you won't need all of them, all you'll need is the authentication/session cookie.
I've converted an existing web application (HTML5, JS, CSS, etc.) into a Windows UWP app so that (hopefully) I can distribute it via the Windows Store to Surface Hubs so it can run offline. Everything is working fine, except PDF viewing. If I open a PDF in a new window, the Edge-based browser window simply crashes. If I open an IFRAME and load PDFJS into it, that also crashes. What I'd really like to do is just hand off the PDF to the operating system so the user can view it in whatever PDF viewer they have installed.
I've found some windows-specific Javascript APIs that seem promising, but I cannot get them to work. For example:
Windows.System.Launcher.launchUriAsync(
new Windows.Foundation.Uri(
"file:///"+
Windows.ApplicationModel.Package.current.installedLocation.path
.replace(/\//g,"/")+"/app/"+url)).then(function(success) {
if (!success) {
That generates a file:// URL that I can copy into Edge and it shows the PDF, so I know the URL stuff is right. However, in the application it does nothing.
If I pass an https:// URL into that launchUriAsync function, that works. So it appears that function just doesn't like file:// URLs.
I also tried this:
Windows.ApplicationModel.Package.current.installedLocation.getFileAsync(url).then(
function(file) { Windows.System.Launcher.launchFileAsync(file) })
That didn't work either. Again, no error. It just didn't do anything.
Any ideas of other things I could try?
-- Update --
See the accepted answer. Here is the code I ended up using. (Note that all my files are in a subfolder called "app"):
if (location.href.match(/^ms-appx:/)) {
url = url.replace(/\?.+/, "");
Windows.ApplicationModel.Package.current.installedLocation.getFileAsync(("app/" + url).replace(/\//g,"\\")).then(
function (file) {
var fn = performance.now()+url.replace(/^.+\./, ".");
file.copyAsync(Windows.Storage.ApplicationData.current.temporaryFolder,
fn).then(
function (file2) {
Windows.System.Launcher.launchFileAsync(file2)
})
});
return;
}
Turns out you have to turn the / into \ or it won't find the file. And copyAsync refuses to overwrite, so I just use performance.now to ensure I always use a new file name. (In my application, the source file names of the PDFs are auto-generated anyway.) If you wanted to keep the filename, you'd have to add a bunch of code to check whether it's already there, etc.
LaunchFileAsync is the right API to use here. You can't launch a file directly from the install directory because it is protected. You need to copy it first to a location that is accessible for the other app (e.g. your PDF viewer). Use StorageFile.CopyAsync to make a copy in the desired location.
Official SDK sample: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/AssociationLaunching
I just thought I'd add a variation on this answer, which combines some details from above with this info about saving a blob as a file in a JavaScript app. My case is that I have a BLOB that represents the data for an epub file, and because of the UWP content security policy, it's not possible simply to force a click on a URL created from the BLOB (that "simple" method is explicitly blocked in UWP, even though it works in Edge). Here is the code that worked for me:
// Copy BLOB to downloads folder and launch from there in Edge
// First create an empty file in the folder
Windows.Storage.DownloadsFolder.createFileAsync(filename,
Windows.Storage.CreationCollisionOption.generateUniqueName).then(
function (file) {
// Open the returned dummy file in order to copy the data to it
file.openAsync(Windows.Storage.FileAccessMode.readWrite).then(function (output) {
// Get the InputStream stream from the blob object
var input = blob.msDetachStream();
// Copy the stream from the blob to the File stream
Windows.Storage.Streams.RandomAccessStream.copyAsync(input, output).then(
function () {
output.flushAsync().done(function () {
input.close();
output.close();
Windows.System.Launcher.launchFileAsync(file);
});
});
});
});
Note that CreationCollisionOption.generateUniqueName handles the file renaming automatically, so I don't need to fiddle with performance.now() as in the answer above.
Just to add that one of the things that's so difficult about UWP app development, especially in JavaScript, is how hard it is to find coherent information. It took me hours and hours to put the above together from snippets and post replies, following false paths and incomplete MS documentation.
You will want to use the PDF APIs https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/PdfDocument/js
https://github.com/Microsoft/Windows-universal-samples/blob/master/Samples/PdfDocument/js/js/scenario1-render.js
Are you simply just trying to render a PDF file?
I have a local file download, which gets triggered in the following fashion.
panelBody.append('<button onclick="downloadCsv(\''+skl+'\')">Download csv</button>')
function downloadCsv(data){
var filename=data+'_report.csv'
var form = $('<form>', {action: '/admin/download', method: 'GET'});
form.append($('<input>', {name: 'file_name', value: filename}));
form.submit();
return false;
}
router.get('/download',helper.isSchoolAdmin,feedController.downloadCsv);
downloadCsv:function(req,res){
var fileName = process.cwd()+'/reports/'+ req.query.file_name;
res.download(fileName,function(err){
if(!err)console.log('down')
});
}
The file itself is written to the specified path in the immediate previous request to a different route.
Now this works fine on my local..My question is, what would happen once I deploy it onto heroku? would my file upload and download still work, as I've read that heroku uses an ephimeral file system which allows writing only to a tmp directory.
If that's the case, could someone walk me through on how exactly to write to that directory through node...and also, would the file end up getting deleted before I could download it, since the upload and download are part of separate requests? Thanks.
Heroku's "ephemeral filestystem" is due to the containerisation of the app. When your app is built, an LXC container is sent to the runtimes and executed.
That means any file written on the filesystem will not be persisted accross redeploys, app restarts and multiple dynos. Even in the tmp folder.
You shouldn't store those files on disk, but on a dedicated file storage platform such as Amazon S3 or Google Cloud Storage.
See https://devcenter.heroku.com/articles/s3-upload-node