nodejs fs createWriteStream not working with file path prefix - javascript

I am trying to call an API, loop through an array of images, assign unique names to each image in the array and then write them to a local directory. If I simplify the code I can write them to the root folder .I already created the sub folder manually, so it existed prior to running the function.
Here is my basic function:
const imageFolder = './img';
function downloadImage(url, filepath) {
client.get(url, res => {
res.pipe(fs.createWriteStream(`${imageFolder}/${filepath}`));
});
}
...make api call
const imagesArray = generations.data.map(item => item.generation.image_path);
imagesArray.forEach(item => {
// const fileName = uuid.v4() + '.webp'; // trying to assign unique filename with uuid
const fileName = new Date().getTime().toString() + '.webp'; // trying to assign unique filename with date object
downloadImage(item, fileName);
});
If I change
res.pipe(fs.createWriteStream(`${imageFolder}/${filepath}`));
to
res.pipe(fs.createWriteStream(filepath));
then it will work but just dumps the images in the root. I was thinking perhaps I was trying to concatenate a variable name with a string (for fileName + '.webp', but it is working in the root as mentioned. See attached image.
I also tried adding the path into the actual function call inside the forEach loop like so
downloadImage(item, `${imageFolder}/${fileName}`);
I did wonder about needing the __dirname variable, or whether it could be a permissions issue, but I don't see any errors.
I am assuming this is pretty straightforward.

OK, was fairly simple and I guess I sort of knew it once I got it working, changing to
downloadImage(item, path.join('src', 'img', fileName));
path.join concatenates folder names and fixes issues when working across platforms (OSX, Windows etc) which applies in this case as I am testing from both Windows and Mac.

Related

Delete all files with a specific name, and * extension at Azure Blob Storage

I have a process where a client uploads a document. This document can be in the form of a PDF, JPG or PNG file only and it should be reuploaded once a year (it is an errors and omissions insurance policy).
I am saving this file in a container.
For deleting files from anywhere at the application, I have this function (Node):
deleteFromBlob = async function (account, accountKey, containerName, blobFolder, blobName) {
try {
const {
BlobServiceClient,
StorageSharedKeyCredential
} = require("#azure/storage-blob");
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const blobServiceClient = new BlobServiceClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential
);
const containerClient = blobServiceClient.getContainerClient(containerName);
const blockBlobClient = containerClient.getBlockBlobClient(blobFolder + '/' + blobName);
const uploadblobResponse = await blockBlobClient.deleteIfExists()
return true
}
catch(e) {
return false
}
}
And this works perfect when I know the file name and extension I want to delete, like "2448.pdf":
let deleteFile = await utils.deleteFromBlob(account, accountKey, "agents", "/eopolicies/", userData.agentid.toString() + ".pdf" )
But the problem Im facing is that the function above is to delete a file I know exists; for example, if the agent ID is 2448 and he uploads "policy.pdf" I save it as "2448.pdf" for easy file identification.
The problem Im facing is if the agent uploaded a .PNG last year. a .DOC a year before, and a .PDF now. If that's the case, I want to delete 2448.* and keep only the latest version of the document.
So I tried changing my function to
let deleteFile = await utils.deleteFromBlob(account, accountKey, "agents", "/eopolicies/", userData.agentid.toString() + ".*" )
And of course it is not working...
I tried to find a solution and all I found is one to list the content of a folder, then loop it and delete the specific file I want; but that will not work for me since there are 37,000 EO policies on that folder.
Is there a way to delete files with a specific name, and whatever extension?
Thanks.
I've never tried using a wildcard on the extension side of the file name. However, I would iterate through the files in the directory and find the one that contains the specific string you are looking for. Get it's index, and delete from there.

Questions about fs.writeFile

I want to use fs.WriteFile in my JS project. I am building an algorithm that outputs random data and I want to give the user the opportunity to save the data as a txt file. I have been able to implement fs.WriteFile in my project, but I have a couple of questions on how to proceed next as the function remains somewhat unclear.
How do I specify that I want to include the contents of various vars? Is it as simple as data = let1 + let2 + let3 and all of the data will be included?
can I add the current date and time in the .txt file name? If so, how?
How do I tell writeFile to save the contents to a .txt file and open a download blob so that people can specify their own download locations?
Thanks in advance!
I've tried looking at basic documentation but its mainly the same: a template using a simple string that saves into the same directory, which is what I don't want.
For you first question, you are correct. You can just combine different string variables into a larger string variable. See the documentation for string concatenation for more information.
For your second question, yes you can. You can get the current date and time with new Date() and turn it into a variety of formats. For file names, using mydate.toISOString() will probably be the most clean.
Here's an example of both of these in practice:
import fs from 'fs';
// Here's some data that we want to put in the file.
const name = "Bob";
const age = 43;
// Create the data we want to put in our file.
const data = name + '\n' + age;
// Let's grab the date and use it as part of our file name.
const date = new Date();
const fileName = `${date.toISOString()}.txt`;
// Call fs.writeFile to put the data in the file.
fs.writeFile(fileName, data, () => {
console.log(`Wrote data to ${fileName}.`);
});
Your third question is more complicated and probably worth a separate post. fs.writeFile can't do this for you. You'll have to come up with some mechanism for the user to enter their own file name and build off of that.
Edit:
To address your question in the comments, you might be a little confused with how NodeJS works. NodeJS runs on the server and doesn't have any way to deal with buttons or UIs by default like browser JavaScript does. It might be helpful to look at the differences between the two. So you won't be able to save it to the downloads folder on a button click.
With that said, we can save the file to the user's Downloads folder with the same script I posted above by adding the path to the Downloads folder to the beginning of the file name.
Here's the code above adjusted to do that:
import fs from 'fs';
import os from 'os'; // NEW
import path from 'path'; // NEW
const name = "Bob";
const age = 43;
const data = name + '\n' + age;
const date = new Date();
const fileName = `${date.toISOString()}.txt`;
// Get the user's home directory.
const homedir = os.homedir();
// Append the Downloads directory and fileName to the user's home directory.
const fullPath = path.join(homedir, 'Downloads', fileName);
// Use fullPath here instead of fileName.
fs.writeFile(fullPath, data, () => {
console.log(`Wrote data to ${fileName}.`);
});

Is there a way to get all file names in a folder in website files?

I am currently on a Chromebook using HTML, CSS, JS on repl.it, and I can't find a way to check the files in a folder.
I have a folder called objects containing a bunch of JSON files. I want the object dataQueue to hold every object, and I have got that working. But I don't want to have to specifically type out each file to make it run the function I made called getFile for every single file, as is done below in the code.
I am also trying to do this without any libraries (so I can understand everything), and also I want it to be able to run both as a site on the computer and on repl.it.
This is my current code:
//defined the dataQueue object where the object files
//will be stored
var dataQueue={};
//a function to get files by just inputting their
//filename and what i want to call them with in the future
function getFile(theFile,index){
//sets to a new request
var txtFile = new XMLHttpRequest();
//works with any file type, gets the specified file
txtFile.open("GET", theFile, true);
//I don't know what this does
txtFile.send(null);
//runs when the file is recieved
txtFile.onreadystatechange = function(){
//test to make sure it is ready
if(txtFile.readyState==4){
//add object file to dataQueue object
dataQueue[index]=JSON.parse(txtFile.response);
}
}
}
//get each file individually
getFile("objects/ship1.json","ship1");
getFile("objects/box1.json","box1");
getFile("objects/bigplatform1.json","platform1");
getFile("objects/bigbox1.json","bigbox1");
I want to be able to just add another JSON file to the objects folder and have it put in the dataQueue object without needing to add another getFile function.
I think a solution would be if I could somehow get a list of strings containing all of the filenames in the Objects folder, but I have tried finding a way to, and I couldn't find anything.
The question: How do I get each file name in a specific folder in a website's files?

Is there a way to count how many files are in a folder and then import it to a variable?

I'm trying to call the number of files in a folder into a variable so that I can set the array size into that number.
Is this possible using JS and HTML?
I've found a piece of code that might be useful. However I don't understand it well, is it useful to the function I visioned? How?
const fs = require('fs');
var dir = '../imgvids';
fs.readdir(dir, (err, files) => {
console.log(files.length);
});
The intended porpuse is to create a variable and set the array size to it, so that then i can create a function that concatenates 'video' + 'i++' (i=0) and to stop at the size of the array so i can make a playlist of videos that autoplay muted.
You can use fs-finder module in Node.js.
Here is a link
let Finder = require('fs-finder');
let finder = new Finder('./path/dir');
let files = finder.recursively().findFiles();
console.log(files);
This will log all the files inside directories and subdirectories

Javascript to get .pdf file from partial match

Complete and utter javascript newbie here with a problem fetching .pdf files from a web server based on a partial match. I have made a program that outputs data to a webserver, and one of the components is a folder of .pdf files. I want to be able to click on a link that will pull up the corresponding .pdf file based on a value in the data table that's generated (I'm using slickgrid for this). Each of the .pdf files contains the value that's in the data table and serves as good query to the .pdf folder, and I've been successful at getting the .pdfs I want with the following code:
var value = grid.getData().getItem(row)['data'];
var locpath = window.location.pathname.substring(0,window.location.pathname.lastIndexOf('/'));
var plotsFolder = window.location.protocol + "//" + window.location.host + locpath + "/CovPlots/";
var href = plotsFolder + value + ".pdf";
return "<a href='" + href + "'>" + value + "</a>";
The catch here is that sometimes the .pdf file that's generated is a concatenation of two or more (I've seen up to 4 so far) of the 'data' strings, separated by '_' as a delimiter for reasons not worth getting into. So, if the .pdf file is 'somestring.pdf', I can get it without problem. However, if the .pdf file is 'somestring_anotherstring.pdf', I can't figure out how to get that .pdf file if I have either 'somestring' or 'anotherstring' as the value of 'data'.
I've tried a ton of different things to get some kind of lookup that I can use to pull down the correct file based on a partial match. The latest attempt is with the FilenameFilter object in javascript, but without any knowledge of javascript, I'm having a hard time to get it working. I tried to create a new function that I could call as a lookup for the .pdf URL:
function lookup() {
File directory = new File(plotsFolder);
String[] myFiles = directory.list(new FilenameFilter() {
public boolean accept(File directory, String fileName) {
return fileName.match(value);
}
});
}
That only seems to thrown an error. Can anyone point me in the right direction to be able to download the correct .pdf file based on a partial match? I also tried to see if there was a jquery way to do it, but couldn't seem to find something that works. Thanks in advance!
Without support from the server, JavaScript cannot find a file from a partial filename. What you can do, however, is have a little script on the server that does the partial-filename-matching for JavaScript, and then JavaScript can ask the server to do the match, and then when it gets the match back, it can use that filename.
If you don't mind loading a whole index of all the PDFs at once, you could use this little Python script to generate an index in a nice, JavaScript-friendly JSON format:
#!/usr/bin/env python
# Create an index of a bunch of PDF files.
# Usage: python make_index.py directory_with_pdf_files
import os
import sys
import json
def index(directory):
index = {}
for filename in os.listdir(directory):
base, ext = os.path.splitext(filename)
if ext.lower() != '.pdf':
continue
for keyword in base.split('_'):
index[keyword] = filename
with open(os.path.join(directory, 'index.json'), 'w') as f:
f.write(json.dumps(index))
if __name__ == '__main__':
index(sys.argv[1])
Then you can just load index.json with jQuery or what-have-you. When you need to find a particular PDF's filename, you can do something like this (assuming the object loaded from index.json is in the indexOfPDFs variable):
var href = plotsFolder + indexOfPDFs[value];

Categories