We Keep Coding

Uploading images to AWS S3 bucket - specifying file path for randomly generated file names

I'm working on an automated testing project using PuppeteerJS in headless Chrome and trying to integrate existing screenshot functionality with AWS-SDK to upload images to an AWS S3 bucket on test failure.
The problem i'm having is the sub directories in a screenshots folder and the image file names are generated randomly in another file based on the current date and test environment, and run every time a test runs. The format of the generated directories/files is "screenshots/year/month/day/randomname.png".
The next step in the test is after the screenshots are generated, the folder containing the newly created images should be uploaded to AWS, and I've tried to achieve this using a glob to get every subdirectory and file with a png extension, like "screenshots/**/**/**/*.png", but i get a "no such file or directory" error". The folders/file names will be different everytime the tests run.
I've just started using AWS and I haven't been able to find a specific answer to my problem while researching.
import { PutObjectCommand } from "#aws-sdk/client-s3";
import { s3Client } from "../libs/s3Client.js";
import path from "path";
import fs from "fs";
const file = "../../screenshots/**/**/**/*.png";
const fileStream = fs.createReadStream(file);
// Set the parameters
export const uploadParams = {
Bucket: "bucket-name",
Key: path.basename(file),
// Add the required 'Body' parameter
Body: fileStream,
};
// Upload file to specified bucket.
export const run = async () => {
try {
const data = await s3Client.send(new PutObjectCommand(uploadParams));
console.log("Success", data);
return data; // For unit tests.
} catch (err) {
console.log("Error", err);
}
};
run();

Worked this out with the help of Jarmod. I needed to use the nodeJS:fs module to get the file paths recursively and returns a string which can be passed into the AWS fileStream variable for it to be uploaded to AWS. Jarmod shared the webmound article and i found the coder rocket fuel article hepful also.
https://www.webmound.com/nodejs-get-files-in-directories-recursively/
https://coderrocketfuel.com/article/recursively-list-all-the-files-in-a-directory-using-node-js

How to download file from deployed server?

Can someone tell me why function below properly download my file from server when i work locally (by localhost) but not download me and return me 500 internal server error when i try do is when i deploy my app on remote server?
async downloadFile(fileId: number): Promise<Buffer> {
const fileName = await this.getFileName(fileId);
const fileBuffer = await new Promise<Buffer>((resolve, reject) => {
fs.readFile(process.cwd() + '/files/' + fileName + '.xlsx', {}, (err, data) => {
if (err) reject(err)
else resolve(data)
});
});
return fileBuffer ;
}
thanks for any help
EDIT, ERROR FROM LOG:
ENOENT: no such file or directory

If you are willing to access your file relatively to your script dir you should use __dirname
Also using the path module in order to build your file location in a platform agnostic way is a good practice.
const path = require('path')
const filePath = path.join(__dirname, 'files', `${fileName}.xlsx`)
process.cwd() refers to you node process working dir. Using it in your context would tie your code to how the entry point has been called. This is bad. Code should not have to be aware of its execution context to work whenever this is possible.
An even better way would be to make your file location configurable (using an environment variable or a config file) and pass your download folder value to your code this way.
see https://12factor.net/config
example
const baseDir = process.env.FILES_PATH || '/some/default/location';
const filePath = path.join(baseDir, 'files', `${fileName}.xlsx`);
then run your program with
FILES_PATH=/your/directory node your_script.js

Read files from a directory with a given path in JS

Is it possible to return the contents of a static path to a directory instead of using an .
I want to write a script that reads the contents of a directory on the file system to a given time daily. This is integrated in a webapp I can't edit.

Short answer: you can't.
You need to do this server-side. Here is an answer from a similar question, using node.js.
You can use the fs.readdir or fs.readdirSync methods.
fs.readdir
const testFolder = './tests/';
const fs = require('fs');
fs.readdir(testFolder, (err, files) => {
files.forEach(file => {
console.log(file);
});
});
fs.readdirSync
const testFolder = './tests/';
const fs = require('fs');
fs.readdirSync(testFolder).forEach(file => {
console.log(file);
});
The difference between the two methods, is that the first one is asynchronous, so you have to provide a callback function that will be executed when the read process ends.
The second is synchronous, it will return the file name array, but it will stop any further execution of your code until the read process ends.

Loop through all files/images in a folder with Angular

I have an app with some products and each product has a gallery with a different amount of images. Each of the images has a name that is completely random / no correlation with the other image names.
Each of the product images are in /src/assets/images/products/:id/.
I need to add the paths to a gallery component but I can't loop through them because the names are random. Is there any way to just loop through each file from a folder using only Angular? If not can I do it on the back-end without renaming the files? I'm also running the app on a Node.js back-end if that matters.

You can't do that with frontend.
What you need to is using your back-end and return file in it.
You are using NodeJs as back-end so can use the fs.readdir or fs.readdirSync methods.
fs.readdir
const testFolder = './images/';
const fs = require('fs');
fs.readdir(testFolder, (err, files) => {
files.forEach(file => {
console.log(file); // use those file and return it as a REST API
});
})
fs.readdirSync
const testFolder = './images/';
const fs = require('fs');
fs.readdirSync(testFolder).forEach(file => {
console.log(file);
})
Read the full documenation, it may help you to how you can proceed.

How to get files contained in a path

In the following path:
app/test/vehicle/
there are several .js file. Is there any way in Javascript where I can get the number of .js file contains in the above-mentioned path?
I am using webStorm environment

Yes, using node:
const fs = require('fs');
const dir = './directory';
fs.readdir(dir, (err, files) => {
console.log(files.length);
});
https://stackoverflow.com/a/43747896/3650835

Since javascript doesn't have local filesystem access when run in the browser, there is no way to access local files from javascript. If you are asking about accessing all files hosted on a cdn or something similar in that directory, there is still no way to tell which requests will return a 20x.
If you are using nodejs, use the fs.readdir function:
const fs = require('fs');
fs.readdir('app/test/vehicle/', (err, files) => {
files.forEach(file => { console.log(file);} );
})

There is no way to access the file system from a web page.
If you are using nodejs, then you can use fs.readdir or fs.readdirSync:
const fs = require('fs');
const dir = './directory';
fs.readdir(dir, (err, files) => {
console.log(files.filter(file => file.endsWith(".js")).length);
});

Where are you trying to get list of the files? On the client side it is impossible. JavaScript does not have access to the client file system. If you are trying to do it in the server side using for example node.js then you can do something like:
var filesys = require('fs');
var files = filesys.readdirSync('/app/test/vehicle/');