NodeJS: Get filename from readStream - javascript

TLDR; Can I get filename from readable stream?
Got a method to download file from Google Drive and save it to local directory as I'm requesting response as stream , I would like to save the file with the same name as it has on the drive without explicitly asking user.
Do I have to make 1 extra request prior, to get filename from drive , is that the only way ?
async fetchFile(fileID) {
// Get file as stream
const { data: gdriveFile } = await this.drive.files.get(
{
fileId: fileID,
alt: "media"
},
{
responseType: "stream"
}
)
// FIXME: get filename from gdrive
console.log("Saving File: " + JSON.stringify(gdriveFile))
// fs writeStream
// const destStream = fs.createWriteStream(
// path.resolve(destDir + "/" + filename || "bleh.log")
// )
// pipeline(gdriveFile, destStream).end(() => {
// return "File copied"
// })
}
Thnx

Related

RNFetchBlob - Not able to get API data in generated image file

Hello guys I designed certificate Image module on my website when I click on Download I able to generate a certificate image file with all data.
but right now, I designing same for React Native using RNFetchBlob from 'rn-fetch-blob' when i calling to api I able to generate a image but not able getting my Name and city as its show in code..
One more thing when I console.log I able to see api data with Name and City
I guess the problem is Remote_Image_Path
OnPress = downloadimage
Console.log
https://example.com/xxxApi/xxxxpath?name=JasonBourne&city=NewYork&uniqueId=xx4434cc/xxx.png
App.js
const downloadImage = () => {
const REMOTE_IMAGE_PATH =
(`${API_URL}/xxxApi/xxxxpath?name=${apiData.name}&city=${apiData.city}&uniqueId=${route.params.id}`+"/xxx.png");
setImageUrl(REMOTE_IMAGE_PATH);
console.log(imageUrl);
// Main function to download the image
// To add the time suffix in filename
let date = new Date();
// Image URL which we want to download
let image_URL = (`${API_URL}/puxxxApiblicApi/xxxxpath?name=${apiData.name}&city=${apiData.city}&uniqueId=${route.params.id}`+"/xxx.png")
// Getting the extention of the file
let ext = getExtention(imageUrl);
ext = '.' + ext[0];
// Get config and fs from RNFetchBlob
// config: To pass the downloading related options
// fs: Directory path where we want our image to download
const { config, fs } = RNFetchBlob;
let PictureDir = fs.dirs.PictureDir;
let options = {
fileCache: true,
addAndroidDownloads: {
// Related to the Android only
useDownloadManager: true,
notification: true,
path:
PictureDir +
'/xxpicturepath' +
Math.floor(date.getTime() + date.getSeconds() / 2) +
ext,
description: 'Image',
},
};
config(options)
.fetch('GET', imageUrl)
.then(res => {
// Showing alert after successful downloading
console.log('res -> ', JSON.stringify(res));
alert('Image Downloaded Successfully.');
});
};
Just added &x after uniqueId=${route.params.id} for separating link with name extension
const downloadImage = () => {
const REMOTE_IMAGE_PATH =
(`${API_URL}/xxxApi/xxxxpath?name=${apiData.name}&city=${apiData.city}&uniqueId=${route.params.id}&x`+"/xxx.png");
// Main function to download the image
// To add the time suffix in filename
let date = new Date();
// Image URL which we want to download
let image_URL = REMOTE_IMAGE_PATH;
// Getting the extention of the file
let ext = getExtention(imageUrl);
ext = '.' + ext[0];
// Get config and fs from RNFetchBlob
// config: To pass the downloading related options
// fs: Directory path where we want our image to download
const { config, fs } = RNFetchBlob;
let PictureDir = fs.dirs.PictureDir;
let options = {
fileCache: true,
addAndroidDownloads: {
// Related to the Android only
// xxpicturepath is a directory name in android storage for storing image
useDownloadManager: true,
notification: true,
path:
PictureDir +
'/xxpicturepath' +
Math.floor(date.getTime() + date.getSeconds() / 2) +
ext,
description: 'Image',
},
};
config(options)
.fetch('GET', imageUrl)
.then(res => {
// Showing alert after successful downloading
console.log('res -> ', JSON.stringify(res));
alert('Image Downloaded Successfully.');
});
};

How can I duplicate a file (completely) but change it's name?

So I need to generate the smaller previews of the image files that will be uploaded and I have to append "_preview" at the end of each file name.
Currently I'm doing this:
uploadFile.map((file) => {
if (file.type.includes('image')) {
console.log('Generating thumbnail for ' + file.name)
const fileName = file.name.split('.').slice(0, -1).join('.')
const fileExtension = file.name.split('.').pop()
const compressedFile = new File(
[file.slice(0, file.size, file.type)],
fileName + '_preview.' + fileExtension,
)
console.log('Generated file:', compressedFile)
convert({
file: compressedFile,
width: 300,
height: 300,
type: fileExtension,
})
.then((resp) => {
uploadFile.push(resp)
})
.catch((error) => {
// Error
console.error('Error compressing ', file.name, '-', error)
})
}
})
The problem is that "compressedFile" is missing some fields which were present in the original file and so the convert functoin throws the error "File type not supported". As you can see "type" and "webkitRelativePath" are not copied.
Can anybody suggest how I can retain all the information from the original file and just append _preview at the end of file name?
I realized File API provides an option to pass "options" object as well which can specify the file type. For instance:
const file = new File(["foo"], "foo.txt", {
type: "text/plain",
});
Source: https://developer.mozilla.org/en-US/docs/Web/API/File/File
for copy code in js or duplicate, you can use this code
//copyfile.js
const fs = require('fs');
// destination will be created or overwritten by default.
fs.copyFile('C:\folderA\myfile.txt', 'C:\folderB\myfile.txt', (err) => {
if (err) throw err;
console.log('File was copied to destination');
});

Uploading multiple large json files in one go React-native to NodeJS

I am trying to upload multiple large size JSON files from React-native to node js.
The files are being uploaded unless the file in larger in size, in which case, it does not upload in one try.
I suspect that:
Since the upload code is in a for loop the code is starting the upload but not waiting for the file to upload and starting to upload the next file
Is there any way to ensure that each file gets uploaded in one go?
syncFunction() {
var RNFS = require('react-native-fs');
var path = RNFS.DocumentDirectoryPath + '/toBeSynced';
RNFS.readDir(path)
.then((success) => {
for (let i = 0; i < success.length; i++) {
var fileName = success[i].name
var filePath = success[i].path
var uploadUrl = 'http://192.168.1.15:3333/SurveyJsonFiles/GetFiles/'
if (Platform.OS === 'android') {
filePath = filePath.replace("file://", "")
} else if (Platform.OS === 'ios') {
filePath = filePath
}
const data = new FormData();
data.append("files", {
uri: filePath,
type: 'multipart/form-data',
name: fileName,
});
const config = {
method: 'POST',
headers: {
'Accept': 'application/json',
},
body: data,
};
fetch(uploadUrl, config)
.then((checkStatusAndGetJSONResponse) => {
console.log(checkStatusAndGetJSONResponse);
this.moveFile(filePath, fileName)
}).catch((err) => {
console.log(err)
});
}
})
.catch((err) => {
console.log(err.message);
});
}
The JSON files will more than 50Mb depending on data, since it contains base64 image data the size will increase as the user takes more photos.
The app will be creating new files when the user records any information, There is no error message displayed for partial file upload.
The this.moveSyncedFiles() is moving the synced files to another folder so that the same file does not get uploaded multiple times
moveFile(oldpath, oldName) {
var syncedPath = RNFS.DocumentDirectoryPath + '/syncedFiles'
RNFS.mkdir(syncedPath)
syncedPath = syncedPath + "/" + oldName
RNFS.moveFile(oldpath, syncedPath)
.then((success) => {
console.log("files moved successfully")
})
.catch((err) => {
console.log(err.message)
});
}
It turns out the fault was on the nodejs side and nodemon was restarting the server every time a new file was found so we just moved the uploads folder outside the scope of the project

How to get file properties and upload a file from ionic 4?

I am trying to upload a file from mobile to google bucket using ionic 4. Although a file can upload into the could. I am struggling to get the file properties out of file object.
Here is my method,
async selectAFile() {
const uploadFileDetails = {
name: '',
contentLength: '',
size: '',
type: '',
path: '',
};
this.fileChooser.open().then(uri => {
this.file.resolveLocalFilesystemUrl(uri).then(newUrl => {
let dirPath = newUrl.nativeURL;
const dirPathSegments = dirPath.split('/');
dirPathSegments.pop();
dirPath = dirPathSegments.join('/');
(<any>window).resolveLocalFileSystemURL(
newUrl.nativeURL,
function(fileEntry) {
uploadFileDetails.path = newUrl.nativeURL;
const file: any = getFileFromFileEntry(fileEntry);
//log 01
console.log({ file });
uploadFileDetails.size = file.size;
uploadFileDetails.name = `${newUrl.name
.split(':')
.pop()}.${file.type.split('/').pop()}`;
uploadFileDetails.type = file.type;
async function getFileFromFileEntry(fileEntry) {
try {
return await new Promise((resolve, reject) =>
fileEntry.file(resolve, reject)
);
} catch (err) {
console.log(err);
}
}
},
function(e) {
console.error(e);
}
);
});
});
// here uploadFileDetails is simller to what I declared at the top ;)
// I wan't this to be populated with file properties
// console.log(uploadFileDetails.name) --> //''
const uploadUrl = await this.getUploadUrl(uploadFileDetails);
const response: any = this.uploadFile(
uploadFileDetails,
uploadUrl
);
response
.then(function(success) {
console.log({ success });
this.presentToast('File uploaded successfully.');
this.loadFiles();
})
.catch(function(error) {
console.log({ error });
});
}
even though I can console.log the file in log 01. I am unable to get file properties like, size, name, type out of the resolveLocalFileSystemURL function. basically, I am unable to populate uploadFileDetails object. What am I doing wrong? Thank you in advance.
you actually need 4 Ionic Cordova plugins to upload a file after getting all the metadata of a file.
FileChooser
Opens the file picker on Android for the user to select a file, returns a file URI.
FilePath
This plugin allows you to resolve the native filesystem path for Android content URIs and is based on code in the aFileChooser library.
File
This plugin implements a File API allowing read/write access to files residing on the device.
File Trnafer
This plugin allows you to upload and download files.
getting the file's metadata.
file.resolveLocalFilesystemUrl with fileEntry.file give you all the metadata you need, except the file name. There is a property called name in the metadata but it always contains value content.
To get the human readable file name you need filePath. But remember you can't use returning file path to retrieve metadata. For that, you need the original url from fileChooser.
filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1) is used to get only file name from filePath.
You need nativeURL of the file in order to upload it. Using file path returning from filePath is not going to work.
getFileInfo(): Promise<any> {
return this.fileChooser.open().then(fileURI => {
return this.filePath.resolveNativePath(fileURI).then(filePathUrl => {
return this.file
.resolveLocalFilesystemUrl(fileURI)
.then((fileEntry: any) => {
return new Promise((resolve, reject) => {
fileEntry.file(
meta =>
resolve({
nativeURL: fileEntry.nativeURL,
fileNameFromPath: filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1),
...meta,
}),
error => reject(error)
);
});
});
});
});
}
select a file from the file system of the mobile.
async selectAFile() {
this.getFileInfo()
.then(async fileMeta => {
//get the upload
const uploadUrl = await this.getUploadUrl(fileMeta);
const response: Promise < any > = this.uploadFile(
fileMeta,
uploadUrl
);
response
.then(function(success) {
//upload success message
})
.catch(function(error) {
//upload error message
});
})
.catch(error => {
//something wrong with getting file infomation
});
}
uploading selected file.
This depends on your backend implementation. This is how to use File Transfer to upload a file.
uploadFile(fileMeta, uploadUrl) {
const options: FileUploadOptions = {
fileKey: 'file',
fileName: fileMeta.fileNameFromPath,
headers: {
'Content-Length': fileMeta.size,
'Content-Type': fileMeta.type,
},
httpMethod: 'PUT',
mimeType: fileMeta.type,
};
const fileTransfer: FileTransferObject = this.transfer.create();
return fileTransfer.upload(file.path, uploadUrl, options);
}
hope it helps. :)

AWS S3 Upload after GET Request to Image, Not Uploading Correctly

I'm trying to upload an image to my AWS S3 bucket after downloading the image from another URL using Node (using request-promise-native & aws-sdk):
'use strict';
const config = require('../../../configs');
const AWS = require('aws-sdk');
const request = require('request-promise-native');
AWS.config.update(config.aws);
let s3 = new AWS.S3();
function uploadFile(req, res) {
function getContentTypeByFile(fileName) {
var rc = 'application/octet-stream';
var fn = fileName.toLowerCase();
if (fn.indexOf('.png') >= 0) rc = 'image/png';
else if (fn.indexOf('.jpg') >= 0) rc = 'image/jpg';
return rc;
}
let body = req.body,
params = {
"ACL": "bucket-owner-full-control",
"Bucket": 'testing-bucket',
"Content-Type": null,
"Key": null, // Name of the file
"Body": null // File body
};
// Grabs the filename from a URL
params.Key = body.url.substring(body.url.lastIndexOf('/') + 1);
// Setting the content type
params.ContentType = getContentTypeByFile(params.Key);
request.get(body.url)
.then(response => {
params.Body = response;
s3.putObject(params, (err, data) => {
if (err) { console.log(`Error uploading to S3 - ${err}`); }
if (data) { console.log("Success - Uploaded to S3: " + data.toString()); }
});
})
.catch(err => { console.log(`Error encountered: ${err}`); });
}
The upload succeeds when I test it out, however after trying to redownload it from my bucket the image is unable to display. Additionally, I notice after uploading the file with my function, the file listed in the bucket is much larger in filesize than the originally uploaded image. I'm trying to figure out where I've been going wrong but cannot find where. Any help is appreciated.
Try to open the faulty file with a text editor, you will see some errors written in it.
You can try using s3.upload instead of putObject, it works better with streams.

Categories