I have file hosted on S3 bucket from where I am fetching the Excel file using an Axios request. After that I am writing that Excel file in the local filesystem. After that I am converting that Excel file data into JSON and sending it as a response; for that I am using XLSX library.
After sending the Excel file data as a JSON response I want to delete that Excel file from local filesystem.
When I am trying to delete the file it's failing to delete as well as failed to read the file. But when file is already available then it's reading the file and sending the response also.
Below is my code:
const response = await axios.get(data.me_url, { responseType: 'arraybuffer' });
if (response) {
const ostream = await fs.createWriteStream(`./${filename}`, 'binary');
ostream.write(response.data);
ostream.end();
const file = xlsx.readFile(`./${filename}`);
const sheetNames = file.SheetNames;
const totalSheets = sheetNames.length;
let parsedData = [];
for (let i = 0; i < totalSheets; i++) {
const tempData = xlsx.utils.sheet_to_json(file.Sheets[sheetNames[i]]);
tempData.shift();
parsedData.push(...tempData);
}
if (parsedData.length > 0) {
console.log('if filename',filename);
try {
fs.unlinkSync(`./${filename}`);
console.log("Delete File successfully.");
return res.status(200).send({ 'data': parsedData, 'message': 'Success', 'code': 200 });
} catch (error) {
console.log(error);
}
}
}
What am I doing wrong in the above code?
You don't need to create file in the local machine, send the file Array Buffer directly to xlsx directly as mentioned in the link here
Related
I have excel file stored in S3 bucket fro where I am storing the file url inside mongodb.I
want to read data from this excel file and transform into JSON.So for that I am XLSX package but it is not reading the file as I am passing the file url inside the method,
Below is my code:
try {
const data = await productSchema.findOne({ report_id: reportId });
if (data) {
console.log(data.me_url); //I am getting the file url
const file = xlsx.readFile(data.me_url);
const sheetNames = file.SheetNames;
const totalSheets = sheetNames.length;
console.log(sheetNames);
console.log(totalSheets);
}
}
catch (err) {
return err;
}
Any idea why it is not reading the file or anyhthing I have missed.
How do I download the document I receive in return in react?
Here is the my node.js app. fetchContracts is a function which getting data from mongodb then ganere a excel file by using json2xls npm package.
Its returns as like this:
const xls = json2xls(contracts);
return xls;
If tying to write file fs.writeFileSync(path.join(__dirname, filename), xls, 'binary'); generating successfully xlsx file in the server.
But I need to send the file to the server without writing file. For this, I made some experiments that you can see below.
export const EXPORT_EXCEL: SessionedAsyncControllerType = async (req: SessionedRequest, res: Response) => {
const fileName = 'hello_world.xlsx'
const fileType = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
const xls = await fetchContracts({}, "fileName.xlsx")
const fileData = xls;
res.writeHead(200, {
'Content-Disposition': `attachment; filename="${fileName}"`,
'Content-Type': fileType,
})
const download = Buffer.from(fileData, 'base64')
res.end(download)
}
I getting response like this.
But i don't know how can i download the response file in react?
In react side:
return api.get(`api/excel`).then((response: any) => {
console.log(response);
})
I just log into console. How can i download directly file which coming node response in react.js?
Try this
return api.get(`api/excel`).then((response: any) => {
const outputFilename = `${Date.now()}.xlsx`;
// If you want to download file automatically using link attribute.
const url = URL.createObjectURL(new Blob([response.data]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', outputFilename);
link.click();
})
I am currently building a chatbot that is able to receive attachments and save them to a local directory.
I want to find out how to use that same attachment and send it via email.
async downloadAttachmentAndWrite(attachment) {
// Retrieve the attachment via the attachment's contentUrl.
const url = attachment.contentUrl;
console.log(attachment)
// Local file path for the bot to save the attachment.
const localFileName = path.join(__dirname, attachment.name);
try {
// arraybuffer is necessary for images
const response = await axios.get(url, { responseType: 'arraybuffer' });
console.log('#####')
console.log(response.data)
// If user uploads JSON file, this prevents it from being written as "{"type":"Buffer","data":[123,13,10,32,32,34,108..."
if (response.headers['content-type'] === 'application/json') {
response.data = JSON.parse(response.data, (key, value) => {
return value && value.type === 'Buffer' ? Buffer.from(value.data) : value;
});
}
fs.writeFile(localFileName, response.data, (fsError) => {
console.log(localFileName)
console.log(response.data)
if (fsError) {
throw fsError;
}
});
} catch (error) {
console.error(error);
return undefined;
}
// If no error was thrown while writing to disk, return the attachment's name
// and localFilePath for the response back to the user.
return {
fileName: attachment.name,
localPath: localFileName
};
}
That is currently the function to receive and save to directory, but how do I actually capture the attachment and send it to another function?
Look over the 24.bot-authentication-msgraph sample in the BotBuilder-Samples repo. This sample demonstrates how a bot can be setup to send an email on the user's behalf.
Using that sample as a reference/template, you can infer how this process could work for you (in case you are not using MS Graph). The docs here explain how a file can be included as an attachment in an email.
If you preserve the location of the saved file, you should be able to read the file from the local directory and, using the method referenced above, attach the file before sending.
Hope of help.
I am trying to upload multiple large size JSON files from React-native to node js.
The files are being uploaded unless the file in larger in size, in which case, it does not upload in one try.
I suspect that:
Since the upload code is in a for loop the code is starting the upload but not waiting for the file to upload and starting to upload the next file
Is there any way to ensure that each file gets uploaded in one go?
syncFunction() {
var RNFS = require('react-native-fs');
var path = RNFS.DocumentDirectoryPath + '/toBeSynced';
RNFS.readDir(path)
.then((success) => {
for (let i = 0; i < success.length; i++) {
var fileName = success[i].name
var filePath = success[i].path
var uploadUrl = 'http://192.168.1.15:3333/SurveyJsonFiles/GetFiles/'
if (Platform.OS === 'android') {
filePath = filePath.replace("file://", "")
} else if (Platform.OS === 'ios') {
filePath = filePath
}
const data = new FormData();
data.append("files", {
uri: filePath,
type: 'multipart/form-data',
name: fileName,
});
const config = {
method: 'POST',
headers: {
'Accept': 'application/json',
},
body: data,
};
fetch(uploadUrl, config)
.then((checkStatusAndGetJSONResponse) => {
console.log(checkStatusAndGetJSONResponse);
this.moveFile(filePath, fileName)
}).catch((err) => {
console.log(err)
});
}
})
.catch((err) => {
console.log(err.message);
});
}
The JSON files will more than 50Mb depending on data, since it contains base64 image data the size will increase as the user takes more photos.
The app will be creating new files when the user records any information, There is no error message displayed for partial file upload.
The this.moveSyncedFiles() is moving the synced files to another folder so that the same file does not get uploaded multiple times
moveFile(oldpath, oldName) {
var syncedPath = RNFS.DocumentDirectoryPath + '/syncedFiles'
RNFS.mkdir(syncedPath)
syncedPath = syncedPath + "/" + oldName
RNFS.moveFile(oldpath, syncedPath)
.then((success) => {
console.log("files moved successfully")
})
.catch((err) => {
console.log(err.message)
});
}
It turns out the fault was on the nodejs side and nodemon was restarting the server every time a new file was found so we just moved the uploads folder outside the scope of the project
I am trying to upload a file from mobile to google bucket using ionic 4. Although a file can upload into the could. I am struggling to get the file properties out of file object.
Here is my method,
async selectAFile() {
const uploadFileDetails = {
name: '',
contentLength: '',
size: '',
type: '',
path: '',
};
this.fileChooser.open().then(uri => {
this.file.resolveLocalFilesystemUrl(uri).then(newUrl => {
let dirPath = newUrl.nativeURL;
const dirPathSegments = dirPath.split('/');
dirPathSegments.pop();
dirPath = dirPathSegments.join('/');
(<any>window).resolveLocalFileSystemURL(
newUrl.nativeURL,
function(fileEntry) {
uploadFileDetails.path = newUrl.nativeURL;
const file: any = getFileFromFileEntry(fileEntry);
//log 01
console.log({ file });
uploadFileDetails.size = file.size;
uploadFileDetails.name = `${newUrl.name
.split(':')
.pop()}.${file.type.split('/').pop()}`;
uploadFileDetails.type = file.type;
async function getFileFromFileEntry(fileEntry) {
try {
return await new Promise((resolve, reject) =>
fileEntry.file(resolve, reject)
);
} catch (err) {
console.log(err);
}
}
},
function(e) {
console.error(e);
}
);
});
});
// here uploadFileDetails is simller to what I declared at the top ;)
// I wan't this to be populated with file properties
// console.log(uploadFileDetails.name) --> //''
const uploadUrl = await this.getUploadUrl(uploadFileDetails);
const response: any = this.uploadFile(
uploadFileDetails,
uploadUrl
);
response
.then(function(success) {
console.log({ success });
this.presentToast('File uploaded successfully.');
this.loadFiles();
})
.catch(function(error) {
console.log({ error });
});
}
even though I can console.log the file in log 01. I am unable to get file properties like, size, name, type out of the resolveLocalFileSystemURL function. basically, I am unable to populate uploadFileDetails object. What am I doing wrong? Thank you in advance.
you actually need 4 Ionic Cordova plugins to upload a file after getting all the metadata of a file.
FileChooser
Opens the file picker on Android for the user to select a file, returns a file URI.
FilePath
This plugin allows you to resolve the native filesystem path for Android content URIs and is based on code in the aFileChooser library.
File
This plugin implements a File API allowing read/write access to files residing on the device.
File Trnafer
This plugin allows you to upload and download files.
getting the file's metadata.
file.resolveLocalFilesystemUrl with fileEntry.file give you all the metadata you need, except the file name. There is a property called name in the metadata but it always contains value content.
To get the human readable file name you need filePath. But remember you can't use returning file path to retrieve metadata. For that, you need the original url from fileChooser.
filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1) is used to get only file name from filePath.
You need nativeURL of the file in order to upload it. Using file path returning from filePath is not going to work.
getFileInfo(): Promise<any> {
return this.fileChooser.open().then(fileURI => {
return this.filePath.resolveNativePath(fileURI).then(filePathUrl => {
return this.file
.resolveLocalFilesystemUrl(fileURI)
.then((fileEntry: any) => {
return new Promise((resolve, reject) => {
fileEntry.file(
meta =>
resolve({
nativeURL: fileEntry.nativeURL,
fileNameFromPath: filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1),
...meta,
}),
error => reject(error)
);
});
});
});
});
}
select a file from the file system of the mobile.
async selectAFile() {
this.getFileInfo()
.then(async fileMeta => {
//get the upload
const uploadUrl = await this.getUploadUrl(fileMeta);
const response: Promise < any > = this.uploadFile(
fileMeta,
uploadUrl
);
response
.then(function(success) {
//upload success message
})
.catch(function(error) {
//upload error message
});
})
.catch(error => {
//something wrong with getting file infomation
});
}
uploading selected file.
This depends on your backend implementation. This is how to use File Transfer to upload a file.
uploadFile(fileMeta, uploadUrl) {
const options: FileUploadOptions = {
fileKey: 'file',
fileName: fileMeta.fileNameFromPath,
headers: {
'Content-Length': fileMeta.size,
'Content-Type': fileMeta.type,
},
httpMethod: 'PUT',
mimeType: fileMeta.type,
};
const fileTransfer: FileTransferObject = this.transfer.create();
return fileTransfer.upload(file.path, uploadUrl, options);
}
hope it helps. :)