I have a problem. In my project I get a text and send this text to remote API in .txt file. Now the program does this: getting a text, saving a text in a .txt file in filesystem, uploading a .txt file to remote API. Unfortunately, remote API accepts only files, I can't send plain text in request.
//get the wallPost with the field text
fs.writeFileSync(`./tmp/${wallPostId}.txt`, wallPost.text)
remoteAPI.uploadFileFromStorage(
`${wallPostPath}/${wallPostId}.txt`,
`./tmp/${wallPostId}.txt`
)
UPD: In function uploadFileFromStorage, I made a PUT request to remote api with writing a file. Remote API is API of cloud storage which can save only files.
const uploadFileFromStorage = (path, filePath) =>{
let pathEncoded = encodeURIComponent(path)
const requestUrl = `https://cloud-api.yandex.net/v1/disk/resources/upload?&path=%2F${pathEncoded}`
const options = {
headers: headers
}
axios.get(requestUrl, options)
.then((response) => {
const uploadUrl = response.data.href
const headersUpload = {
'Content-Type': 'text/plain',
'Accept': 'application/json',
'Authorization': `${auth_type} ${access_token}`
}
const uploadOptions = {
headers: headersUpload
}
axios.put(
uploadUrl,
fs.createReadStream(filePath),
uploadOptions
).then(response =>
console.log('uploadingFile: data '+response.status+" "+response.statusText)
).catch((error) =>
console.log('error uploadFileFromStorage '+ +error.status+" "+error.statusText)
)
})
But i guess in the future such a process will be slow. I want to create and upload a .txt file in RAM memory (without writing on drive). Thanks for your time.
You're using the Yandex Disk API, which expects files because that's what it's for: it explicitly stores files on a remote disk.
So, if you look at that code, the part that supplies the file content is supplied via fs.createReadStream(filePath), which is a Stream. The function doesn't care what builds that stream, it just cares that it is a stream, so build your own from your in-memory data:
const { Readable } = require("stream");
...
const streamContent = [wallPost.text];
const pretendFileStream = Readable.from(streamContent);
...
axios.put(
uploadUrl,
pretendFileStream,
uploadOptions
).then(response =>
console.log('uploadingFile: data '+response.status+" "+response.statusText)
)
Although I don't see anything in your code that tells the Yandex Disk API what the filename is supposed to be, but I'm sure that's just because you edited the post for brevity.
Related
I making a cloudflare worker that have to fetch a url pointing to a file. I want to redirect the request to download this file. The download happens, but with no extension and no name of the file. Can i read the response (return binary/octet-stream) and set the filename before download the file? Or some way to read the binary response and build a file with a name and download the file?
Im using fetch:
let file_response = await fetch(url)
.then((data) => {
}
thanks for the help!
After a while, i found a way. Maybe can help others!
var requestOptions = {
method: 'GET',
redirect: 'follow'
};
let file_response = await fetch(url,requestOptions)
let data = await file_response.blob();
contentDisposition = "attachment; filename=" + "file123.zip";
return new Response(data, {
status: 200,
headers: { "content-type": "application/octet-stream", "Content-Disposition": contentDisposition}
})
and now, the file is downloaded with the given name "file123.zip"
I have a sting variable of a csv. I want to upload it to a slack channel as a .csv file, not as text.
async function run() {
const csvData = 'foo,bar,baz';
const url = 'https://slack.com/api/files.upload';
const res = await axios.post(url, {
channel: '#csvchannel',
filename: 'CSVTest.csv',
content: csvData
}, { headers: { authorization: `Bearer ${slackToken}` } });
console.log('Done', res.data);
}
This code returns: error: 'no_file_data', Changing content to file gives the same response.
What do I have to do to convert the csv sting into a file that can be uploaded? I can't use fs to write out the file.
I have tried to use fs.createReadStream(csvData) but that needs a file, not a string.
Slack API documentation: https://api.slack.com/methods/files.upload
You don't need to convert the CSV into a file, seems you are missing a couple of things here:
fileType property, it needs to be CSV.
Slack file upload API supports multipart/form-data and
application/x-www-form-urlencoded content types.
You're missing the Content-Type.
Check out a working example of how you could send the data using application/x-www-form-urlencoded
Send a CSV to Slack
View in Fusebit
const csvData = 'foo,bar,baz';
const url = 'https://slack.com/api/files.upload';
const params = new URLSearchParams()
params.append('channels', slackUserId);
params.append('content', csvData);
params.append('title', 'CSVTest');
params.append('filetype', 'csv');
const result = await axios.post(url, params,
{
headers:
{
authorization: `Bearer ${access_token}`,
'Content-Type': 'application/x-www-form-urlencoded'
}
});
ctx.body = { message: `Successfully sent a CSV file to Slack user ${slackUserId}!` };
I've been trying to create a file using the Google APIs for Browser. I reused some of the code that I used to communicate with the api from NodeJS in the past and repurposed it for the browser.
const content = "this is some content";
const fileMetadata = {
name: "my-file.txt",
alt: "media",
};
const media = {
mimeType: "text/plain",
body: content,
};
const {
result: { id: fileId },
} = await gapi.client.drive.files.create({
resource: fileMetadata,
media: media,
fields: "id",
});
I typically get a successful response saying that my file was created.
However, when I try to get the contents of the file the body field is an empty string.
const { body } = await gapi.client.drive.files.get({
fileId: fileId,
});
console.log(body)
// ""
I think the request to create the file might not be formatted correctly but it works when it runs on the backend so I'm confused as to why it doesn't work in the browser.
You are mentioning that you are using the Google APIs for browser, and not node.js.
I would recommend to send the request directly against the Google REST API, as gapi.client.drive.create() appears to have problems sending the actual binary file (while sending metadata appears to work). Look here, for example: https://stackoverflow.com/a/53841879/7821823, https://stackoverflow.com/a/35182924/7821823 or https://stackoverflow.com/a/68595887/7821823
You can send the data as a blob and create the request using the FormData class.
async upload(blob, name, mimeType, parents = ["root"]) {
const metadata = { name, mimeType, parents };
const form = new FormData();
form.append("metadata", new Blob([JSON.stringify(metadata)], { type: "application/json" }));
form.append("file", blob);
return fetch("https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart&supportsAllDrives=true", {
method: "POST",
headers: new Headers({ Authorization: `Bearer ${gapi.auth.getToken().access_token}` }),
body: form,
});
}
I have not tested if you can send a String instead of a Blob, but you can easily create a Blob from a String:
const content = "this is some content";
const blob = new Blob([content], { type: 'text/plain' });
The body of the file should be in the form of a stream. Not plain text.
var media = {
mimeType: 'text/plain',
body: fs.createReadStream('files/my-file.txt')
};
as for console.log(body)
Also file.get returns a file.resource File resource does not give you the option to see the contents of the file. Google Drive doesn't have the ability to read the file itself it can only tell you about the file.
I have been stuck with trying to upload a video to S3 for a while and was hoping to get some pointers. Currently, what I've read and was told is that we need to send an actual file to S3 and not the url (which we might do if we were sending it to the backend before aws).
I am trying to do this by
const getBlob = async (fileURi) => {
console.log('THIS IS IT', fileURi);
const resp = await fetch(fileURi);
const videoBody = await resp.blob();
console.log(videoBody);
};
getBlob(video.uri);
The problem I am having is I am unable to actually get the video file. When I stop recording a video with await camera.stopRecording(); what I get in return is
Object {
"uri": "file:///path/20DD0E08-11CA-423D-B83D-BD5ED40DFB25.mov",
}
Is there a recommended approach in order to successfully get the actual file in order to send it to S3 through the client?
The way I am trying to currently send the video which doesn't work is:
const formData = new FormData();
formData.append('file', video.uri);
await fetch(url, {
method: 'POST',
body: formData,
headers: {
'Content-Type': 'multipart/form-data'
}
url: refers to the presignedUrl we get in return from aws.
P.S - Sending to the server through a fetch call does work but I noticed this approach also leave the User waiting for 10+ seconds since I need to send the video to the server then wait for it to finish uploading in AWS.
Thank you for all the help.
If I understand correctly you know how to upload file to your own server, but you want to send it directly to S3.
In that case I would suggest to use presigned URLs. https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html
You can generate presigned URL on your backend, it is basically regular URL pointing to S3 file and some key values. You need to send those values to mobile app and do the same fetch call you are already using, but replace url with the one generated on backend and add all key-values to FormData.
Example for node backend would look like this
import AWS from 'aws-sdk';
...
const client = new AWS.S3(config);
...
const presignedUrl = client.createPresignedPost({
Bucket: 'example-bucket-name',
Fields: { key: 'example-file-name' },
});
and in mobile app you would
const form = new FormData();
Object.keys(presignedUrl.fields).forEach(key => {
form.append(key, presignedUrl.fields[key]);
})
form.append('file', fileToUpload);
await fetch(presignedUrl.url, {
method: 'POST',
body: form,
headers: {
'Content-Type': 'multipart/form-data'
}
})
My solutions. Please review the sample application. https://github.com/expo/examples/tree/master/with-aws-storage-upload
const response = await fetch(video.uri);
const blob = await response.blob();
const params = {
Bucket: myBucket,
Metadata: {
long: long.toString(),
lat: lat.toString(),
size: videoSize.toString()
},
Key: myKey,
Body: blob
};
Server Side
I trying to send file from NodeJs
/**
* Exports data to PDF format route.
*/
app.post('/export/pdf', upload.single('imageBlob'), function (request, response) {
var PDF = require('./services/PdfService').PDF;
var fileUrl = PDF.generatePDFExport(request.body, request.file.buffer);
setTimeout(() => {
response.sendFile(fileUrl);
}, 200);
});
This piece of code creates a valid pdf file (I can open it browsers URL hit file)
But some browser hides the pop-up window and I wanted to download a file instead of opening it.
I check response in client and it is some BLOB looking response.
Client Side
I try to create a file from the response but there is only an empty pdf file.
return axios.post('http://172.18.0.2:8001/export/pdf', formData).then(response => {
let blob = new Blob([response.data]);
FileSaver.saveAs(blob, "st-seatmap-shop.pdf");
})
What is a mistake here? On the server side with a sending file or on the client with saving file?
The only problem was in sending a request to the server.
Server by default returns stream and for saving file on client response needs to be a BLOB so I just updated request.
let requestOptions = {
responseType: 'blob'
};
return axios.post('http://172.18.0.2:8001/export/pdf', formData, requestOptions).then(response => {
let blob = new Blob([response.data]);
FileSaver.saveAs(blob, "st-seatmap-shop.pdf");
}).catch(error => {
console.log("error.response is : ", error);
});