I have a sting variable of a csv. I want to upload it to a slack channel as a .csv file, not as text.
async function run() {
const csvData = 'foo,bar,baz';
const url = 'https://slack.com/api/files.upload';
const res = await axios.post(url, {
channel: '#csvchannel',
filename: 'CSVTest.csv',
content: csvData
}, { headers: { authorization: `Bearer ${slackToken}` } });
console.log('Done', res.data);
}
This code returns: error: 'no_file_data', Changing content to file gives the same response.
What do I have to do to convert the csv sting into a file that can be uploaded? I can't use fs to write out the file.
I have tried to use fs.createReadStream(csvData) but that needs a file, not a string.
Slack API documentation: https://api.slack.com/methods/files.upload
You don't need to convert the CSV into a file, seems you are missing a couple of things here:
fileType property, it needs to be CSV.
Slack file upload API supports multipart/form-data and
application/x-www-form-urlencoded content types.
You're missing the Content-Type.
Check out a working example of how you could send the data using application/x-www-form-urlencoded
Send a CSV to Slack
View in Fusebit
const csvData = 'foo,bar,baz';
const url = 'https://slack.com/api/files.upload';
const params = new URLSearchParams()
params.append('channels', slackUserId);
params.append('content', csvData);
params.append('title', 'CSVTest');
params.append('filetype', 'csv');
const result = await axios.post(url, params,
{
headers:
{
authorization: `Bearer ${access_token}`,
'Content-Type': 'application/x-www-form-urlencoded'
}
});
ctx.body = { message: `Successfully sent a CSV file to Slack user ${slackUserId}!` };
Related
I am working on a project (using React.js Express.js and Node.js) to convert a user uploaded image into and NFT on Ethereum blockchain and for that, I need to upload the image on an IPFS (I am using Pinata) and then use the pinata URI in the metadata to mint a new NFT. (Do let me know if I am wrong here, I am still newbie to web3)
For uploading my image onto the Pinata IPFS, I am sending the base64 string of the image from the client side to the server side and then calling the pinFileToIPFS method. This is the code of my server side file
const axios = require('axios');
const fs = require('fs');
const FormData = require('form-data');
const router = require('express').Router();
const { Readable } = require('stream');
const pinFileToIPFS = (image) => {
const url = `https://api.pinata.cloud/pinning/pinJSONToIPFS`;
const buffer = Buffer.from(image);
const stream = Readable.from(buffer);
const filename = `an-awesome-nft_${Date.now()}.png`;
stream.path = filename;
const formData = new FormData();
formData.append("file", stream);
return axios.post(url,
formData,
{
headers: {
'Content-Type': `multipart/form-data; boundary= ${formData._boundary}`,
'pinata_api_key': "*******************",
'pinata_secret_api_key': "**********************************",
}
}
).then(function (response) {
console.log("Success: ", response);
}).catch(function (error) {
console.log("Fail! ", error.response.data);
});
};
router.route('/').post((req, res) => {
const image = req.body.image;
pinFileToIPFS(image);
});
module.exports = router;
Here req.body.image contains the base64 string of the user uploaded file.
I have tried to convert the base64 string into a buffer and then convert the buffer into a readable stream (as done in the official Pianata documentation but for a localy file) and then wrap it up in FormData(), but I keep getting the following error.
data: {
error: 'This API endpoint requires valid JSON, and a JSON content-type'
}
I know the problem is with the format my image/file is being sent to the API but I can't figure out. I am still a newbie to web3 and blockchains so please help!
The recommended way of interacting with Pinata, is by using their Node.JS SDK. This SDK has a pinFileToIPFS function, allows you to upload an image to their IPFS nodes in the form of a readableStream.
A sample of this would look like
const fs = require('fs');
const readableStreamForFile = fs.createReadStream('./yourfile.png');
const options = {
pinataMetadata: {
name: MyCustomName,
keyvalues: {
customKey: 'customValue',
customKey2: 'customValue2'
}
},
pinataOptions: {
cidVersion: 0
}
};
pinata.pinFileToIPFS(readableStreamForFile, options).then((result) => {
//handle results here
console.log(result);
}).catch((err) => {
//handle error here
console.log(err);
});
However, if you are deadset on using their API endpoints and simply posting to them via axios, there is a seperate API endpoint. /pinning/pinFileToIPFS. Examples of this method can be found in their API Docs.
You may want to consider changing the following two lines and using the https://api.pinata.cloud/pinning/pinFileToIPFS endpoint instead:
const buffer = Buffer.from(image); -> const buffer = Buffer.from(image, "base64");
and
formData.append("file", stream); -> formData.append("file", stream, "fileNameOfChoiche.png);
When you are uploading an image or file to Pinata IPFS with node js. These are the steps that even don't need Pinata Node.js SDK.
1- You can upload an image from the front end with React or Next.js. Code is given below.
const uploadAttachment = async (data, token) => {
try {
return await Api.post(`${ApiRoutes.upload_attachment}`, data, {
headers: {
Authorization: "Bearer " + token, //the token is a variable which holds the token
},
});
} catch (error) {
return {
status: 404,
};
}
};
export default uploadAttachment;
2- You need to install multer to upload an image.
const multer = require("multer");
global.uploadSingleFile = multer({ dest: "uploads/" });
3- Set up your route with multer middleware and action which you are going to call.
.post(
"/attachments/upload",
uploadSingleFile.single("file"),
actions.attachments.upload.pinFileToIPFSLocal
);
4- Last step where you will hit the Pinata endpoint with Pinata API & Secret key.
pinFileToIPFSLocal: async (req, res, next) => {
try {
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
let formData = new FormData();
formData.append("file", JSON.stringify(req.file), req.file.originalname);
axios
.post(url, formData, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${formData._boundary}`,
pinata_api_key: process.env.PINATA_API_KEY,
pinata_secret_api_key: process.env.PINATA_API_SECRET,
path: "somename",
},
})
.then((data) => {
console.log("Result...", data);
return utils.response.response(
res,
"Upload image to ipfs.",
true,
200,
data.data
);
})
.catch((err) => {
return utils.response.response(
res,
"Image not upload to ipfs",
false,
200,
err
);
});
} catch (error) {
next(error);
}
The error message is clear. You are using url that used for json file upload. this is the url you should use to upload image
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`;
you don't have to convert buffer to a readable stream.
I am not sure if this is correct ${formData._boundary}. should be
"Content-Type": `multipart/form-data: boundary=${formData.getBoundary()}`,
There must be an error on the image parameter. A simple buffer representation of the image should work. The readable stream is not necessary.
Instead of creating the buffer, you could use middleware like express-fileupload to access the buffer representation of the file uploaded on the client-side directly.
const file = req.files;
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
const data = new FormData();
data.append("file", file.file.data, { filepath: "anyname" });
const result = await axios.post(url, data, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${data._boundary}`,
pinata_api_key: process.env.PINATA_API_KEY,
pinata_secret_api_key: process.env.PINATA_API_SECRET,
path: "somename",
},
});
I've been trying to create a file using the Google APIs for Browser. I reused some of the code that I used to communicate with the api from NodeJS in the past and repurposed it for the browser.
const content = "this is some content";
const fileMetadata = {
name: "my-file.txt",
alt: "media",
};
const media = {
mimeType: "text/plain",
body: content,
};
const {
result: { id: fileId },
} = await gapi.client.drive.files.create({
resource: fileMetadata,
media: media,
fields: "id",
});
I typically get a successful response saying that my file was created.
However, when I try to get the contents of the file the body field is an empty string.
const { body } = await gapi.client.drive.files.get({
fileId: fileId,
});
console.log(body)
// ""
I think the request to create the file might not be formatted correctly but it works when it runs on the backend so I'm confused as to why it doesn't work in the browser.
You are mentioning that you are using the Google APIs for browser, and not node.js.
I would recommend to send the request directly against the Google REST API, as gapi.client.drive.create() appears to have problems sending the actual binary file (while sending metadata appears to work). Look here, for example: https://stackoverflow.com/a/53841879/7821823, https://stackoverflow.com/a/35182924/7821823 or https://stackoverflow.com/a/68595887/7821823
You can send the data as a blob and create the request using the FormData class.
async upload(blob, name, mimeType, parents = ["root"]) {
const metadata = { name, mimeType, parents };
const form = new FormData();
form.append("metadata", new Blob([JSON.stringify(metadata)], { type: "application/json" }));
form.append("file", blob);
return fetch("https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart&supportsAllDrives=true", {
method: "POST",
headers: new Headers({ Authorization: `Bearer ${gapi.auth.getToken().access_token}` }),
body: form,
});
}
I have not tested if you can send a String instead of a Blob, but you can easily create a Blob from a String:
const content = "this is some content";
const blob = new Blob([content], { type: 'text/plain' });
The body of the file should be in the form of a stream. Not plain text.
var media = {
mimeType: 'text/plain',
body: fs.createReadStream('files/my-file.txt')
};
as for console.log(body)
Also file.get returns a file.resource File resource does not give you the option to see the contents of the file. Google Drive doesn't have the ability to read the file itself it can only tell you about the file.
I have a problem. In my project I get a text and send this text to remote API in .txt file. Now the program does this: getting a text, saving a text in a .txt file in filesystem, uploading a .txt file to remote API. Unfortunately, remote API accepts only files, I can't send plain text in request.
//get the wallPost with the field text
fs.writeFileSync(`./tmp/${wallPostId}.txt`, wallPost.text)
remoteAPI.uploadFileFromStorage(
`${wallPostPath}/${wallPostId}.txt`,
`./tmp/${wallPostId}.txt`
)
UPD: In function uploadFileFromStorage, I made a PUT request to remote api with writing a file. Remote API is API of cloud storage which can save only files.
const uploadFileFromStorage = (path, filePath) =>{
let pathEncoded = encodeURIComponent(path)
const requestUrl = `https://cloud-api.yandex.net/v1/disk/resources/upload?&path=%2F${pathEncoded}`
const options = {
headers: headers
}
axios.get(requestUrl, options)
.then((response) => {
const uploadUrl = response.data.href
const headersUpload = {
'Content-Type': 'text/plain',
'Accept': 'application/json',
'Authorization': `${auth_type} ${access_token}`
}
const uploadOptions = {
headers: headersUpload
}
axios.put(
uploadUrl,
fs.createReadStream(filePath),
uploadOptions
).then(response =>
console.log('uploadingFile: data '+response.status+" "+response.statusText)
).catch((error) =>
console.log('error uploadFileFromStorage '+ +error.status+" "+error.statusText)
)
})
But i guess in the future such a process will be slow. I want to create and upload a .txt file in RAM memory (without writing on drive). Thanks for your time.
You're using the Yandex Disk API, which expects files because that's what it's for: it explicitly stores files on a remote disk.
So, if you look at that code, the part that supplies the file content is supplied via fs.createReadStream(filePath), which is a Stream. The function doesn't care what builds that stream, it just cares that it is a stream, so build your own from your in-memory data:
const { Readable } = require("stream");
...
const streamContent = [wallPost.text];
const pretendFileStream = Readable.from(streamContent);
...
axios.put(
uploadUrl,
pretendFileStream,
uploadOptions
).then(response =>
console.log('uploadingFile: data '+response.status+" "+response.statusText)
)
Although I don't see anything in your code that tells the Yandex Disk API what the filename is supposed to be, but I'm sure that's just because you edited the post for brevity.
I have been stuck with trying to upload a video to S3 for a while and was hoping to get some pointers. Currently, what I've read and was told is that we need to send an actual file to S3 and not the url (which we might do if we were sending it to the backend before aws).
I am trying to do this by
const getBlob = async (fileURi) => {
console.log('THIS IS IT', fileURi);
const resp = await fetch(fileURi);
const videoBody = await resp.blob();
console.log(videoBody);
};
getBlob(video.uri);
The problem I am having is I am unable to actually get the video file. When I stop recording a video with await camera.stopRecording(); what I get in return is
Object {
"uri": "file:///path/20DD0E08-11CA-423D-B83D-BD5ED40DFB25.mov",
}
Is there a recommended approach in order to successfully get the actual file in order to send it to S3 through the client?
The way I am trying to currently send the video which doesn't work is:
const formData = new FormData();
formData.append('file', video.uri);
await fetch(url, {
method: 'POST',
body: formData,
headers: {
'Content-Type': 'multipart/form-data'
}
url: refers to the presignedUrl we get in return from aws.
P.S - Sending to the server through a fetch call does work but I noticed this approach also leave the User waiting for 10+ seconds since I need to send the video to the server then wait for it to finish uploading in AWS.
Thank you for all the help.
If I understand correctly you know how to upload file to your own server, but you want to send it directly to S3.
In that case I would suggest to use presigned URLs. https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html
You can generate presigned URL on your backend, it is basically regular URL pointing to S3 file and some key values. You need to send those values to mobile app and do the same fetch call you are already using, but replace url with the one generated on backend and add all key-values to FormData.
Example for node backend would look like this
import AWS from 'aws-sdk';
...
const client = new AWS.S3(config);
...
const presignedUrl = client.createPresignedPost({
Bucket: 'example-bucket-name',
Fields: { key: 'example-file-name' },
});
and in mobile app you would
const form = new FormData();
Object.keys(presignedUrl.fields).forEach(key => {
form.append(key, presignedUrl.fields[key]);
})
form.append('file', fileToUpload);
await fetch(presignedUrl.url, {
method: 'POST',
body: form,
headers: {
'Content-Type': 'multipart/form-data'
}
})
My solutions. Please review the sample application. https://github.com/expo/examples/tree/master/with-aws-storage-upload
const response = await fetch(video.uri);
const blob = await response.blob();
const params = {
Bucket: myBucket,
Metadata: {
long: long.toString(),
lat: lat.toString(),
size: videoSize.toString()
},
Key: myKey,
Body: blob
};
I am working on integrating Cloudinary with my React Native app and am running into a problem when I go to upload using the Cloudinary API. I'm using React Native Image Picker to select an image from the camera roll and using that I get a source uri - example below.
I am getting an error response back from Cloudinary and I'm not sure what it's referring to. "Invalid file parameter. Make sure your file parameter does not include '[]'"
When I use the debugger, I can console log out all the params I am sending in the body of my request. Any suggestions would be much appreciated!
source.uri: /Users/IRL/Library/Developer/CoreSimulator/Devices/817C678B-7028-4C1C-95FF-E6445FDB2474/data/Containers/Data/Application/BF57AD7E-CA2A-460F-8BBD-2DA6846F5136/Documents/A2F21A21-D08C-4D60-B005-67E65A966E62.jpg
async postToCloudinary(source) {
let timestamp = (Date.now() / 1000 | 0).toString();
let api_key = ENV.cloudinary.api;
let api_secret = ENV.cloudinary.api_secret
let cloud = ENV.cloudinary.cloud_name;
let hash_string = 'timestamp=' + timestamp + api_secret
let signature = CryptoJS.SHA1(hash_string).toString();
let upload_url = 'https://api.cloudinary.com/v1_1/' + cloud + '/image/upload'
try {
let response = await fetch(upload_url, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({
file: {
uri: source.uri,
type: 'image/jpeg'
},
api_key: api_key,
timestamp: timestamp,
signature: signature
})
});
let res = await response.json();
console.log(res);
} catch(error) {
console.log("Error: ", error);
}
}
UPDATE
So I now have base64 encoding working, I think, but I am still getting the same error.
var wordArray = CryptoJS.enc.Utf8.parse(source.uri);
var file = CryptoJS.enc.Base64.stringify(wordArray);
try {
let response = await fetch(upload_url, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({
file: {
uri: file,
type: 'image/jpeg;base64'
},
api_key: api_key,
timestamp: timestamp,
signature: signature
})
});
So it turns out the source data I was passing in was not formatted correctly. I was able to pass it in from the ImagePicker plugin I was using as an already formatted data URI (the ImagePicker example comes with two ways to capture your source file and I was using the wrong one). I was able to get rid of the CryptoJS stuff and simply pass in file: source.uri
If you are using axios, make sure to include {'X-Requested-With': 'XMLHttpRequest'} in your request headers. eg.
const uploadAgent = axios.create({
baseURL: 'https://api.cloudinary.com',
headers: {
'X-Requested-With': 'XMLHttpRequest'
}
});