I'm trying to use the native fetch() API in NodeJS to upload a file to a REST API.
So far, I've made other GET and POST requests successfully, but this file upload is causing me a lot of trouble.
I have the following function -
async function uploadDocumentToHub(hub_entity_id, document_path) {
let formData = new FormData();
formData.append("type", "Document");
formData.append("name", "ap_test_document.pdf");
formData.append("file", fs.createReadStream("ap_test_document.pdf"));
formData.append("entity_object_id", hub_entity_id);
const form_headers = {
Authorization: auth_code,
...formData.getHeaders(),
};
console.log(
`Uploading document ap_test_document.pdf to hub (${hub_entity_id}) `
);
console.log(formData);
let raw_response = await fetch(urls.attachments, {
method: "POST",
headers: form_headers,
body: formData,
});
console.log(raw_response);
}
which I then run with the following code -
async function async_main() {
......
.......
await uploadDocumentToHub(hub_entity_id, document_path);
}
// main();
async_main();
And I keep getting the following error -
node:internal/deps/undici/undici:5536
p.reject(Object.assign(new TypeError("fetch failed"), { cause: response.error }));
^
TypeError: fetch failed
at Object.processResponse (node:internal/deps/undici/undici:5536:34)
at node:internal/deps/undici/undici:5858:42
at node:internal/process/task_queues:140:7
at AsyncResource.runInAsyncScope (node:async_hooks:202:9)
at AsyncResource.runMicrotask (node:internal/process/task_queues:137:8)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
cause: TypeError: object2 is not iterable
at action (node:internal/deps/undici/undici:1660:39)
at action.next (<anonymous>)
at Object.pull (node:internal/deps/undici/undici:1708:52)
at ensureIsPromise (node:internal/webstreams/util:172:19)
at readableStreamDefaultControllerCallPullIfNeeded (node:internal/webstreams/readablestream:1884:5)
at node:internal/webstreams/readablestream:1974:7
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
}
I'm baffled about what's going on and what this error is about. Any ideas?
The following code correctly uploads the file (auto-generated from postman, some data <removed> for security) -
var axios = require('axios');
var FormData = require('form-data');
var fs = require('fs');
var data = new FormData();
data.append('type', 'Document');
data.append('name', 'ap_test_document.pdf');
data.append('file', fs.createReadStream('kX3bdHb1G/ap_test_document.pdf'));
data.append('entity_object_id', '<id>');
var config = {
method: 'post',
url: '<url>',
headers: {
'Authorization': '<token>',
...data.getHeaders()
},
data : data
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});
Some help would be much appreciated.
Thanks!
You can append a file to the Node 18 native Fetch by reading the file with fs.readFile and converting the contents to a Blob.
The following is some pseudo-code I whipped up in TypeScript (YMMV):
import fs from 'fs/promises';
async function createBlobFromFile(path: string): Promise<Blob> {
const file = await fs.readFile(path);
return new Blob([file]);
}
const path = '/path/to/file.txt';
const formData = new FormData();
formData.append('file', await createBlobFromFile(path), 'file.txt');
const response = await fetch('https://some.url', {
method: 'POST',
body: formData
});
You can't use the fetch API in Node.js, either you can install axios or node-fetch.
import fetch from 'node-fetch';
const body = {a: 1};
const response = await fetch('https://httpbin.org/post', {
method: 'post',
body: JSON.stringify(body),
headers: {'Content-Type': 'application/json'}
});
const data = await response.json();
console.log(data);```
https://www.npmjs.com/package/node-fetch
https://www.npmjs.com/package/axios
Related
I am trying to use the Piniata api. Here it is:
https://docs.pinata.cloud/
The idea is to upload and pin and image using the api, into my account in Piniata.
I got this sample to upload a file in base64, using Node.js and server side.
The sample use this api call:
"https://api.pinata.cloud/pinning/pinFileToIPFS"
I am suppose to be able to do this in client side as well.
However, there is no sample of client side without using Node.js. And I can't seem to find exactly a documentation of what the api call expects.
Here is the sample I got from the Piniata support:
const { Readable } = require("stream");
const FormData = require("form-data");
const axios = require("axios");
(async () => {
try {
const base64 = "BASE64 FILE STRING";
const imgBuffer = Buffer.from(base64, "base64");
const stream = Readable.from(imgBuffer);
const data = new FormData();
data.append('file', stream, {
filepath: 'FILENAME.png'
})
const res = await axios.post("https://api.pinata.cloud/pinning/pinFileToIPFS", data, {
maxBodyLength: "Infinity",
headers: {
'Content-Type': `multipart/form-data; boundary=${data._boundary}`,
pinata_api_key: pinataApiKey,
pinata_secret_api_key: pinataSecretApiKey
}
});
console.log(res.data);
} catch (error) {
console.log(error);
}
})();
Here is my attempt to perform an upload from client side without Node.js
async function uploadFile(base64Data)
{
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`;
var status = 0;
try {
let data = new FormData();
var fileName = "FILENAME.png";
var file = new File([base64Data], fileName, {type: "image/png+base64"});
data.append(`data`, file, file.name);
data.append(`maxBodyLength`, "Infinity");
const response = await postData('POST', url, {
'Content-Type': `multipart/form-data; boundary=${data._boundary}`,
"Authorization": "Bearer Redacted"
},
data
);
} catch (error) {
console.log('error');
console.log(error);
}
}
What I get as a response from the server is 400 and the error being:
{"error":"Invalid request format."}
What am I doing wrong?
Also, it seems like when I try to use FormData .append with a stream as a sample, it doesn't work. As if it only expects a blob.
I am attempting to upload a file using the Node example provided in the HubSpot docs.
I am receiving 415(Unsupported media type). The response says I am sending the header application/json even though I am setting multipart/form-data.
const uploadFile = async () => {
const postUrl = `https://api.hubapi.com/filemanager/api/v3/files/upload?hapikey=${HAPI_KEY}`;
const filename = `${APP_ROOT}/src/Files/Deal/4iG_-_CSM_Additional_Capacity/test.txt`;
const headers = {
'Content-Type': 'multipart/form-data'
}
var fileOptions = {
access: 'PUBLIC_INDEXABLE',
overwrite: false,
duplicateValidationStrategy: 'NONE',
duplicateValidationScope: 'ENTIRE_PORTAL'
};
var formData = {
file: fs.createReadStream(filename),
options: JSON.stringify(fileOptions),
folderPath: '/Root'
};
try {
const resp = await axios.post(postUrl, formData, headers); // API request
console.log(resp.data)
} catch (error) {
console.log("Error: ", error);
}
}
Can you see what the problem is or recommend a better way of uploading the file?
Thanks!
The Node example you link to uses the (deprecated) request module, not Axios.
To use Axios (source) you would rewrite that as:
const FormData = require('form-data');
const form = new FormData();
form.append('file', fs.createReadStream(filename));
form.append('options', JSON.stringify(fileOptions));
form.append('folderPath', '/Root');
const config = { headers: form.getHeaders() };
axios.post(postUrl, form, config);
We can Run API in Postman and check NodeJs - Axios Detail in Postman Code Snippet and I Think That's the Better way for this.
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');
const uploadFile = async () => {
try {
let data = new FormData();
data.append('folderPath', '/Root');
form.append('file', fs.createReadStream(`${APP_ROOT}/src/Files/Deal/4iG_-_CSM_Additional_Capacity/test.txt`));
data.append('options', JSON.stringify({
access: 'PUBLIC_INDEXABLE',
overwrite: false,
duplicateValidationStrategy: 'NONE',
duplicateValidationScope: 'ENTIRE_PORTAL'
}));
var config = {
method: 'post',
url: `https://api.hubapi.com/filemanager/api/v3/files/upload?hapikey=${HAPI_KEY}`,
headers: {
'Content-Type': 'multipart/form-data'
},
data: data
};
const resp = await axios(config); // API request
console.log(resp.data)
} catch (error) {
// error
}
}
I have some issues parsing a request made from the front-end using FormData. This is the example request generated from Postman for node.js Axios. If I use the postman app with the request, it works as expected.
Frontend example generated from Postman code feature.
var axios = require('axios');
var FormData = require('form-data');
var fs = require('fs');
var data = new FormData();
data.append('file', fs.createReadStream('/some_file.jpg')); **//I am using Electron and I have acces to the FileSystem from the client.**
data.append('resizeLargeImage[width]', '1920');
data.append('resizeLargeImage[height]', '1080');
data.append('resizeLargeImage[type]', 'cover');
var config = {
method: 'post',
url: 'localhost:3030/api/v1/optimize-single',
headers: {
'x-api-key': '123',
...data.getHeaders()
},
data : data
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});
Backend
#Post('/optimize-single')
#UseInterceptors(FileInterceptor('file'))
async uploadFile(
#UploadedFile() file: FileDto,
#Body() body: UploadFileParametersDto,
#Res() response: Response,
): Promise<any> {
console.log('file', file, 'body', body);
**//File is undefined, body is a null Object**
return await this.appService.uploadFile(file, body, response);
}
Any ideas as to why Nest doesn't recognize this type of request?
Thanks!
I managed to figure it out.
If you are in the Renderer process, it works by attaching the file as a blob.
const fileBuffer = fs.readFileSync(filePath);
const fileName = path.basename(filePath);
const blob = new Blob([fileBuffer], {
type: mime.lookup(filePath),
});
formData.append('file', blob, fileName);
If you move the same functionality in the Main process it will work as expected with data.append('file', fs.createReadStream('/some_file.jpg'));
presently I am attempting to make 2 different api calls one after the other within 1 java/nodejs script. It seems after my first function runs successfully, the second one errors with the following:
FetchError: request to failed, reason: socket hang up;
type: 'system',
errno: 'ECONNRESET',
code: 'ECONNRESET'
Below is a code snippet of what I have tried thus far:
const fetch = require("node-fetch");
const formData = require("form-data");
const fs = require("fs");
//const express = require("express");
var apiName = "<LOCAL SYSTEM FILE TO UPLOAD>";
var lookupName = "<LOCAL SYSTEM FILE TO UPLOAD>";
var accessToken = "Bearer <ACCESS TOKEN>";
var url = '<URL API #1>';
var url2 = '<URL API #2>;
var headers = {
'Accept': 'application/json',
'Authorization': accessToken,
};
const form = new formData();
const buffer2 = fs.readFileSync(lookupName);
const buffer = fs.readFileSync(apiName);
const uploadAPI = function uploadAPI() {
form.append("Content-Type", "application/octect-stream");
form.append('file', buffer);
fetch(url, {method: 'POST', headers: headers, body: form})
.then(data => {
console.log(data)
})
.catch(err => {
console.log(err)
});
};
const uploadLookup = function uploadLookup() {
form.append("Content-Type", "application/octect-stream");
form.append('file', buffer2);
fetch(url2, {method: 'PUT', headers: headers, body: form})
.then(data => {
console.log(data)
})
.catch(err => {
console.log(err)
});
};
if (!apiName !== true) {
uploadAPI()
} else {}
if (!lookupName !== true) {
console.log("Uploading Lookup File");
uploadLookup()
} else {}
I tried using a "setTimeout" function which does not seem to work as I would have liked it to. My best guess is each API call would need to be it's own separate socket connection? Any help with getting me in the right direction is appreciated.
Promise.all([
fetch(url),
fetch(url2)
]).then(function (res) {
// Get a JSON object from each of the responses
return res.json();
}).then(function (data) {
// do something with both sets of data here
console.log(data);
}).catch(function (error) {
// if there's an error, log it
});
I'm getting this error when trying to do a POST request using axios:
Error: Data after transformation must be a string, an ArrayBuffer, a Buffer, or a Stream at createError
Here's my request:
async function fetchAndHandleErrors() {
const url = `/claim/${claimId}`;
const headers = {
Accept: 'application/json',
Authorization: `Bearer ${token}`,
};
const body = new FormData();
body.append('damage_description', damageDescription);
body.append('damaged_phone', {
uri: imageUri,
type: 'image/jpeg', // or photo.type
name: imageUri,
});
const result = await axios({
'post',
url: `${baseUrl}${url}`,
data: body,
headers,
});
return result.data;
}
I tried removing result.data and still get the same error. Why is that?
If you eventually still need a solution for this, I managed to get rid of this error by using the formData.pipe() method. For your case, it could look like this :
import axios from 'axios'
import concat from 'concat-stream'
import fs from 'fs'
import FormData from 'form-data'
async function fetchAndHandleErrors() {
const file = fs.createReadStream(imageUri)
let body = new FormData();
body.append('damage_description', damageDescription);
body.append('damaged_phone', file);
body.pipe(concat(data => {
const url = `/claim/${claimId}`;
const headers = {
'Authorization': `Bearer ${token}`,
...body.getHeaders()
};
const result = await axios({
'post',
url: `${baseUrl}${url}`,
data: body,
headers,
});
return result.data;
}))
}
Please let me know if you still encounters your issue, I'll be glad to help !