We Keep Coding

Image upload functionality not working on deployed Heroku app but working on Localhost?

So I created my first big project: https://rate-n-write.herokuapp.com/
In brief, this is a blog app where the user can write reviews and publish them along with pictures.
I have used firebase as the database to store the articles. The app is working fine on localhost. Whenever I am trying to upload an image on Heroku, I get this error
The error is showing up in line number 8 of the following code (editor.js):
uploadInput.addEventListener('change', () => {
uploadImage(uploadInput, "image");
})
const uploadImage = (uploadFile, uploadType) => {
const [file] = uploadFile.files;
if(file && file.type.includes("image")){
const formdata = new FormData();
formdata.append('image', file);
//Error shows up here in the fetch line
fetch('/upload', {
method: 'post',
body: formdata
}).then(res => res.json())
.then(data => {
if(uploadType == "image"){
addImage(data, file.name);
} else{
bannerPath = `${location.origin}/${data}`;
banner.style.backgroundImage = `url("${bannerPath}")`;
}
})
const change_text = document.getElementById("uploadban");
change_text.innerHTML = " ";
} else{
alert("upload Image only");
}
}
This is just a snippet of the whole editor.js file.
Is it because I am trying to upload the file to the project directory? (server.js snippet below):
app.post('/upload', (req, res) => {
let file = req.files.image;
let date = new Date();
// image name
let imagename = date.getDate() + date.getTime() + file.name;
// image upload path
let path = 'public/uploads/' + imagename;
// create upload
file.mv(path, (err, result) => {
if(err){
throw err;
} else{
// our image upload path
res.json(`uploads/${imagename}`)
}
})
})
Do I need to use an online storage service like AWS S3?

Heroku is not suitable for persistent storage of data, the uploaded pictures would be deleted after a while (when the dyno is restarted) read this.
I would suggest using 3rd party object Storage services like
cloudinary or AWS S3

Upload image to cloud storage from firebase cloud functions

I am trying so hard to upload one image from cloud functions
I am sending an image from the web to the cloud function using onRequest. I am sending a base64 string and the fileName. Now I was following different tutorials on the internet and couldn't seem to solve my problem.
Here is my code. I think I am doing something wrong with the service account json. Although i generated the json file and used it but still it didn't work.
I get the error of The caller does not have permission at Gaxios._request when i don't use service account json
And when i do use serviceAccount.json then i get this error The "path" argument must be of type string. Received an instance of Object which is from file.createWriteStream() i think
Anyway here is the code can anyone please help me with this
The projectId that I am using is shown in the picture below
const functions = require("firebase-functions");
const admin = require("firebase-admin");
const projectId = functions.config().apikeys.projectid; // In the picture below
const stream = require("stream");
const cors = require("cors")({ origin: true });
const { Storage } = require("#google-cloud/storage");
// Enable Storage
const storage = new Storage({
projectId: projectId, // I did use serviceAccount json here but that wasn't working
});
// With serviceAccount.json code
// const storage = new Storage({
// projectId: projectId,
// keyFilename: serviceAccount,
// });
// This is giving the error of: The "path" argument must be of type string. Received an instance of Object
exports.storeUserProfileImage = functions.https.onRequest((req, res) => {
cors(req, res, async () => {
try {
const bucket = storage.bucket(`gs://${projectId}.appspot.com`);
let pictureURL;
const image = req.body.image;
const userId = req.body.userId;
const fileName = req.body.fileName;
const mimeType = image.match(
/data:([a-zA-Z0-9]+\/[a-zA-Z0-9-.+]+).*,.*/
)[1];
//trim off the part of the payload that is not part of the base64 string
const base64EncodedImageString = image.replace(
/^data:image\/\w+;base64,/,
""
);
const imageBuffer = Buffer.from(base64EncodedImageString, "base64");
const bufferStream = new stream.PassThrough();
bufferStream.end(imageBuffer);
// Define file and fileName
const file = bucket.file("images/" + fileName);
bufferStream
.pipe(
file.createWriteStream({
metadata: {
contentType: mimeType,
},
public: true,
validation: "md5",
})
)
.on("error", function (err) {
console.log("error from image upload", err.message);
})
.on("finish", function () {
// The file upload is complete.
console.log("Image uploaded");
file
.getSignedUrl({
action: "read",
expires: "03-09-2491",
})
.then((signedUrls) => {
// signedUrls[0] contains the file's public URL
console.log("Signed urls", signedUrls[0]);
pictureURL = signedUrls[0];
});
});
console.log("image url", pictureURL);
res.status(200).send(pictureURL);
} catch (e) {
console.log(e);
return { success: false, error: e };
}
});
});

const storage = new Storage({
projectId: projectId
keyFilename: "" // <-- Path to a .json, .pem, or .p12 key file
});
keyFilename accepts path to where your service account is stored and the credentials themselves.
folder
|-index.js
|-credentials
|-serviceAccountKey.json
If your directory structure looks like about then the path should be like this:
const storage = new Storage({
projectId: projectId
keyFilename: "./credentials/serviceAccountKey.json"
});
Do note that if you are using Cloud functions then the SDK will use Application Default Credentials so you don't have to pass those params. Simply initialize as shown below:
const storage = new Storage()

So first of all I didn't give any serviceaccounts because I am using the firebase cloud functions as #Dharmaraj said in his answer
Secondly, this was a permission problem in the google cloud platform which can be solved by going through the following steps
Go to your project's Cloud Console (https://console.cloud.google.com/) > IAM & admin > IAM, Find the App Engine default service account then click on the pencil at far left > Click on add role > In the filter field enter Service Account Token Creator and click on it save and you are good to go
Found this solution from here
https://github.com/firebase/functions-samples/issues/782

NodeJS: Get filename from readStream

TLDR; Can I get filename from readable stream?
Got a method to download file from Google Drive and save it to local directory as I'm requesting response as stream , I would like to save the file with the same name as it has on the drive without explicitly asking user.
Do I have to make 1 extra request prior, to get filename from drive , is that the only way ?
async fetchFile(fileID) {
// Get file as stream
const { data: gdriveFile } = await this.drive.files.get(
{
fileId: fileID,
alt: "media"
},
{
responseType: "stream"
}
)
// FIXME: get filename from gdrive
console.log("Saving File: " + JSON.stringify(gdriveFile))
// fs writeStream
// const destStream = fs.createWriteStream(
// path.resolve(destDir + "/" + filename || "bleh.log")
// )
// pipeline(gdriveFile, destStream).end(() => {
// return "File copied"
// })
}
Thnx

Uploading multiple large json files in one go React-native to NodeJS

I am trying to upload multiple large size JSON files from React-native to node js.
The files are being uploaded unless the file in larger in size, in which case, it does not upload in one try.
I suspect that:
Since the upload code is in a for loop the code is starting the upload but not waiting for the file to upload and starting to upload the next file
Is there any way to ensure that each file gets uploaded in one go?
syncFunction() {
var RNFS = require('react-native-fs');
var path = RNFS.DocumentDirectoryPath + '/toBeSynced';
RNFS.readDir(path)
.then((success) => {
for (let i = 0; i < success.length; i++) {
var fileName = success[i].name
var filePath = success[i].path
var uploadUrl = 'http://192.168.1.15:3333/SurveyJsonFiles/GetFiles/'
if (Platform.OS === 'android') {
filePath = filePath.replace("file://", "")
} else if (Platform.OS === 'ios') {
filePath = filePath
}
const data = new FormData();
data.append("files", {
uri: filePath,
type: 'multipart/form-data',
name: fileName,
});
const config = {
method: 'POST',
headers: {
'Accept': 'application/json',
},
body: data,
};
fetch(uploadUrl, config)
.then((checkStatusAndGetJSONResponse) => {
console.log(checkStatusAndGetJSONResponse);
this.moveFile(filePath, fileName)
}).catch((err) => {
console.log(err)
});
}
})
.catch((err) => {
console.log(err.message);
});
}
The JSON files will more than 50Mb depending on data, since it contains base64 image data the size will increase as the user takes more photos.
The app will be creating new files when the user records any information, There is no error message displayed for partial file upload.
The this.moveSyncedFiles() is moving the synced files to another folder so that the same file does not get uploaded multiple times
moveFile(oldpath, oldName) {
var syncedPath = RNFS.DocumentDirectoryPath + '/syncedFiles'
RNFS.mkdir(syncedPath)
syncedPath = syncedPath + "/" + oldName
RNFS.moveFile(oldpath, syncedPath)
.then((success) => {
console.log("files moved successfully")
})
.catch((err) => {
console.log(err.message)
});
}

It turns out the fault was on the nodejs side and nodemon was restarting the server every time a new file was found so we just moved the uploads folder outside the scope of the project

AWS S3 Upload after GET Request to Image, Not Uploading Correctly

I'm trying to upload an image to my AWS S3 bucket after downloading the image from another URL using Node (using request-promise-native & aws-sdk):
'use strict';
const config = require('../../../configs');
const AWS = require('aws-sdk');
const request = require('request-promise-native');
AWS.config.update(config.aws);
let s3 = new AWS.S3();
function uploadFile(req, res) {
function getContentTypeByFile(fileName) {
var rc = 'application/octet-stream';
var fn = fileName.toLowerCase();
if (fn.indexOf('.png') >= 0) rc = 'image/png';
else if (fn.indexOf('.jpg') >= 0) rc = 'image/jpg';
return rc;
}
let body = req.body,
params = {
"ACL": "bucket-owner-full-control",
"Bucket": 'testing-bucket',
"Content-Type": null,
"Key": null, // Name of the file
"Body": null // File body
};
// Grabs the filename from a URL
params.Key = body.url.substring(body.url.lastIndexOf('/') + 1);
// Setting the content type
params.ContentType = getContentTypeByFile(params.Key);
request.get(body.url)
.then(response => {
params.Body = response;
s3.putObject(params, (err, data) => {
if (err) { console.log(`Error uploading to S3 - ${err}`); }
if (data) { console.log("Success - Uploaded to S3: " + data.toString()); }
});
})
.catch(err => { console.log(`Error encountered: ${err}`); });
}
The upload succeeds when I test it out, however after trying to redownload it from my bucket the image is unable to display. Additionally, I notice after uploading the file with my function, the file listed in the bucket is much larger in filesize than the originally uploaded image. I'm trying to figure out where I've been going wrong but cannot find where. Any help is appreciated.

Try to open the faulty file with a text editor, you will see some errors written in it.
You can try using s3.upload instead of putObject, it works better with streams.