I'm having issues uploading video files to cloudinary in my react app, deployed in netlify. In my App I have a react page with a form that sends the data to my API. My API handles the HTTP requests to my netlify functions (using Axios), and then with the serverless function I call the cloudinary node API to store the video file. The problem happens when I'm passing the data from my API to the serverless function, I'm getting "Error: Stream body too big", because the video exceeds the payload limit of netlify functions (6mb). Do I have to compress the file? It's okay to do it like this (frontend page -> api.js -> serverless function)? Thanks for all the help you guys provide everyday, it helped me a lot!
Files that I have and the error:
page.jsx
const formHandler = async (formValues) => {
try {
...
const res = await addExercise(formValues);
...
} catch (error) {
console.log("error ", error);
}
};
api.js
import { instance } from "./instance";
...
export const addExercise = async (data) => {
try {
const reader = new FileReader();
reader.readAsDataURL(data.exerciseVideo[0]);
reader.onloadend = async () => {
const cloudVideo = reader.result;
const cloudinaryResponse = await instance.post("upload-exerciseVideo", { cloudVideo });
...
}
} catch (error) {
console.log("error", error);
}
};
serverless function (upload-exerciseVideo.js)
import cloudinary from '../src/config/cloudinary';
export const handler = async (event, context) => {
try {
const data = JSON.parse(event.body);
const { cloudVideo } = data;
const cloudinaryRequest = await cloudinary.uploader
.upload(cloudVideo, {
resource_type: "video",
upload_preset: "z9qxfc6q"
});
...
} catch (error) {
console.log('error', error)
return {
statusCode: 400,
body: JSON.stringify(error)
}
}
}
Error:
Netlify Serverless functions are built on top of AWS Lambda functions so there is a hard limit to the size of the file and the amount of time it takes to run the code in the file. You didn't mention the size of your video, but the video does take longer to upload, and even if you are within the 1GB size limit, you may be exceeding the 10-second processing limit. Your video likely already has been compressed, so compression is not a viable option, and decompressing it in the serverless function would probably exceed the time limit. https://www.netlify.com/blog/intro-to-serverless-function.
If you're uploading a large file, like a video, from front-end code, consider using the Upload Widget with an unsigned preset. Here's a link to a code sandbox showing how to create and use the upload widget in React: https://codesandbox.io/s/cld-uw-uymkb. You will need to add your Cloudinary cloudname and an unsigned preset to make this work. You'll find instructions for creating unsigned presets here: https://cloudinary.com/documentation/upload_presets
I have an simple Node application that allows me to pass an AWS S3 URL link to a file (in this case video files). It uses the FFMPEG library to read the video file and return data like codecs, duration, bitrate etc..
The script is called from PHP script which in turn send the data to the Node endpoint and passes the Amazon S3 URL to node. Sometimes for no obvious reasons the video file fails to return the expected values regarding container, codec, duration etc... and just returns '0'. But when I try the exact same file/request again it returns this data correctly e.g container:mp4
I'm not sure but I think the script somehow needs the createWriteStream to be closed but I cannot be sure, the problem is the issue I have found doesn't happen all the time but sporadically so its hard to get to the issue when its difficult to replicate it.
Any ideas?
router.post('/', async function(req, res) {
const fileURL = new URL(req.body.file);
var path = fileURL.pathname;
path = 'tmp/'+path.substring(1); // removes the initial / from the path
let file = fs.createWriteStream(path); // create the file locally
const request = https.get(fileURL, function(response) {
response.pipe(file);
});
// after file has saved
file.on('finish', function () {
var process = new ffmpeg(path);
process.then(function (video) {
let metadata = formatMetadata(video.metadata);
res.send ({
status: '200',
data: metadata,
errors: errors,
response: 'success'
});
}, function (err) {
console.warn('Error: ' + err);
res.send ({
status: '400',
data: 'Something went wrong processing this video',
response: 'fail',
});
});
});
file.on('error', function (err) {
console.warn(err);
});
});
function formatMetadata(metadata) {
const data = {
'video' : metadata.video,
'audio' : metadata.audio,
'duration' : metadata.duration
};
return data;
}
// Expected output
{"data":{"video":{"container":"mov","bitrate":400,"stream":0,"codec":"h264","resolution":{"w":1280,"h":720},"resolutionSquare":{"w":1280,"h":720},"aspect":{"x":16,"y":9,"string":"16:9","value":1.7777777777777777},"rotate":0,"fps":25,"pixelString":"1:1","pixel":1},"audio":{"codec":"aac","bitrate":"127","sample_rate":44100,"stream":0,"channels":{"raw":"stereo","value":2}},"duration":{"raw":"00:00:25.68","seconds":25}}
// Actual output
{"data":{"video":{"container":"","bitrate":0,"stream":0,"codec":"","resolution":{"w":0,"h":0},"resolutionSquare":{"w":0,"h":null},"aspect":{},"rotate":0,"fps":0,"pixelString":"","pixel":0},"audio":{"codec":"","bitrate":"","sample_rate":0,"stream":0,"channels":{"raw":"","value":""}},"duration":{"raw":"","seconds":0}}
Note - this happens sporadically
You are not accounting for a failed fetch from AWS. You should check the status code of the response before you move on to your pipe.
const request = https.get(fileURL, function(response) {
if(response.statusCode == 200)
response.pipe(file);
else
// Handle error case
});
I have a voice-recorder that records user-audio and stores it as state in a ReactJS component. When I pass the file to my API route it is recognised as a string and I don't seem to be able to upload it to AWS S3 correctly.
Here is the ReactJS button Handler:
const [audioFile, setAudioFile] = useState(null)
...
const submitVoiceMemo = async () => {
console.log('this is the payload', audioFile)
// the file is named 'audio.mp3'
try {
await fetch('/api/createVoiceMemo', {
method: 'PUT',
body: audioFile
})
} catch (error) {
console.error(error)
}
}
However, when the file 'arrives' at the API Route it is a string and I don't know how to convert it back to an MP3:
module.exports = requireAuth(async (req, res) => {
console.log(req.body)
// this prints 'characters' the to the console and is typeof string
I also can't seem to restructure req.body - if I try to console.log(req.body.audioFile) I get undefined.
How do I format the file so that I can upload it as an MP3 to AWS S3?
I'm using Vue js and firebase for my SPA app. I want to upload image, resize it to fixed dimension and upload it to firebase.
I'm using Jimp npm package to resize the uploaded image.
Below is my code:
Main.js
import Jimp from 'jimp';
Window.Jimp = Jimp;
Snippet that handle upload image
uploadImage(e) {
if (e.target.files[0]) {
let file = e.target.files[0];
Jimp.read(file)
.then(lenna => {
return lenna
.resize(256, 256) // resize
.quality(60) // set JPEG quality
.write(file.name); // save
})
.catch(err => {
console.error(err);
});
conslole.log(lenna);
var storageRef = fb.storage().ref("products/" + file.name);
let uploadTask = storageRef.put(file);
uploadTask.on(
"state_changed",
snapshot => {},
error => {},
() => {
uploadTask.snapshot.ref.getDownloadURL().then(downloadURL => {
this.product.images.push(downloadURL);
console.log("File available at", downloadURL);
});
}
);
}
}
With the above code I get the following error message:
No matching constructor overloading was found. Please see the docs for how to call the Jimp constructor.
Can anyone help me to figure out the mistake I'm making here ?
I want to upload a photo in firebase storage and once its uploaded a downloadURL to be printed on console.
Photo is uploaded perfectly without any issue but in console instead of downloadURL , undefined is printed and an error thrown which states that 'TypeError: Cannot read property 'ref' of undefined'.
const file = $('#exampleInputFile').get(0).files[0]; // getting file to be uploaded
const name = (+new Date() + '-' + file.name); // creating filename with timestamp
const task = ref.child(name).put(file, {contentType: file.type}); //setting file
task.then( (snapshot) => console.log (snapshot.downloadURL))
.then(downloadURL => {
console.log(`Successfully uploaded file and got download link -
${downloadURL}`);
// once done trying to get downloadURL against this upload
})
.catch(error => {
// Use to signal error if something goes wrong.
console.log(`Failed to upload file and get link - ${error}`);
});
//or throw and error in console
save() {
this.storageref.child(this.file.name).put(this.file).then(snapshot => {
let image = snapshot.metadata.downloadURLs[0];
consol.log(image);
});
}