We Keep Coding

Convert mp4 or Avi to m3u8 in pure node js

I am searching a way to convert a mp4 or an .avi to .m3u8 in pure node js (firebase cloud function). Do you have ideas ?
Thank's, but I tried that :
const ffmpegInstaller = require('#ffmpeg-installer/ffmpeg');
const ffmpeg = require('fluent-ffmpeg');
ffmpeg.setFfmpegPath(ffmpegInstaller.path);
const ffmpeg_static = require('ffmpeg-static');
var cmd = ffmpeg('./flir_20191202T174341.mp4')
.setFfmpegPath(ffmpeg_static.path)
.videoBitrate(1024)
.videoCodec('divx')
.format('m3u8')
.on('end', () => {
// ...
})
.on('error', err => {
console.error(err);
})
.save('./file-out.m3u8');
console.log('Hello !');
console.log(cmd);
And I have this error :
Error: Cannot find ffmpeg
at /Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/processor.js:136:22
at FfmpegCommand.proto._getFfmpegPath (/Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/capabilities.js:90:14)
at FfmpegCommand.proto._spawnFfmpeg (/Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/processor.js:132:10)
at FfmpegCommand.proto.availableFormats.proto.getAvailableFormats (/Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/capabilities.js:517:10)
at /Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/capabilities.js:568:14
at nextTask (/Users/jeremy/Dev/ssv-api/node_modules/async/dist/async.js:4576:27)
at Object.waterfall (/Users/jeremy/Dev/ssv-api/node_modules/async/dist/async.js:4587:9)
at Object.awaitable [as waterfall] (/Users/jeremy/Dev/ssv-api/node_modules/async/dist/async.js:208:32)
at FfmpegCommand.proto._checkCapabilities (/Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/capabilities.js:565:11)
at /Users/jeremy/Dev/ssv-api/node_modules/fluent-ffmpeg/lib/processor.js:298:14
Any ideas ?
Thanks in advance.
Jérémy.

Found this answer: https://stackoverflow.com/a/42777596/8006046. It shows how you can run FFmpeg in the firebase cloud. You can replace 'path_or_readstream.mp4' with either the path to the file you want to convert, or, which is more probable in a cloud function, you could pass the readable stream with the file you want to convert.
const ffmpeg = require('fluent-ffmpeg');
const ffmpeg_static = require('ffmpeg-static');
var cmd = ffmpeg('path_or_readstream.mp4')
.setFfmpegPath(ffmpeg_static.path)
.videoBitrate(1024)
.videoCodec('divx')
.format('m3u8')
.on('end', () => {
// ...
})
.on('error', err => {
console.error(err);
})
.save('/tmp/file-out.m3u8');

read video from storage and send it to model ML kit in Firebase

Hoping you are all good. My team have a problem in our Graduation project .
we are converting any video that's send from android application to text .
any video uploaded to Firebase storage and we had finished our model ML in TensorFlow .
we tried to read video from storage in function cloud to send the video to model ,but we couldn't find any resources .
in last Try, our function code was like :
exports.readVideo = functions.storage
.object()
.onFinalize(async (object) => {
const bucket = admin.storage().bucket(object.bucket);
const tempFilePath = path.join(os.tmpdir(), object.name);
console.log(tempFilePath);
console.log('download');
// note download
await bucket
.file(object.name!)
.download({
destination: tempFilePath,
})
.then()
.catch((err) => {
console.log({
type: 'download',
err: err,
});
});
console.log('read');
// note read
let stream = await bucket
.file(object.name!)
.createReadStream({
start: 10000,
end: 20000,
})
.on('error', function (err) {
console.log('error 1');
console.log({ error: err });
})
.pipe(fs.createWriteStream(object.name));
await new Promise((resolve, reject) => {
stream.on('finish', resolve);
console.log('error 3');
stream.on('error', reject);
});
console.log('tempFile size2', fs.statSync(tempFilePath).size);­­­
return fs.unlinkSync(tempFilePath);
});
and there's an error log in error3 line :
Error:EROFS:read-only file system,open'zoom_0.mp4'.

returning status once the file is written to local from s3 bucket

trying to fetch a file from s3 bucket and storing it on the local, once its written to the local reading the file from the local and converting the data to json format and sending it.
i need to check whether the file is downloaded and written to local, once the file exist only read and convert it to json else send an error message.
once the file is on open i am writing the file and making end. So after end i can't send a return value. So how i can solve this one and use try catch to send proper error message.
const fetchFileDownloadAndWriteIt = () => {
let Bucket = "DataBucket";
let filename = "sample_data.csv";
let s3 = new AWS.S3();
const params = {
Bucket: Bucket,
Key: filename
};
return s3.getObject(params)
.promise()
.then(data => {
const file = fs.createWriteStream('./localdata/' + filename);
file.on("open", () => {
file.write(data.Body);
file.end();
})
.on("error", err => {
console.log("Error Occured while writing", err.message)
})
})
.catch(err => {
console.log("unable to fetch file from s3 Bucket", err.message)
})
}
exports.fetchData = async (req,res) => {
let fileDownloadAndWrite = await fetchFileAndDownloadWriteIt();
// need to check file is downloaded and written properly
const path = "./localdata/sample_data.csv";
const json = await csv().fromFile(path);
res.send({data: json})
}

You can return a new Promise instead of the one instead of the one you get by calling the SDK's API.
return new Promise((res, rej) => {
s3.getObject(params)
.promise()
.then(data => {
const file = fs.createWriteStream('./localdata/' + filename);
file
.on("open", () => {
file.write(data.Body);
file.end();
//success
res();
})
.on("error", err => {
rej(err);
})
})
.catch(err => {
rej(err);
})
});
This will resolve to undefined and rejected with the proper error occured, like while writing file, etc.
How to Call it in your handler?
Something like this would be fine.
exports.fetchData = async (req, res, next) => {
try {
await fetchFileDownloadAndWriteIt();
// need to check file is downloaded and written properly - here the file is actually downloaded and written properly.
const path = "./localdata/sample_data.csv";
const json = await csv().fromFile(path);
res.send({ data: json })
}
catch (err) {
return next(err);
}
}

Graphql Apollo Server + Vue => Image Upload

Im using Vue with vue-apollo in the frontend and graphql stand-alone Apollo Server 2 with mongodb through mongoose in the backend. I have a simple blog application in which posts also have an Image. Everything works fine except uploading Images. I want the images to be uploaded to my local filesystem in a folder on my backend and only the path to the image saved in my mongodb document.
the mutation:
async createPost(parent, args, context, info) {
//...
const {stream, filename} = await args.img
const img_path = await upload({stream, filename})
const post = await Post.save({
//img is a string in my mongo model
img: img_path,
author_name: args.user.username,
author_email: args.user.email
});
}
the upload method that should return the path and save the image to local:
const upload = ({ stream, filename }) => {
const id = shortid.generate()
const path = `${UPLOAD_DIR}/${filename}-${id}`
new Promise((resolve, reject) =>
stream
.pipe(fs.createWriteStream(filename))
.on("finish", () => resolve(path))
.on("error", reject(Error))
);
}
The error im getting is that stream and filename are undefined when calling upload() but args.img is an object if i log it. And uploading them to my local folder doesnt work neither. Any help is appreciated and marked as accepted answer

It would be nice to share your graphql Schema so that we can see the types you're returning. However, Here's how i have been handling file uploads in most of my apps.
graphql-schema
type File {
id: ID!
filename: String!
mimetype: String!
path: String!
}
mongoose schema
import { Schema, model } from "mongoose";
const fileSchema = new Schema({
filename: String,
mimetype: String,
path: String,
});
export default model("File", fileSchema);
Function to store uploads:
const storeUpload = async ({ stream, filename, mimetype }) => {
const id = shortid.generate();
const path = `images/${id}-${filename}`;
// (createWriteStream) writes our file to the images directory
return new Promise((resolve, reject) =>
stream
.pipe(createWriteStream(path))
.on("finish", () => resolve({ id, path, filename, mimetype }))
.on("error", reject)
);
};
To process the uploads
const processUpload = async (upload) => {
const { createReadStream, filename, mimetype } = await upload;
const stream = createReadStream();
const file = await storeUpload({ stream, filename, mimetype });
return file;
};
Mutation
export default {
Mutation: {
uploadFile: async (_, { file }) => {
mkdir("images", { recursive: true }, (err) => {
if (err) throw err;
});
const upload = await processUpload(file);
// save our file to the mongodb
await File.create(upload);
return upload;
},
},
};
Here you can find an article i wrote on how to handle file uploads

Upload Image to Cloud Storage from Cloud Function Trigger

I'm currently seeking some help with my Cloud Function that is triggered by a Cloud Storage Upload. It checks if the file is a Video, if so we process this Video through ffmpeg to extract a single frame to be used for a Poster Image later.
It all seems to work except my upload of the image back to Cloud Storage doesn't work. At this point where my Cloud Function is it doesn't produce any errors at all, so i have no clue why the upload of the image to Cloud Storage is not working. I would greatly appreciate if anyone with the experience can review my Cloud Function below and provide some insight into why it's not working. Please advice if possible!! Thank you!!!! ^_^
Note: Screenshot of Cloud Function Log is provided below the code snippet
const admin = require('firebase-admin'); // Firebase Admin SDK
const functions = require('firebase-functions'); // Firebase Cloud Functions
const gcs = require('#google-cloud/storage')(); // Cloud Storage Node.js Client
const path = require('path'); // Node.js file and directory utility
const os = require('os'); // Node.js operating system-related utility
const fs = require('fs'); // Node.js file system API
const ffmpeg = require('fluent-ffmpeg');
const ffmpegPath = require('#ffmpeg-installer/ffmpeg').path;
const ffprobePath = require('#ffprobe-installer/ffprobe').path;
// Initialize Firebase Admin
admin.initializeApp(functions.config().firebase);
// Listen for changes in Cloud Storage bucket
exports.storageFunction = functions.storage.object()
.onChange((event) => {
const file = event.data; // The Storage object.
const fileBucket = file.bucket; // The Storage bucket that contains the file.
const filePath = file.name; // File path in the bucket.
const fileName = path.basename(filePath); // Get the file name.
const fileType = file.contentType; // File content type.
if (!fileType.startsWith('video/')) {
return;
}
const bucket = gcs.bucket(fileBucket);
const tempFilePath = path.join(os.tmpdir(), fileName);
const tempFolderPath = os.tmpdir();
// Download video to temp directory
return bucket.file(filePath).download({
destination: tempFilePath
}).then(() => {
console.log('Video downloaded locally to', tempFilePath);
// Generate screenshot from video
ffmpeg(tempFilePath)
.setFfmpegPath(ffmpegPath)
.setFfprobePath(ffprobePath)
.on('filenames', (filenames) => {
console.log(`Will generate ${filenames}`);
})
.on('error', (err) => {
console.log(`An error occurred: ${err.message}`);
})
.on('end', () => {
console.log(`Output image created at ${tempFilePath}`);
const targetTempFileName = `${fileName}.png`;
const targetFilePath = path.join(path.dirname(filePath), targetTempFileName);
console.log(targetTempFileName);
console.log(targetFilePath);
// Uploading the image.
return bucket.upload(tempFilePath, { destination: targetFilePath })
.then(() => {
console.log('Output image uploaded to', filePath);
})
.catch((err) => {
console.log(err.message);
});
})
.screenshots({
count: 1,
folder: tempFolderPath
});
});
});
Cloud Function Log

It looks like you're trying to return a promise from the ffmpeg callback API:
.on('end', () => {
return bucket.upload(tempFilePath, { destination: targetFilePath })
.then(...)
})
I don't know the ffmpeg API, but I'm almost certain that will not cause the function to wait for the upload to complete. Instead, you need to return a promise from directly from your function that resolves only after all the async work is complete.
If the last item of work is inside a callback, and you need to wait for that, you can wrap the entire thing into a new promise and manually resolve it at the right time. In pseudocode:
return new Promise((resolve, reject) => {
// ffmpeg stuff here...
.on('end', () => {
// the last bit of work here...
bucket.upload(...)
.then(() => { resolve() })
})
})
Notice how the resolve method provided by the new promise is being called to indicate when that promise should itself resolve.