I'm trying to do gapless playback of segments generated using ffmpeg:
I use ffmpeg to encode 3 files from a source with exactly 240000 samples # 48kHz, i.e. 5 seconds.
ffmpeg -i tone.wav -af atrim=start_sample=24000*0:end_sample=240000*1 -c:a opus 0.webm
ffmpeg -i tone.wav -af atrim=start_sample=24000*1:end_sample=240000*2 -c:a opus 1.webm
ffmpeg -i tone.wav -af atrim=start_sample=24000*2:end_sample=240000*3 -c:a opus 2.webm
When looking at the meta data (using ffprobe and ffmpeg -loglevel debug) from the file I get the following which seems to me inconsistent values:
Duration: 5.01,
Start 0.007
discard 648/900 samples
240312 samples decoded
If I have several of these files how would I play them seamlessly without gaps?
i.e. in a browser I've tried:
sourceBuffer.timestampOffset = 5 * n - 648/48000;
sourceBuffer.appendWindowStart = 5 * n;
sourceBuffer.appendWindowEnd = 5 * (n+1);
sourceBuffer.appendBuffer(new Uint8Array(buffer[n]));
However, there are audible gaps.
How many samples am I actually supposed to discard? 0.007 * 48000, 648, or 240312 - 240000?
Here is a html page which can be opened in Chrome to test.
You need a simple http server to run it:
<< ls
>> index.html 0.webm 1.webm 2.webm
<< npm install -g http-server
<< http-server --cors
Related
I am trying to build a real-time streaming app using websockets and ffmpeg.
I have 2 processes on the host machine for capturing video(screen) and audio(microphone) every second. Those 2 files (1 file for a second of audio and 1 file for a second of video) are combined into one mp4 file using ffmpeg.
ffmpeg -y -i video.mkv -i audio.wav -map 0:v -map 1:a -c:v libx264 -tune zerolatency -preset ultrafast -shortest -movflags frag_keyframe+empty_moov+default_base_moof -r 25 -g 50 -pass 1 -f mp4 -b:v 2M pipe:1
This file is sent to the client machine using a websocket.
On the web part I initialize a MediaSource object with a sourceBuffer.
videoSourceBuffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.4D0033, mp4a.40.2"');
But I had a problem. Both audio and video were lagging behind. When I started the stream the delay was about 2 seconds. After 2 minutes the delay was 10 seconds.
I managed to see how many buffered frames I had in the sourceBuffer and speed up the stream to keep up with the source using this piece of code from How can I find remaing frames in a MediaSource SourceBuffer?.
let getBufferedLength = () => videoSourceBuffer.buffered.end(0) - videoElement.currentTime;
Now the video stream has always a delay about 1.5 seconds but the audio lags behind.
I cannot find a way to access how many audio frames are buffered or something so I can sync both the audio and video
I am struggling on a ffmpeg media conversion script.
I am using fluent-ffmpeg library with node.js.
My app is supposed to receive a stream as input, resize it using ffmpeg, and then outputing a stream.
However, I am absolutely unable to process an input stream with ffmpeg, even when specifying the input format (-f ffmpeg's option).
However, when execuyting the exact same ffmpeg command on an mp4 file (without extension), it works and converts properly the media !
Working code (no stream)
import * as ffmpeg from 'fluent-ffmpeg';
ffmpeg('myMp4File')
.inputFormat('mp4')
.audioCodec('aac')
.videoCodec('libx264')
.format('avi')
.size('960x540')
.save('mySmallAviFile');
Failing code (using stream)
import * as ffmpeg from 'fluent-ffmpeg';
import { createReadStream } from 'fs';
ffmpeg(createReadStream('myMp4File'))
.inputFormat('mp4')
.audioCodec('aac')
.videoCodec('libx264')
.format('avi')
.size('960x540')
.save('mySmallAviFile');
It generates the following ffmpeg's error:
Error: ffmpeg exited with code 1: pipe:0: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
Conversion failed!
This error explicitely says that ffmpeg could not identify the input format, in despite of argument -f mp4.
I read pages and pages of ffmpeg's man but I could find any relevant information concerining my issue.
Complementary information
Here is the output of command._getArguments(), showing the full ffmpeg command baked by the library:
[
'-f', 'mp4',
'-i', 'pipe:0',
'-y', '-acodec',
'aac', '-vcodec',
'libx264', '-filter:v',
'scale=w=960:h=540', '-f',
'avi', 'mySmallAviFile'
]
So the full ffmpeg command is the following:
ffmpeg -f mp4 -i pipe:0 -y -acodec 'aac' -vcodec 'libx264 -filter:v 'scale=w=960:h=540' -f 'avi' mySmallAviFile
I was getting the same error but only for files which had moov atom metadata at the end of a file.
After moving mov atom to the beginning of the file with:
ffmpeg -i input.mp4 -movflags faststart out.mp4
the error dissapeared.
I am creating a streaming app like youtube while I am creating it I am facing many challenges related to different quality video converting.
My question is
Should I convert orginal video file into multiple video file (like 240p, 480p and 720p) and storage them? Or there is anyway where I can create a single video file which can be play in multiple qualities like youtube.
Multiple video files is the way to go. Currently, the most common approach to adaptive streaming is MPEG-DASH. The different video sizes and a MPD manifest, which is like a playlist for the different video sizes, can be generated using ffmpeg and mp4box. Many videoplayers, e.g. Video.js or Dash.js support adaptive streaming with MPEG-DASH.
Generate video files:
ffmpeg -y -i movie.avi -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 1500k -maxrate 1500k -bufsize 3000k -vf "scale=-1:720" movie-720.mp4
ffmpeg -y -i movie.avi -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 800k -maxrate 800k -bufsize 1600k -vf "scale=-1:540" movie-540.mp4
ffmpeg -y -i movie.avi -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 400k -maxrate 400k -bufsize 800k -vf "scale=-1:360" movie-360.mp4
ffmpeg -y -i movie.avi -vn -c:a aac -b:a 128k movie.m4a
Generate the manifest:
mp4box -dash-strict 2000 -rap -frag-rap -bs-switching no -profile "dashavc264:live" -out movie-dash.mpd movie-720.mp4 movie-540.mp4 movie-360.mp4 movie.m4a
Original source: https://gist.github.com/andriika/8da427632cf6027a3e0036415cce5f54
I have an audio file with extension .raw and recorded in stereo, and I need to convert it to mono with Node. I'm not able to find a example of how to do it this process or a library.
Any help would be great
You can use the following library
https://github.com/fluent-ffmpeg/node-fluent-ffmpeg to use ffmpeg on node.
And following to convert to stereo to mono via ffmpeg
ffmpeg -i stereo.flac -ac 1 mono.flac
Just pass the above options to ffmpeg via the library.
Here's my code that seems to work
var FFmpeg = require('fluent-ffmpeg');
var command = FFmpeg({
source: 'test.webm'
}).addOption('-ac', 1)
.saveToFile('out.mp3');
Let me Explain by my Code what issue i am facing...
This is my js file for using with PhantomJS. It simple tell it to open a page and take screenshots of it and store them in stdout.
var page = require("webpage").create();
page.viewportSize = { width: 640, height: 480 };
page.open("http://www.goodboydigital.com/pixijs/examples/12-2/", function() {
setInterval(function() {
page.render("/dev/stdout", { format: "png" });
}, 25);
});
And this is the cmd command I'm running to receive the captured images in ffmpeg in Windows Command Prompt.
phantomjs runner.js | ffmpeg -y -c:v png -f image2pipe -r 25 -t 10 -i - -c:v libx264 -pix_fmt yuv420p -movflags +faststart dragon.mp4
This command successfully starts the processes of PhantomJS and ffmpeg. But nothing happens for quite some time, after 15 minutes it gives an error saying:
"Failed to reallocate parser buffer"
thats it. I have referenced this code from this site on which the developer claims that it works
https://mindthecode.com/recording-a-website-with-phantomjs-and-ffmpeg/
Please see the attached Image for more explanation.
Image of Code
It could be related to stdout the ffmpeg process as it is being stdin through the pipe and after taking continuous image buffer is filled up and gives error.
You can review this from a well organized canvas recording application "puppeteer-recorder" on nodeJS
https://github.com/clipisode/puppeteer-recorder