Recording and displaying a live video feed and pausing the recording during silence
Cameron Blomquist
cameron at blomqu.ist
Tue Apr 26 12:52:51 UTC 2022
For context, I've been tasked with digitizing too many tapes.
I'm using gstreamer-rs to record the video and audio input from a UVC
capture device. I'm also displaying the video stream so that I can
monitor it for whatever issues might arise. When there are gaps in the
video, I want to be able to pause recording while still displaying the
stream. My pipeline looks something like this:
"splitmuxsink name=sink location=video%04d.mp4 async-finalize=true
mfvideosrc name=vsrc ! tee name=t
t. ! queue ! rsvgoverlay name=overlay ! queue ! autovideosink
t. ! queue ! valve name=valve ! videoconvert ! x264enc key-int-max=10
tune=zerolatency ! queue ! sink.video
wasapisrc ! queue ! audioconvert ! removesilence remove=true
silent=false ! audioconvert ! audioresample ! opusenc ! queue !
sink.audio_0"
I monitor the pipeline bus for messages from removesilence to split the
video. On silence_detected, I emit split-after to the splitmuxsink and
open the valve and update rsvgoverlay. On silence_finished, I close the
valve and update rsvgoverlay to show that we are once again recording. I
use a ctrlc handler to send EOS to the pipeline.
You can probably see where this is going. When I open the valve, the
entire pipeline freezes. I can't even quit the program with Ctrl+C since
the EOS is dropped. If I don't do something to prevent frames from
pushing into the splitmuxsink though, I'll end up with long periods of
blank screen in the video files.
I also tried using output-selector into the splitmuxsink and a fakesink,
doing something along the lines of
for pad in vselect.iterate_pads() {
let pad = pad.unwrap();
let elem = pad.peer().unwrap().parent_element().unwrap();
if elem.name() == "sink" {
vselect.set_property("active-pad", pad);
}
}
on the removesilence messages to swap the pads. Interestingly enough,
this also immediately freezes the pipeline in the same way that valve
does and it also appears to drop the EOS event so that the pipeline runs
forever. It might actually just be hanging somewhere since I don't
receive clock messages either and I would expect to see NewClock when
starting the pipeline.
Email threads from ten years ago describing similar issues with valves
didn't provide much help that was relevant to how my pipeline is set up,
so I am unsure of how to properly tackle this problem. The only
recommendation I remember was to manually swap out pads, but I would
have expected output-selector to be equivalent.
And this doesn't even touch on the potential issue of timestamps between
those separate video files. Please tell me that splitmuxsink will handle
that automagically.
Sorry to ramble, I tend to do that. How should I approach pausing the
splitmuxsink to avoid recording dead air?
More information about the gstreamer-devel
mailing list