Frames are not output correctly

[Bilel Bghiel]

Hello

Thanks for the help. I've been stuck on some other issues so haven't had a
lot of time to work on this. But I continued today and did a few things:

- I rewrote Marianna's code for my project as so:

static GstPadProbeReturn buffer_probe(GstPad * pad, GstPadProbeInfo * info,
gpointer pipe)
{
  using namespace std::literals;
  Pipeline* pipeline = (Pipeline*) pipe;

  now = std::chrono::steady_clock::now();

  if (GST_PAD_PROBE_INFO_TYPE(info) & GST_PAD_PROBE_TYPE_BUFFER) {
    GstBuffer* buffer = gst_pad_probe_info_get_buffer(info);
    if (!GST_BUFFER_FLAG_IS_SET(buffer, GST_BUFFER_FLAG_DELTA_UNIT)){
      const std::chrono::time_point<std::chrono::system_clock> nowLoc =
          std::chrono::system_clock::now();

      const std::time_t t_c = std::chrono::system_clock::to_time_t(nowLoc);
      std::cout << "Pad name: " << gst_pad_get_name(pad) << std::endl;
      std::cout << "time: "  << std::put_time(std::localtime(&t_c), "%F
%T.\n") << std::flush;

      if (doTheSwitch){
        doTheSwitch = false;

        int camNum = counter % pipeline->config->cameras.size();

        std::cout << "Switch cam to num: " << camNum << std::endl;

        for (VideoWriter* writer: pipeline->writers)
          writer->textOverlay->setText("Switching cam to: " +
std::to_string(camNum));

        //pipeline->setCameraSource(camNum);
        lastMeasuredTime = now;
        counter++;
        nextCam = false;

        // TODO: Implement this in every camera.
        //  Use cam number to doTheSwitch (make -1 disabled)
        //  Wait for appropriate cam to have keyframe ready
        //   Then drop probe and re-add element and probe
        return GST_PAD_PROBE_OK;
      }

    }

  }

  return GST_PAD_PROBE_OK;
}

Doing so I found out I only had 1 keyframe per 10 seconds. Problem were the
cameras where I set I-frame interval 1 to. Which I thought would be every
frame, so 25/seconds. But it seems it can only do 1/sec minimum.
I used ffprobe to verify this. I also had to switch from encoding
h264->h265 to even get this working, since the settings seem to be a bit
buggy.

If I'm right I should now implement the following:
        - Implement this probe in every camera.
        - Use cam number to doTheSwitch (make -1 disabled)
        - Wait for appropriate cam to have keyframe ready
        - Then drop probe and re-add element and probe

Will keep y'all posted.

Kind regards,
Bilel Bghiel.


On Thu, Jun 9, 2022 at 3:50 PM Marc Leeman <marc.leeman at gmail.com> wrote:

> Oh forgot,
>
> you can inspect the buffers and see if there is *GST_BUFFER_FLAG_DELTA_UNIT
> *to get an idea if it is I-Frame/SPS/PPS: these have not.
>
> On Tue, 7 Jun 2022 at 09:40, Marc Leeman <marc.leeman at gmail.com> wrote:
>
>>
>> I am doing something similar for ages (switching in the compressed
>> domain), based on an I-Frame (and SPS/PPS).
>>
>> When using different cameras, you need to make certain that you have the
>> new SPS/PPS before switching based on an I-Frame. If you have 'the same'
>> cameras, make certain that  your configuration is the same so they generate
>> the same (or compatible) SPS/PPS (though that is not a fault tolerant
>> implementation).
>>
>> After switching, you need to call
>>
>> gst_bin_recalculate_latency(GST_BIN(self));
>>
>> It should result a feed that is free of hiccups
>>
>> .From what I can tell, your SPS/PPS corrupts your video until you get the
>> new one in the video feed.
>>
>>
>> On Fri, 3 Jun 2022 at 18:45, Bilel Bghiel via gstreamer-devel <
>> gstreamer-devel at lists.freedesktop.org> wrote:
>>
>>> Hello
>>>
>>> I'm currently using the following setup:
>>>
>>> 9 IP camera's:
>>> RTSPsrc - rtph264depay - h264parse - tee
>>>
>>> VideoWriter
>>> Input-selector - tee - avdec264 - <video editing stuff > - videorate -
>>> avenc_flv - flvmux - RTMPsink OR filesink
>>>
>>> I have 9 camera's which are all connected to the videowriter
>>> input-selector. This works fine. Now when switching the camera's the screen
>>> seems to freeze, only changing the pixels where they are different.
>>>
>>> Demo:  https://www.youtube.com/watch?v=GJU1eRMgKt0
>>> (e.g: 0:26, 0:35)
>>>
>>> I've been playing around with I-frame settings on the camera's. Which I
>>> now set to 1. But still I get the results as seen in the video. Has anybody
>>> ever seen this before? And know how to fix this?
>>>
>>> Thanks in advance
>>>
>>
>>
>> --
>> g. Marc
>>
>
>
> --
> g. Marc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220610/a97bb9c5/attachment.htm>