No video pad available after manual caps set on audio pad (FlvDemux)

[Bilel Bghiel]

Hi

I'm current working on a project to stream from multiple IP camera's and
send the stream to a Twitch channel while also simultaneously saving to
files. The files are combined with an audiomixer and a wav parser to add
background music later.

Current I'm dynamically attaching video and audio from a flvdemuxer, which
doesn't work since the negotiation fails. Note here that both the audio AND
video pads become available for linking.

Data comes in as:

audio/x-alaw

However the audiomixer wants S16LE or U8 data, so I change the format on
the downstream pad using the following callback:
---------------------------------------------------------------------------------------------------------------
GstPad *sinkpad;

sinkpad = gst_element_get_request_pad(writer->audiomixer, "sink_%u");

GstCaps* newCaps = gst_caps_new_simple ("audio/x-raw",
"format", G_TYPE_STRING, "S16LE",
"layout", G_TYPE_STRING, "interleaved",
"channels", G_TYPE_INT, 1,
NULL);


if (!gst_pad_set_caps (pad, newCaps)){
    std::cout << "Couldnt set caps " << std::endl;
} else {
    std::cout << "Caps set \n";
}

std::cout << "Link mux audio to mixer : " << gst_pad_link(pad, sinkpad) <<
std::endl;
-----------------------------------------------------------------------------------------------------------------
This enabled the link to work, however the video pad never becomes
available when I do this.

Debugging shows me the following:

Caps set
Link mux audio to mixer : 0:00:03.987475878 24051 0x556d42969aa0 INFO
         GST_PADS gstpad.c:2382:gst_pad_link_prepare: trying to link
flvdemux17171:audio and audiomixer17171:sink_1
0:00:03.987613945 24051 0x556d42969aa0 INFO         audioaggregator
gstaudioaggregator.c:933:gst_audio_aggregator_convert_sink_getcaps:<audiomixer17171:sink_1>
Getting caps with filter (NULL)
0:00:03.987764127 24051 0x556d42969aa0 INFO         audioaggregator
gstaudioaggregator.c:998:gst_audio_aggregator_convert_sink_getcaps:<audiomixer17171:sink_1>
returned sink caps : audio/x-raw, format=(string){ F64LE, F64BE, F32LE,
F32BE, S32LE, S32BE, U32LE, U32BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE,
S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE,
U18LE, U18BE, S16LE, S16BE, U16LE, U16BE, S8, U8 }, rate=(int){ 5512,
11025, 22050, 44100 }, channels=(int)[ 1, 2147483647 ],
layout=(string)interleaved
0:00:03.987872341 24051 0x556d42969aa0 INFO                GST_PADS
gstpad.c:2590:gst_pad_link_full: linked flvdemux17171:audio and
audiomixer17171:sink_1, successful
0:00:03.987956251 24051 0x556d42969aa0 INFO               GST_EVENT
gstevent.c:1660:gst_event_new_reconfigure: creating reconfigure event
0
0:00:03.988203187 24051 0x556d42969aa0 INFO               GST_EVENT
gstevent.c:972:gst_event_new_segment: creating segment event time segment
start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999,
rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000,
base=0:00:00.000000000, position 0:00:00.000000000, duration
99:99:99.999999999
0:00:03.988368401 24051 0x556d42969aa0 INFO               GST_EVENT
gstpad.c:5946:gst_pad_send_event_unchecked:<audiomixer17171:sink_1>
Received event on flushing pad. Discarding
0:00:03.988462123 24051 0x556d42969aa0 INFO               GST_EVENT
gstpad.c:5946:gst_pad_send_event_unchecked:<audiomixer17171:sink_1>
Received event on flushing pad. Discarding
0:00:03.988572641 24051 0x556d42969aa0 FIXME             aggregator
gstaggregator.c:1394:gst_aggregator_aggregate_func:<flvmux0> Subclass
should call gst_aggregator_selected_samples() from its aggregate
implementation.

Does anybody have a clue why the audio pad never becomes available when I
set the audio caps manually?

Kind regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220516/14dc4fa4/attachment.htm>