gstreamer problem (br Pio)

[Nicolas Dufresne]

Le mercredi 22 avril 2020 à 16:37 +0200, Saint Pierre a écrit :
> Hi!
>  
> I hope you are not bothered by the virus. I am br Pio. I come from France and discovered gstreamer a few months ago. I use it to stream multiple video sources on YoutubeLive. May I say you guys do a great job!
>  
> I have few problems I really can’t get through.
>  
> I have two live video sources:
>  
> -an axis ip camera, which gives me a rtsp flow
> -a videocamera flow going through a hdmi to usb adaptater, and streamed by vlc, which gives me a rtsp flow.
>  
> I use voctomix to mix the 2 video sources with the audio input of a sound card.
>  
> Here are the pipelines for the video sources:
>  
> gst-launch-1.0 rtspsrc location=$URLCAM latency=2000 ! decodebin name=decoder max-size-time=100000000 use-buffering=true low-percent=1 \
>        decoder. ! queue min-threshold-time=$DELAY ! videoscale ! videorate ! videoconvert  ! video/x-raw,format=UYVY,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE/1,pixel-aspect-ratio=1/1 ! queue ! matroskamux name=mux !\
>         tcpclientsink host=localhost port=$PORT \
>        audiotestsrc freq=550 !\
>         audio/x-raw,format=S16LE,channels=2,layout=interleaved,rate=48000 ! mux.
>  
> And for the audio source:
>  
> ffmpeg -y -nostdin \
>     -itsoffset $retard \
>      -loop 1 -i "/home/mixer/voctomix/lanceurs/sound.jpg" \
>      -f pulse -i default  \
>     -ac 2 \
>     -vcodec rawvideo -r "$FRAMERATE" \
>     -vf "scale=1280:-1,format=uyvy422" \
>     -acodec pcm_s16le -ar "$AUDIORATE" -af $channel \
>     -map 1:a:0  -map 0:v:0 \
>     -f matroska \
>     tcp://localhost:10003
>  
> My first problem is:
>  
> - The pipeline of the videocamera randomly renders a partially grey image. I have the same problem with some other ip cameras. The problem does not seem to come from the network gear, since the ip camera and the videocamera are using the same “way”.
> The problem also seem to decrease with the bitrate: the lower the BR is, the lower the grey image occurs.
> It also decreases when the preset used to encode the rtsp stream is slower: preset = medium, greater chance to get grey than with preset =slow. But when I use preset=slow, the pipeline output randomly goes faster of slower: the video is slow then suddenly accelerate, and then gets slow again etc.
>  
> Here’s what it looks like.

Usually that happens if you have packet lost, or late packets too. The
last few packet are dropped, and you look few slices of your video.
You'll get this when the sended isn't smoothly real-time, or is bursty.

This can be solved by finding a encoding profile on your sender that is
truly real-time, or sometimes by increasing the buffering (the latency)
on the sender side.

On the receiver side, the only way to workaround would be to move to
rtspt:// (TCP streaming) and use a high latency.

>  
> 
>  
>  
> My second problem is :
>  
> The audio input is randomly delayed. The offset between video and audio is random. One day it is perfectly synchronized, the other, there may be up to 500-1000ms offset!

I think some clarification on the final receiver is needed, I see two
tcp matroska stream being sent. To gain synchronization, you have to
mux the audio and the video together.

>  
> In advance, thanks for taking time to answer me. I hope to hear from you soon.
>  
> Best regards.
>  
> br Pio
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel