Can't decode frames for H264 stream encoded using NVENC over RTP

[Nicolas Dufresne]

Le mer. 21 oct. 2020 01 h 15, Moiz <moiz at playpixel.app> a écrit :

> I am testing out a sender and receiver pipeline using nvh264enc. I am able
> to
> get this pipeline to work using x264enc however when I switch to nvh264,
> although the snder pipeline works in that it starts a clock, I am not
> getting any frames nor is the direct3d11 render window showing up when I
> run
> the receiver pipeline.
>
> I know nvh264enc is working because I am able to encode and save to an mp4
> file, however when I switch to udpsink, it doesn't decode frames.
>
> Here is the pipeline I used to send/receive h264 stream encoded using
> x264enc over udp
>
> Sender: .\gst-launch-1.0.exe -v dxgiscreencapsrc ! queue !
> video/x-raw,framerate=20/1 ! videoconvert ! x264enc ! rtph264pay ! udpsink
> host=127.0.0.1 port=5000
> Receiver: .\gst-launch-1.0.exe -v udpsrc port=5000 caps =
> "application/x-rtp, media=(string)video, clock-rate=(int)90000,
> encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin !
> videoconvert ! autovideosink
>
> When I use this same pipeline switching x264enc to nvh264 enc, the sender
> works but the receiver is not decoding frames nor is it opening the window,
> so I tried playing around with my rtph264pay and nvh264enc caps however
> still same result, this was the pipelines I tried
>
> Sender: .\gst-launch-1.0.exe -v dxgiscreencapsrc ! queue !
> video/x-raw,framerate=20/1 ! videoconvert ! nvh264enc !
> "video/x-h264,profile=(string)high" ! rtph264pay !
> "application/x-rtp,payload=(int)123" ! udpsink host=127.0.0.1 port=5000
>

A wild guess, but try adding h264parse. You should also check or force the
video format, to avoid colorspace that might not be supported widely (like
yuv 444). Another item to check is your key frame distance.



> Receiver: .\gst-launch-1.0.exe -v udpsrc port=5000 caps =
> "application/x-rtp, media=(string)video, clock-rate=(int)90000,
> encoding-name=(string)H264, payload=(int)123" ! rtph264depay ! decodebin !
> videoconvert ! autovideosink
>
> To ensure nvh264enc worked, I tried saving a screencap as mp4 and it worked
> successfully and I was able to play the mp4 file using this pipeline
>
> .\gst-launch-1.0.exe -e dxgiscreencapsrc width=1280 height=650 y=32 ! queue
> ! videorate drop-only=true ! video/x-raw,framerate=60/1 ! videoconvert !
> video/x-raw,format=I420,framerate=60/1 ! nvh264enc ! h264parse ! mp4mux !
> filesink location=nvh264.mp4
>
>
> I am assuming it has something to do with the packetization happening
> through rtph264pay or something on my receiver, but I am not able to figure
> out what it is as I am not very familiar with conversion to packets and
> sending packets over udp
>
> Any guidance/direction would be appreciated greatly
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201021/7f88c24a/attachment-0001.htm>