Recover h264parsed h264 stream dumped to disk

[Nicolas Dufresne]

Hi,

Le dimanche 02 juillet 2023 à 23:40 -0400, Judemir Ribeiro via gstreamer-devel a
écrit :
> Hello,
> 
> I created a few videos using a wrong pipeline and now I dont know how to read
> the streams to recover them.
> 
> The pipeline was:
> 
> libcamera-vid -t 0 -n -o - | gst-launch1.0 fdsrc fd=0 ! h264parse config-
> interval=-1 ! tee name=output .output ! queue ! rtph264pay ! udpsink
> host=127.0.0.1 port=5600 .output ! queue ! filesink
> location=/video/$(date).h264 
> As we can see I created the tee after the h264parse step, when it should have
> been before. 
> 
> How can I convert these files back to h264 streams? I have tried ffmpeg, and
> other programs but nothing can recognize the files dumped by this wrong
> pipeline.

Looking at rtph264pay, AVC is placed before Byte Stream (the usual file format).
In short, I suspect that the file is made of AVCc NALs (size of nal header
instead of startcodes/0x000001). I simulated that, and its a bit hard to
recover. Some work on h264parse would be needed to make this work inside gst-
launch-1.0. Please find attached a python script to help with your task. Please
follow the procedure:

1. Find the original codec_data

For this, rerun the "libcamera-vid -t 0 -n -o - | gst-launch1.0 fdsrc..."
pipeline but add the -v option. Look for 
"codec_data=(buffer)<somebighex>" and same that. This is the stream headers,
they will always be the same for the same source/sensor/pipeline, but they are
not being saved in your erroneous pipeline.

2. Run the recovery script provided, a file called fixed.h264 will be created

python3 recover.py <avc_filename> <codec_data>

In my test, I produced a bad file this way:

gst-launch-1.0 videotestsrc num-buffers=1000 ! video/x-raw,format=I420 ! x264enc tune=zerolatency ! h264parse ! tee name=t t. ! queue ! rtph264pay ! fakesink t. ! queue ! filesink location=test.avc -v
python3 recover.py test.avc 01640014ffe1001d67640014acb20283f602d418181a94000003000400000300f23c50a92001000568ebccb22c

regards,
Nicolas

p.s. for those who want to know, but don't want to open the script, the recovery
pipeline is:

f"appsrc name=src block=true caps=\"video/x-h264,stream-
format=(string)avc,codec_data=(buffer){codec_data}\"" +
 " ! h264parse ! video/x-h264,stream-format=(string)byte-stream ! filesink
location=fixed.h264"


And the script just push the data using python API into appsrc. The difference
with that and filesrc, is that appsrc will push a caps event, which properly
signals AVCc, codec_data and no alignment to h264parse. If anyone have
suggestion on how to fix this, one way could be to introduce a caps properly to
filesrc, like udpsrc has.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230703/10a062f3/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: recover.py
Type: text/x-python3
Size: 1766 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230703/10a062f3/attachment.bin>