Python access to video frames (Carl Karsten)
Marianna Smidth Buschle
msb at qtec.com
Wed Oct 18 05:37:19 UTC 2023
Again, you need to start simplifying the pipeline in order to easier
figure out what is going on.
Either you start with just a src and a sink and grow from there as I
showed or you start with the whole pipeline and remove elements/branches
one by one...
A good start is just to divide the src and decode from the encode and
save. Or just start by discarding the audio...
Also, add --gst-debug=*:3 to the pipeline to get some debug output.
gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts --gst-debug=*:3
gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! fakesink --gst-debug=*:3
gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! videoconvert ! autovideosink sync=0 --gst-debug=*:3
And if you start suspecting some of the elements you can do:
--gst-debug=*:3,elem:5 to get even more debug
On 17/10/2023 19.32, Carl Karsten wrote:
> no error, but no play.
>
> gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
> '!' queue '!' decodebin3 '!' avenc_mpeg2video '!' mpegtsmux name=mux1
> '!' filesink location=fixed_13_25_49.ts demux1. '!' queue '!'
> mpegaudioparse '!' mux1.
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
>
> htop shows no cpu and ls -l shows no bytes.
>
> On Tue, Oct 17, 2023 at 4:19 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
>> The demuxer divides the pipeline into audio and video, you get one in
>> each branch.
>>
>> Right now you seem to have both an audioparser and a videoencoder in the
>> same branch.
>>
>> Try instead:
>>
>> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
>> queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts
>> demux1. ! queue ! mpegaudioparse ! mux1.
>>
>>
>> And generally, when in doubt about pipeline errors it is always useful
>> to simplify it and then build up.
>> Fx: filesrc ! fakesink
>> then filesrc ! demux ! fakesink
>> then filesrc ! demux ! decode ! fakesink
>> then filesrc ! demux ! decode ! videosink
>> ...
>>
>> Best Regards
>> Marianna
>>
>> On 16/10/2023 20.00, Carl Karsten wrote:
>>> Ima need a little help with the pipeline...
>>>
>>> # probably needs an aacparse or parser for whatever the codec of the
>>> audio stream is
>>> # also you need queues after both branches of the demuxer
>>>
>>> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
>>> queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux
>>> name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue !
>>> mux1.
>>>
>>>
>>> ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
>>> '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!'
>>> mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1.
>>> '!' queue '!' mux1.
>>> Setting pipeline to PAUSED ...
>>> Pipeline is PREROLLING ...
>>> ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal
>>> data stream error.
>>> Additional debug info:
>>> ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop ():
>>> /GstPipeline:pipeline0/GstTSDemux:demux1:
>>> streaming stopped, reason not-negotiated (-4)
>>> ERROR: pipeline doesn't want to preroll.
>>> Setting pipeline to NULL ...
>>> ERROR: from element
>>> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
>>> Stream doesn't contain enough data.
>>> Additional debug info:
>>> ../plugins/elements/gsttypefindelement.c(1012):
>>> gst_type_find_element_chain_do_typefinding ():
>>> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
>>> Can't typefind stream
>>> ERROR: pipeline doesn't want to preroll.
>>> Freeing pipeline ...
>>>
>>> 13_25_49.ts came from:
>>>
>>> ffmpeg \
>>> -nostdin -y \
>>> -analyzeduration 10000 \
>>> -thread_queue_size 512 \
>>> -i tcp://localhost:11000?timeout=3000000 \
>>> -aspect 16:9 \
>>> -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4
>>> -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5
>>> -me_method:v:0 dia \
>>> -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \
>>> -flags +global_header \
>>> -f segment -segment_time $segment_time \
>>> -segment_format mpegts \
>>> -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts"
>>>
>>> https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54
>>>
>>> and in case more info helps:
>>>
>>> $ mediainfo 13_25_49.ts
>>> General
>>> ID : 1 (0x1)
>>> Complete name : 13_25_49.ts
>>> Format : MPEG-TS
>>> Commercial name : HDV 720p
>>> File size : 838 MiB
>>> Duration : 30 min 0 s
>>> Overall bit rate mode : Variable
>>> Overall bit rate : 3 903 kb/s
>>> Frame rate : 30.000 FPS
>>>
>>> Video
>>> ID : 256 (0x100)
>>> Menu ID : 1 (0x1)
>>> Format : MPEG Video
>>> Commercial name : HDV 720p
>>> Format version : Version 2
>>> Format profile : Main at High 1440
>>> Format settings, BVOP : No
>>> Format settings, Matrix : Default
>>> Format settings, GOP : N=5
>>> Codec ID : 2
>>> Duration : 30 min 0 s
>>> Bit rate mode : Variable
>>> Bit rate : 3 323 kb/s
>>> Width : 1 280 pixels
>>> Height : 720 pixels
>>> Display aspect ratio : 16:9
>>> Frame rate : 30.000 FPS
>>> Color space : YUV
>>> Chroma subsampling : 4:2:0
>>> Bit depth : 8 bits
>>> Scan type : Progressive
>>> Compression mode : Lossy
>>> Bits/(Pixel*Frame) : 0.120
>>> Time code of first frame : 01:29:59:29
>>> Time code source : Group of pictures header
>>> GOP, Open/Closed : Open
>>> Stream size : 713 MiB (85%)
>>> Color primaries : BT.709
>>> Transfer characteristics : BT.709
>>> Matrix coefficients : BT.709
>>>
>>> Audio
>>> ID : 257 (0x101)
>>> Menu ID : 1 (0x1)
>>> Format : MPEG Audio
>>> Format version : Version 1
>>> Format profile : Layer 2
>>> Codec ID : 3
>>> Duration : 30 min 0 s
>>> Bit rate mode : Constant
>>> Bit rate : 384 kb/s
>>> Channel(s) : 2 channels
>>> Sampling rate : 48.0 kHz
>>> Frame rate : 41.667 FPS (1152 SPF)
>>> Compression mode : Lossy
>>> Delay relative to video : -29 ms
>>> Stream size : 82.4 MiB (10%)
>>> Language : English
>>>
>>> Menu
>>> ID : 4096 (0x1000)
>>> Menu ID : 1 (0x1)
>>> Format : MPEG Video / MPEG Audio
>>> Duration : 30 min 0 s
>>> List : 256 (0x100) (MPEG Video) /
>>> 257 (0x101) (MPEG Audio, English)
>>> Language : / English
>>> Service name : Service01
>>> Service provider : FFmpeg
>>> Service type : digital television
>>>
>>>
>>> On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
>>>> As suggested you can go some different ways:
>>>>
>>>>
>>>> - Make a python gst app:
>>>>
>>>> filesrc ! decodebin ! appsink
>>>>
>>>> opencv code
>>>>
>>>> appsrc ! encode ! filesink
>>>>
>>>> Passing the audio as well might be tricky...
>>>>
>>>>
>>>> - Make a python gst app:
>>>>
>>>> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1
>>>> ! filesink demux1. ! mux1
>>>>
>>>> and use a probe to snatch frames at the identify element and do what you
>>>> want there with opencv
>>>>
>>>>
>>>> - Make a filter element to do your stuff
>>>>
>>>> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 !
>>>> filesink demux1. ! mux1
>>>>
>>>>
>>>> Best Regards
>>>>
>>>> Marianna
>>>>
>>>>
>>>> On 13/10/2023 14.00, gstreamer-devel-request at lists.freedesktop.org wrote:
>>>>> I need to alter video frames, and Python is my happy place.
>>>>> my gst skills need some help.
>>>>>
>>>>> I think I need these 2 things:
>>>>>
>>>>> 1. pipeline:
>>>>> read a file,
>>>>> decode the video,
>>>>> hook? to read/write pixels,
>>>>> encode (keep the original audio)
>>>>> save to disk.
>>>>>
>>>>> 2. that hook code...
>>>>> I have done something before: use the wavelevel element to get audio
>>>>> data and generate static png images.
>>>>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72
>>>>>
>>>>> So I am assuming some element / message will give me a frame buffer
>>>>> thing of data, and somehow i can detect black pixels.
>>>>>
>>>>>
>>>>> End goal: fix some video;s I made of a live event where my setup
>>>>> introduced black squares - I want to replace the black with the pixels
>>>>> from the previous frame.
>>>>> example of what needs fixing:
>>>>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148
>>>>>
>>>>> The slide is mostly a static image, so the previous frame will be fine.
>>>>>
>>>>> -- Carl K
>>>> --
>>>> Best regards / Med venlig hilsen
>>>> “Marianna Smidth Buschle”
>>>>
>> --
>> Best regards / Med venlig hilsen
>> “Marianna Smidth Buschle”
>>
>
--
Best regards / Med venlig hilsen
“Marianna Smidth Buschle”
More information about the gstreamer-devel
mailing list