Python access to video frames (Carl Karsten)
Marianna Smidth Buschle
msb at qtec.com
Tue Oct 17 09:19:21 UTC 2023
The demuxer divides the pipeline into audio and video, you get one in
each branch.
Right now you seem to have both an audioparser and a videoencoder in the
same branch.
Try instead:
gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts
demux1. ! queue ! mpegaudioparse ! mux1.
And generally, when in doubt about pipeline errors it is always useful
to simplify it and then build up.
Fx: filesrc ! fakesink
then filesrc ! demux ! fakesink
then filesrc ! demux ! decode ! fakesink
then filesrc ! demux ! decode ! videosink
...
Best Regards
Marianna
On 16/10/2023 20.00, Carl Karsten wrote:
> Ima need a little help with the pipeline...
>
> # probably needs an aacparse or parser for whatever the codec of the
> audio stream is
> # also you need queues after both branches of the demuxer
>
> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
> queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux
> name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue !
> mux1.
>
>
> ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
> '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!'
> mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1.
> '!' queue '!' mux1.
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal
> data stream error.
> Additional debug info:
> ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop ():
> /GstPipeline:pipeline0/GstTSDemux:demux1:
> streaming stopped, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
> Setting pipeline to NULL ...
> ERROR: from element
> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
> Stream doesn't contain enough data.
> Additional debug info:
> ../plugins/elements/gsttypefindelement.c(1012):
> gst_type_find_element_chain_do_typefinding ():
> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
> Can't typefind stream
> ERROR: pipeline doesn't want to preroll.
> Freeing pipeline ...
>
> 13_25_49.ts came from:
>
> ffmpeg \
> -nostdin -y \
> -analyzeduration 10000 \
> -thread_queue_size 512 \
> -i tcp://localhost:11000?timeout=3000000 \
> -aspect 16:9 \
> -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4
> -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5
> -me_method:v:0 dia \
> -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \
> -flags +global_header \
> -f segment -segment_time $segment_time \
> -segment_format mpegts \
> -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts"
>
> https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54
>
> and in case more info helps:
>
> $ mediainfo 13_25_49.ts
> General
> ID : 1 (0x1)
> Complete name : 13_25_49.ts
> Format : MPEG-TS
> Commercial name : HDV 720p
> File size : 838 MiB
> Duration : 30 min 0 s
> Overall bit rate mode : Variable
> Overall bit rate : 3 903 kb/s
> Frame rate : 30.000 FPS
>
> Video
> ID : 256 (0x100)
> Menu ID : 1 (0x1)
> Format : MPEG Video
> Commercial name : HDV 720p
> Format version : Version 2
> Format profile : Main at High 1440
> Format settings, BVOP : No
> Format settings, Matrix : Default
> Format settings, GOP : N=5
> Codec ID : 2
> Duration : 30 min 0 s
> Bit rate mode : Variable
> Bit rate : 3 323 kb/s
> Width : 1 280 pixels
> Height : 720 pixels
> Display aspect ratio : 16:9
> Frame rate : 30.000 FPS
> Color space : YUV
> Chroma subsampling : 4:2:0
> Bit depth : 8 bits
> Scan type : Progressive
> Compression mode : Lossy
> Bits/(Pixel*Frame) : 0.120
> Time code of first frame : 01:29:59:29
> Time code source : Group of pictures header
> GOP, Open/Closed : Open
> Stream size : 713 MiB (85%)
> Color primaries : BT.709
> Transfer characteristics : BT.709
> Matrix coefficients : BT.709
>
> Audio
> ID : 257 (0x101)
> Menu ID : 1 (0x1)
> Format : MPEG Audio
> Format version : Version 1
> Format profile : Layer 2
> Codec ID : 3
> Duration : 30 min 0 s
> Bit rate mode : Constant
> Bit rate : 384 kb/s
> Channel(s) : 2 channels
> Sampling rate : 48.0 kHz
> Frame rate : 41.667 FPS (1152 SPF)
> Compression mode : Lossy
> Delay relative to video : -29 ms
> Stream size : 82.4 MiB (10%)
> Language : English
>
> Menu
> ID : 4096 (0x1000)
> Menu ID : 1 (0x1)
> Format : MPEG Video / MPEG Audio
> Duration : 30 min 0 s
> List : 256 (0x100) (MPEG Video) /
> 257 (0x101) (MPEG Audio, English)
> Language : / English
> Service name : Service01
> Service provider : FFmpeg
> Service type : digital television
>
>
> On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
>> As suggested you can go some different ways:
>>
>>
>> - Make a python gst app:
>>
>> filesrc ! decodebin ! appsink
>>
>> opencv code
>>
>> appsrc ! encode ! filesink
>>
>> Passing the audio as well might be tricky...
>>
>>
>> - Make a python gst app:
>>
>> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1
>> ! filesink demux1. ! mux1
>>
>> and use a probe to snatch frames at the identify element and do what you
>> want there with opencv
>>
>>
>> - Make a filter element to do your stuff
>>
>> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 !
>> filesink demux1. ! mux1
>>
>>
>> Best Regards
>>
>> Marianna
>>
>>
>> On 13/10/2023 14.00, gstreamer-devel-request at lists.freedesktop.org wrote:
>>> I need to alter video frames, and Python is my happy place.
>>> my gst skills need some help.
>>>
>>> I think I need these 2 things:
>>>
>>> 1. pipeline:
>>> read a file,
>>> decode the video,
>>> hook? to read/write pixels,
>>> encode (keep the original audio)
>>> save to disk.
>>>
>>> 2. that hook code...
>>> I have done something before: use the wavelevel element to get audio
>>> data and generate static png images.
>>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72
>>>
>>> So I am assuming some element / message will give me a frame buffer
>>> thing of data, and somehow i can detect black pixels.
>>>
>>>
>>> End goal: fix some video;s I made of a live event where my setup
>>> introduced black squares - I want to replace the black with the pixels
>>> from the previous frame.
>>> example of what needs fixing:
>>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148
>>>
>>> The slide is mostly a static image, so the previous frame will be fine.
>>>
>>> -- Carl K
>> --
>> Best regards / Med venlig hilsen
>> “Marianna Smidth Buschle”
>>
>
--
Best regards / Med venlig hilsen
“Marianna Smidth Buschle”
More information about the gstreamer-devel
mailing list