Python access to video frames (Carl Karsten)
Carl Karsten
carl at nextdayvideo.com
Tue Oct 17 17:32:57 UTC 2023
no error, but no play.
gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
'!' queue '!' decodebin3 '!' avenc_mpeg2video '!' mpegtsmux name=mux1
'!' filesink location=fixed_13_25_49.ts demux1. '!' queue '!'
mpegaudioparse '!' mux1.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
htop shows no cpu and ls -l shows no bytes.
On Tue, Oct 17, 2023 at 4:19 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
>
> The demuxer divides the pipeline into audio and video, you get one in
> each branch.
>
> Right now you seem to have both an audioparser and a videoencoder in the
> same branch.
>
> Try instead:
>
> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
> queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts
> demux1. ! queue ! mpegaudioparse ! mux1.
>
>
> And generally, when in doubt about pipeline errors it is always useful
> to simplify it and then build up.
> Fx: filesrc ! fakesink
> then filesrc ! demux ! fakesink
> then filesrc ! demux ! decode ! fakesink
> then filesrc ! demux ! decode ! videosink
> ...
>
> Best Regards
> Marianna
>
> On 16/10/2023 20.00, Carl Karsten wrote:
> > Ima need a little help with the pipeline...
> >
> > # probably needs an aacparse or parser for whatever the codec of the
> > audio stream is
> > # also you need queues after both branches of the demuxer
> >
> > gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
> > queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux
> > name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue !
> > mux1.
> >
> >
> > ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
> > '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!'
> > mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1.
> > '!' queue '!' mux1.
> > Setting pipeline to PAUSED ...
> > Pipeline is PREROLLING ...
> > ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal
> > data stream error.
> > Additional debug info:
> > ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop ():
> > /GstPipeline:pipeline0/GstTSDemux:demux1:
> > streaming stopped, reason not-negotiated (-4)
> > ERROR: pipeline doesn't want to preroll.
> > Setting pipeline to NULL ...
> > ERROR: from element
> > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
> > Stream doesn't contain enough data.
> > Additional debug info:
> > ../plugins/elements/gsttypefindelement.c(1012):
> > gst_type_find_element_chain_do_typefinding ():
> > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
> > Can't typefind stream
> > ERROR: pipeline doesn't want to preroll.
> > Freeing pipeline ...
> >
> > 13_25_49.ts came from:
> >
> > ffmpeg \
> > -nostdin -y \
> > -analyzeduration 10000 \
> > -thread_queue_size 512 \
> > -i tcp://localhost:11000?timeout=3000000 \
> > -aspect 16:9 \
> > -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4
> > -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5
> > -me_method:v:0 dia \
> > -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \
> > -flags +global_header \
> > -f segment -segment_time $segment_time \
> > -segment_format mpegts \
> > -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts"
> >
> > https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54
> >
> > and in case more info helps:
> >
> > $ mediainfo 13_25_49.ts
> > General
> > ID : 1 (0x1)
> > Complete name : 13_25_49.ts
> > Format : MPEG-TS
> > Commercial name : HDV 720p
> > File size : 838 MiB
> > Duration : 30 min 0 s
> > Overall bit rate mode : Variable
> > Overall bit rate : 3 903 kb/s
> > Frame rate : 30.000 FPS
> >
> > Video
> > ID : 256 (0x100)
> > Menu ID : 1 (0x1)
> > Format : MPEG Video
> > Commercial name : HDV 720p
> > Format version : Version 2
> > Format profile : Main at High 1440
> > Format settings, BVOP : No
> > Format settings, Matrix : Default
> > Format settings, GOP : N=5
> > Codec ID : 2
> > Duration : 30 min 0 s
> > Bit rate mode : Variable
> > Bit rate : 3 323 kb/s
> > Width : 1 280 pixels
> > Height : 720 pixels
> > Display aspect ratio : 16:9
> > Frame rate : 30.000 FPS
> > Color space : YUV
> > Chroma subsampling : 4:2:0
> > Bit depth : 8 bits
> > Scan type : Progressive
> > Compression mode : Lossy
> > Bits/(Pixel*Frame) : 0.120
> > Time code of first frame : 01:29:59:29
> > Time code source : Group of pictures header
> > GOP, Open/Closed : Open
> > Stream size : 713 MiB (85%)
> > Color primaries : BT.709
> > Transfer characteristics : BT.709
> > Matrix coefficients : BT.709
> >
> > Audio
> > ID : 257 (0x101)
> > Menu ID : 1 (0x1)
> > Format : MPEG Audio
> > Format version : Version 1
> > Format profile : Layer 2
> > Codec ID : 3
> > Duration : 30 min 0 s
> > Bit rate mode : Constant
> > Bit rate : 384 kb/s
> > Channel(s) : 2 channels
> > Sampling rate : 48.0 kHz
> > Frame rate : 41.667 FPS (1152 SPF)
> > Compression mode : Lossy
> > Delay relative to video : -29 ms
> > Stream size : 82.4 MiB (10%)
> > Language : English
> >
> > Menu
> > ID : 4096 (0x1000)
> > Menu ID : 1 (0x1)
> > Format : MPEG Video / MPEG Audio
> > Duration : 30 min 0 s
> > List : 256 (0x100) (MPEG Video) /
> > 257 (0x101) (MPEG Audio, English)
> > Language : / English
> > Service name : Service01
> > Service provider : FFmpeg
> > Service type : digital television
> >
> >
> > On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
> >> As suggested you can go some different ways:
> >>
> >>
> >> - Make a python gst app:
> >>
> >> filesrc ! decodebin ! appsink
> >>
> >> opencv code
> >>
> >> appsrc ! encode ! filesink
> >>
> >> Passing the audio as well might be tricky...
> >>
> >>
> >> - Make a python gst app:
> >>
> >> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1
> >> ! filesink demux1. ! mux1
> >>
> >> and use a probe to snatch frames at the identify element and do what you
> >> want there with opencv
> >>
> >>
> >> - Make a filter element to do your stuff
> >>
> >> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 !
> >> filesink demux1. ! mux1
> >>
> >>
> >> Best Regards
> >>
> >> Marianna
> >>
> >>
> >> On 13/10/2023 14.00, gstreamer-devel-request at lists.freedesktop.org wrote:
> >>> I need to alter video frames, and Python is my happy place.
> >>> my gst skills need some help.
> >>>
> >>> I think I need these 2 things:
> >>>
> >>> 1. pipeline:
> >>> read a file,
> >>> decode the video,
> >>> hook? to read/write pixels,
> >>> encode (keep the original audio)
> >>> save to disk.
> >>>
> >>> 2. that hook code...
> >>> I have done something before: use the wavelevel element to get audio
> >>> data and generate static png images.
> >>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72
> >>>
> >>> So I am assuming some element / message will give me a frame buffer
> >>> thing of data, and somehow i can detect black pixels.
> >>>
> >>>
> >>> End goal: fix some video;s I made of a live event where my setup
> >>> introduced black squares - I want to replace the black with the pixels
> >>> from the previous frame.
> >>> example of what needs fixing:
> >>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148
> >>>
> >>> The slide is mostly a static image, so the previous frame will be fine.
> >>>
> >>> -- Carl K
> >> --
> >> Best regards / Med venlig hilsen
> >> “Marianna Smidth Buschle”
> >>
> >
> --
> Best regards / Med venlig hilsen
> “Marianna Smidth Buschle”
>
--
Carl K
More information about the gstreamer-devel
mailing list