Python access to video frames (Carl Karsten)
Carl Karsten
carl at nextdayvideo.com
Mon Oct 16 18:00:27 UTC 2023
Ima need a little help with the pipeline...
# probably needs an aacparse or parser for whatever the codec of the
audio stream is
# also you need queues after both branches of the demuxer
gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux
name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue !
mux1.
++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
'!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!'
mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1.
'!' queue '!' mux1.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal
data stream error.
Additional debug info:
../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop ():
/GstPipeline:pipeline0/GstTSDemux:demux1:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
ERROR: from element
/GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
Stream doesn't contain enough data.
Additional debug info:
../plugins/elements/gsttypefindelement.c(1012):
gst_type_find_element_chain_do_typefinding ():
/GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
Can't typefind stream
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...
13_25_49.ts came from:
ffmpeg \
-nostdin -y \
-analyzeduration 10000 \
-thread_queue_size 512 \
-i tcp://localhost:11000?timeout=3000000 \
-aspect 16:9 \
-map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4
-qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5
-me_method:v:0 dia \
-map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \
-flags +global_header \
-f segment -segment_time $segment_time \
-segment_format mpegts \
-strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts"
https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54
and in case more info helps:
$ mediainfo 13_25_49.ts
General
ID : 1 (0x1)
Complete name : 13_25_49.ts
Format : MPEG-TS
Commercial name : HDV 720p
File size : 838 MiB
Duration : 30 min 0 s
Overall bit rate mode : Variable
Overall bit rate : 3 903 kb/s
Frame rate : 30.000 FPS
Video
ID : 256 (0x100)
Menu ID : 1 (0x1)
Format : MPEG Video
Commercial name : HDV 720p
Format version : Version 2
Format profile : Main at High 1440
Format settings, BVOP : No
Format settings, Matrix : Default
Format settings, GOP : N=5
Codec ID : 2
Duration : 30 min 0 s
Bit rate mode : Variable
Bit rate : 3 323 kb/s
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate : 30.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.120
Time code of first frame : 01:29:59:29
Time code source : Group of pictures header
GOP, Open/Closed : Open
Stream size : 713 MiB (85%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 2
Codec ID : 3
Duration : 30 min 0 s
Bit rate mode : Constant
Bit rate : 384 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 41.667 FPS (1152 SPF)
Compression mode : Lossy
Delay relative to video : -29 ms
Stream size : 82.4 MiB (10%)
Language : English
Menu
ID : 4096 (0x1000)
Menu ID : 1 (0x1)
Format : MPEG Video / MPEG Audio
Duration : 30 min 0 s
List : 256 (0x100) (MPEG Video) /
257 (0x101) (MPEG Audio, English)
Language : / English
Service name : Service01
Service provider : FFmpeg
Service type : digital television
On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <msb at qtec.com> wrote:
>
> As suggested you can go some different ways:
>
>
> - Make a python gst app:
>
> filesrc ! decodebin ! appsink
>
> opencv code
>
> appsrc ! encode ! filesink
>
> Passing the audio as well might be tricky...
>
>
> - Make a python gst app:
>
> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1
> ! filesink demux1. ! mux1
>
> and use a probe to snatch frames at the identify element and do what you
> want there with opencv
>
>
> - Make a filter element to do your stuff
>
> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 !
> filesink demux1. ! mux1
>
>
> Best Regards
>
> Marianna
>
>
> On 13/10/2023 14.00, gstreamer-devel-request at lists.freedesktop.org wrote:
> > I need to alter video frames, and Python is my happy place.
> > my gst skills need some help.
> >
> > I think I need these 2 things:
> >
> > 1. pipeline:
> > read a file,
> > decode the video,
> > hook? to read/write pixels,
> > encode (keep the original audio)
> > save to disk.
> >
> > 2. that hook code...
> > I have done something before: use the wavelevel element to get audio
> > data and generate static png images.
> > https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72
> >
> > So I am assuming some element / message will give me a frame buffer
> > thing of data, and somehow i can detect black pixels.
> >
> >
> > End goal: fix some video;s I made of a live event where my setup
> > introduced black squares - I want to replace the black with the pixels
> > from the previous frame.
> > example of what needs fixing:
> > https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148
> >
> > The slide is mostly a static image, so the previous frame will be fine.
> >
> > -- Carl K
>
> --
> Best regards / Med venlig hilsen
> “Marianna Smidth Buschle”
>
--
Carl K
More information about the gstreamer-devel
mailing list