Python Gstreamer, Webcam, and Pil

[Andrey Nechypurenko]

On 24 August 2013 10:14, Jason (spot) Brower <encompass at gmail.com> wrote:
> I can take the picture and save the image to a file using just Gstreamer.
> But I need to work with the image before I save it anyway, so how can I
> convert it to PIL after I take the picture.
> A code example would be handy. I have googled and no found it as far as I
> have searched.

One possible way would be to create the pipeline which ends with
fakesink element and use it's handoff mechanism to get access to raw
frame. For example:

videotestsrc is-live=true !
video/x-raw-yuv,width=640,height=480,framerate=30/1 ! ffmpegcolorspace
! video/x-raw-rgb, bpp=32, depth=32 ! fakesink sync=1

The following code illustrates how to update OpenGL texture with it.
Here on_gst_buffer(self, fakesink, buff, pad, data=None) will be
invoked by gstreamer for each new frame and data contains raw decoded
frame in format specified by the caps filter in front of fakesink.

import ctypes

import gobject
import gst

class Receiver(object):

    def __init__(self, video_tex_id, pipeline_string):
        self.tex_updated = True
        self.texdata = (ctypes.c_ubyte * 640 * 480 * 4)()
        self.video_tex_id = video_tex_id

        # Create GStreamer pipeline
        self.pipeline = gst.parse_launch(pipeline_string)

        # Create bus to get events from GStreamer pipeline
        self.bus = self.pipeline.get_bus()
        self.bus.add_signal_watch()
        self.bus.connect('message::eos', self.on_eos)
        self.bus.connect('message::error', self.on_error)

        self.fakesink = self.pipeline.get_by_name('fakesink0')
        self.fakesink.props.signal_handoffs = True
        self.fakesink.connect("handoff", self.on_gst_buffer)

        self.pipeline.set_state(gst.STATE_PLAYING)


    def on_gst_buffer(self, fakesink, buff, pad, data=None):
        self.tex_updated = False
        ctypes.memmove(self.texdata, buff.data, buff.size)
        return gst.FLOW_OK


    def updateTexture(self):
        if not self.tex_updated and not self.texdata == None:
            glBindTexture(GL_TEXTURE_2D, self.video_tex_id)
            glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 640, 480, GL_RGBA,
GL_UNSIGNED_BYTE, self.texdata)
            self.tex_updated = True


    def on_eos(self, bus, msg):
        print('on_eos(): seeking to start of video')
        self.pipeline.seek_simple(
            gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH | gst.SEEK_FLAG_KEY_UNIT,
            0L
        )


    def on_error(self, bus, msg):
        print('on_error():', msg.parse_error())


HTH,
Andrey.