Gstreamer with multithread app (Qt)

[Michael Joachimiak]

Does your StreamThread inherits from QThread? If not you have the answer.

I think generally you have a problem here because you are using singal/slot
mechanism which was designed for GUI not for normal (fast) operation. I
could not find it in the documentation (somebody please correct me if wrong)
but my guess is that signal/slot mechanism is using qtimer.
Excerpt from the documentation: "In general, emitting a signal that is
connected to some slots, is approximately ten times slower than calling the
receivers directly, with non-virtual function calls." From here
http://doc.qt.nokia.com/latest/signalsandslots.html.

So if you want some real-time processing to happen and you have a
producer-consumer scenario I would suggest to use semaphores (QSemaphore )
for that.

Otherways (if qtimer is involved in multithreading) you will get performance
issues sooner or later.
Checked on my own skin.



If not I guess it should

2011/4/13 Matias Hernandez Arellano <msdark at archlinux.cl>

> (me again)...
> Well i try it a lot of things and finally i get a proof of concept of the
> final application semi functional using Qt+OpenCV+QGstreamer ...
> but.. yet.. i have a problem..
>
> Here some snippets of code
>
> main.cpp
>  int main(int argc, char *argv[]){
>    QCoreApplication a(argc, argv);
>    QGst::init(&argc,&argv);
>    FrameBuffer *buffer = new FrameBuffer(3);
>    FrameBuffer *buffer2 = new FrameBuffer(3);
>    CaptureThread *capture = new CaptureThread(buffer,0);
>    ProcessingThread *processing = new
> ProcessingThread(buffer,buffer2,capture->getSourceWidth(),capture->getSourceHeight());
>    StreamThread     *stream     = new StreamThread(buffer2);
>
>  QCoreApplication::connect(processing,SIGNAL(newframe()),stream,SLOT(update()),Qt::QueuedConnection);
>    capture->start();
>    processing->start();
>    stream->start();
>    return a.exec();;
> }
>
> buffer and buffer2 are Circular Buffers that hold cv::Mat (OpenCV Images).
> buffer hold cv::Mat obtained with CaptureThread. ProcessingThread pull
> images from here.
> ProcessingThread push images into buffer2 and StreamThread pull images from
> here.
>
> The cycle works good...
> now the StreamThread
>
>  StreamThread::StreamThread(FrameBuffer *b, QObject *parent):
> ImageBuffer(b),QThread(parent){
>    QString pipe_str = QString("appsrc name=\"appsrc\"
> caps=\"video/x-raw-rgb,width=320,height=240\" ! queue ! videoparse format=14
> width=%1 height=%1 ! videorate "
>                               " ! videoscale !
> video/x-raw-rgb,width=320,height=240 "
>                               " ! queue ! ffmpegcolorspace ! jpegenc  !
> queue  "
>                               " ! tcpserversink port=5000
>  ").arg(WIDTH,HEIGHT);
>    this->pipeline =
> QGst::Parse::launch(pipe_str).dynamicCast<QGst::Pipeline>();
>
>  QGlib::connect(this->pipeline->bus(),"message",this,&StreamThread::onBusMessage);
>    this->pipeline->bus()->addSignalWatch();
>    this->pipeline->setState(QGst::StatePlaying);
>    this->m_src.setElement(this->pipeline->getElementByName("appsrc"));
> }
> void StreamThread::onBusMessage(const QGst::MessagePtr &message){
>    switch (message->type()) {
>    case QGst::MessageEos:
>        quit();
>        break;
>    case QGst::MessageError:
>        qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
>        break;
>    default:
>        break;
>    }
> }
> void StreamThread::run(){
>    this->exec();
> }
>
> //This SLOT fires up when ProcessingThread calls the newImage() SIGNAL ..
> // this signal ocurrs when a new processing image was push into the buffer
> void StreamThread::update(){
>    if(!this->m_src.element().isNull()){
>        this->pipeline->setState(QGst::StatePlaying);
>        QGst::BufferPtr buffer =
> QGst::Buffer::create(WIDTH*HEIGHT*3*sizeof(quint8*));
>        quint8 *data = buffer->data();
>        Mat frame = this->ImageBuffer->getFrame();
>        data      = (quint8*)frame.data;
>        QGst::FlowReturn ret = this->m_src.pushBuffer(buffer);
>        if(ret!=QGst::FlowOk){
>            qDebug()<<"Error pushing data";
>        }
>    }
> }
> StreamThread::~StreamThread(){
>    this->pipeline->setState(QGst::StateNull);
> }
>
> When i run this app i don't get errors, just this warning (this doesn't
> show if i don't execute StreamThread)
> QObject::startTimer: QTimer can only be used with threads started with
> QThread
> So i suppose that QGstreamer use QTimer in some hidden part.
>
> And... if i try to see the stream using
> gst-launch tcpclientsrc host=127.0.0.1 port=5000 !  jpegdec ! queue !
> video/x-raw-rgb,width=320,height=240 ! ffmpegcolorspace ! queue !
> glimagesink sync=false
>
> I get:
> Estableciendo el conducto a PAUSA ?
> El conducto est? PREPAR?NDOSE ?
>
> But the pipeline never goes to PLAYING and nothing happen (i can't see the
> result)...
>
> Any idea??
>
> Thanks in advance
>
>
> El 08-04-2011, a las 4:34, Tim-Philipp Müller escribió:
>
> > On Thu, 2011-04-07 at 18:20 -0400, Matias Hernandez Arellano wrote:
> >
> >> It's possible to use gstreamer in a multithread app?
> >
> > Yes.
> >
> >> I have an Producer/Consumer application where i have two groups 1 with
> >> 1 Producer and 1 Cosumer (Capture Video with Opencv/Process Image) and
> >> other group equal ... Take de Process Frame  / Stream the frames...
> >
> > Have you considered writing a simple source element that captures and
> > processes the frame directly in a GStreamer pipeline?
> >
> >> The communication between threads works good, but know i need to add
> >> the stream part .. I have another app where i test the functionality of
> >> AppSrc and works!!! .. So i need to add this (Read from the Buffer
> >> where i have images, push into AppSrc and Stream over network) in a
> >> thread.
> >>
> >> is this possible??
> >
> > Why would this not be possible? Have you tried it and run into problems?
> >
> > GStreamer does (almost) all of its processing in dedicated threads of
> > its own ("streaming threads"). You should be able to push buffers into
> > appsrc from any application thread (they will get queued internally in
> > appsrc and then processed in the GStreamer streaming thread).
> >
> > Cheers
> > -Tim
> >
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> Matías Hernandez Arellano
> Ingeniero de Software/Proyectos en VisionLabs S.A
> CDA Archlinux-CL
> www.msdark.archlinux.cl
>
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>



-- 
Your Sincerely
Michal Joachimiak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20110413/ddccd6b3/attachment.htm>