Gstreamer with multithread app (Qt)

[Matias Hernandez Arellano]

(me again)...
Well i try it a lot of things and finally i get a proof of concept of the final application semi functional using Qt+OpenCV+QGstreamer ...
but.. yet.. i have a problem..

Here some snippets of code

main.cpp
 int main(int argc, char *argv[]){
    QCoreApplication a(argc, argv);
    QGst::init(&argc,&argv);
    FrameBuffer *buffer = new FrameBuffer(3);
    FrameBuffer *buffer2 = new FrameBuffer(3);
    CaptureThread *capture = new CaptureThread(buffer,0);
    ProcessingThread *processing = new ProcessingThread(buffer,buffer2,capture->getSourceWidth(),capture->getSourceHeight());
    StreamThread     *stream     = new StreamThread(buffer2);
    QCoreApplication::connect(processing,SIGNAL(newframe()),stream,SLOT(update()),Qt::QueuedConnection);
    capture->start();
    processing->start();
    stream->start();
    return a.exec();;
}

buffer and buffer2 are Circular Buffers that hold cv::Mat (OpenCV Images).
buffer hold cv::Mat obtained with CaptureThread. ProcessingThread pull images from here.
ProcessingThread push images into buffer2 and StreamThread pull images from here.

The cycle works good...
now the StreamThread

 StreamThread::StreamThread(FrameBuffer *b, QObject *parent): ImageBuffer(b),QThread(parent){
    QString pipe_str = QString("appsrc name=\"appsrc\" caps=\"video/x-raw-rgb,width=320,height=240\" ! queue ! videoparse format=14 width=%1 height=%1 ! videorate "
                               " ! videoscale ! video/x-raw-rgb,width=320,height=240 "
                               " ! queue ! ffmpegcolorspace ! jpegenc  ! queue  "
                               " ! tcpserversink port=5000  ").arg(WIDTH,HEIGHT);
    this->pipeline = QGst::Parse::launch(pipe_str).dynamicCast<QGst::Pipeline>();
    QGlib::connect(this->pipeline->bus(),"message",this,&StreamThread::onBusMessage);
    this->pipeline->bus()->addSignalWatch();
    this->pipeline->setState(QGst::StatePlaying);
    this->m_src.setElement(this->pipeline->getElementByName("appsrc"));
}
void StreamThread::onBusMessage(const QGst::MessagePtr &message){
    switch (message->type()) {
    case QGst::MessageEos:
        quit();
        break;
    case QGst::MessageError:
        qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
        break;
    default:
        break;
    }
}
void StreamThread::run(){
    this->exec();
}

//This SLOT fires up when ProcessingThread calls the newImage() SIGNAL ..
// this signal ocurrs when a new processing image was push into the buffer
void StreamThread::update(){ 
    if(!this->m_src.element().isNull()){
        this->pipeline->setState(QGst::StatePlaying);
        QGst::BufferPtr buffer = QGst::Buffer::create(WIDTH*HEIGHT*3*sizeof(quint8*));
        quint8 *data = buffer->data();
        Mat frame = this->ImageBuffer->getFrame();
        data      = (quint8*)frame.data;
        QGst::FlowReturn ret = this->m_src.pushBuffer(buffer);
        if(ret!=QGst::FlowOk){
            qDebug()<<"Error pushing data";
        }
    }
}
StreamThread::~StreamThread(){
    this->pipeline->setState(QGst::StateNull);
}

When i run this app i don't get errors, just this warning (this doesn't show if i don't execute StreamThread)
QObject::startTimer: QTimer can only be used with threads started with QThread
So i suppose that QGstreamer use QTimer in some hidden part.

And... if i try to see the stream using
gst-launch tcpclientsrc host=127.0.0.1 port=5000 !  jpegdec ! queue ! video/x-raw-rgb,width=320,height=240 ! ffmpegcolorspace ! queue ! glimagesink sync=false

I get:
Estableciendo el conducto a PAUSA ?
El conducto est? PREPAR?NDOSE ?

But the pipeline never goes to PLAYING and nothing happen (i can't see the result)...

Any idea??

Thanks in advance


El 08-04-2011, a las 4:34, Tim-Philipp Müller escribió:

> On Thu, 2011-04-07 at 18:20 -0400, Matias Hernandez Arellano wrote:
> 
>> It's possible to use gstreamer in a multithread app?
> 
> Yes.
> 
>> I have an Producer/Consumer application where i have two groups 1 with
>> 1 Producer and 1 Cosumer (Capture Video with Opencv/Process Image) and
>> other group equal ... Take de Process Frame  / Stream the frames...
> 
> Have you considered writing a simple source element that captures and
> processes the frame directly in a GStreamer pipeline?
> 
>> The communication between threads works good, but know i need to add
>> the stream part .. I have another app where i test the functionality of
>> AppSrc and works!!! .. So i need to add this (Read from the Buffer
>> where i have images, push into AppSrc and Stream over network) in a
>> thread.
>> 
>> is this possible??
> 
> Why would this not be possible? Have you tried it and run into problems?
> 
> GStreamer does (almost) all of its processing in dedicated threads of
> its own ("streaming threads"). You should be able to push buffers into
> appsrc from any application thread (they will get queued internally in
> appsrc and then processed in the GStreamer streaming thread).
> 
> Cheers
> -Tim
> 
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Matías Hernandez Arellano
Ingeniero de Software/Proyectos en VisionLabs S.A
CDA Archlinux-CL
www.msdark.archlinux.cl