Gstreamer with multithread app (Qt)

Tue Apr 12 15:14:42 PDT 2011

Well.. i will try!
El 12-04-2011, a las 17:03, Michael Joachimiak escribió:

> Does your StreamThread inherits from QThread? If not you have the answer.
> 
> I think generally you have a problem here because you are using singal/slot
> mechanism which was designed for GUI not for normal (fast) operation. I
> could not find it in the documentation (somebody please correct me if wrong)
> but my guess is that signal/slot mechanism is using qtimer.
> Excerpt from the documentation: "In general, emitting a signal that is
> connected to some slots, is approximately ten times slower than calling the
> receivers directly, with non-virtual function calls." From here
> http://doc.qt.nokia.com/latest/signalsandslots.html.
> 
> So if you want some real-time processing to happen and you have a
> producer-consumer scenario I would suggest to use semaphores (QSemaphore )
> for that.
> 
> Otherways (if qtimer is involved in multithreading) you will get performance
> issues sooner or later.
> Checked on my own skin.
> 
> 
> 
> If not I guess it should
> 
> 2011/4/13 Matias Hernandez Arellano <msdark at archlinux.cl>
> 
>> (me again)...
>> Well i try it a lot of things and finally i get a proof of concept of the
>> final application semi functional using Qt+OpenCV+QGstreamer ...
>> but.. yet.. i have a problem..
>> 
>> Here some snippets of code
>> 
>> main.cpp
>> int main(int argc, char *argv[]){
>>   QCoreApplication a(argc, argv);
>>   QGst::init(&argc,&argv);
>>   FrameBuffer *buffer = new FrameBuffer(3);
>>   FrameBuffer *buffer2 = new FrameBuffer(3);
>>   CaptureThread *capture = new CaptureThread(buffer,0);
>>   ProcessingThread *processing = new
>> ProcessingThread(buffer,buffer2,capture->getSourceWidth(),capture->getSourceHeight());
>>   StreamThread     *stream     = new StreamThread(buffer2);
>> 
>> QCoreApplication::connect(processing,SIGNAL(newframe()),stream,SLOT(update()),Qt::QueuedConnection);
>>   capture->start();
>>   processing->start();
>>   stream->start();
>>   return a.exec();;
>> }
>> 
>> buffer and buffer2 are Circular Buffers that hold cv::Mat (OpenCV Images).
>> buffer hold cv::Mat obtained with CaptureThread. ProcessingThread pull
>> images from here.
>> ProcessingThread push images into buffer2 and StreamThread pull images from
>> here.
>> 
>> The cycle works good...
>> now the StreamThread
>> 
>> StreamThread::StreamThread(FrameBuffer *b, QObject *parent):
>> ImageBuffer(b),QThread(parent){
>>   QString pipe_str = QString("appsrc name=\"appsrc\"
>> caps=\"video/x-raw-rgb,width=320,height=240\" ! queue ! videoparse format=14
>> width=%1 height=%1 ! videorate "
>>                              " ! videoscale !
>> video/x-raw-rgb,width=320,height=240 "
>>                              " ! queue ! ffmpegcolorspace ! jpegenc  !
>> queue  "
>>                              " ! tcpserversink port=5000
>> ").arg(WIDTH,HEIGHT);
>>   this->pipeline =
>> QGst::Parse::launch(pipe_str).dynamicCast<QGst::Pipeline>();
>> 
>> QGlib::connect(this->pipeline->bus(),"message",this,&StreamThread::onBusMessage);
>>   this->pipeline->bus()->addSignalWatch();
>>   this->pipeline->setState(QGst::StatePlaying);
>>   this->m_src.setElement(this->pipeline->getElementByName("appsrc"));
>> }
>> void StreamThread::onBusMessage(const QGst::MessagePtr &message){
>>   switch (message->type()) {
>>   case QGst::MessageEos:
>>       quit();
>>       break;
>>   case QGst::MessageError:
>>       qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
>>       break;
>>   default:
>>       break;
>>   }
>> }
>> void StreamThread::run(){
>>   this->exec();
>> }
>> 
>> //This SLOT fires up when ProcessingThread calls the newImage() SIGNAL ..
>> // this signal ocurrs when a new processing image was push into the buffer
>> void StreamThread::update(){
>>   if(!this->m_src.element().isNull()){
>>       this->pipeline->setState(QGst::StatePlaying);
>>       QGst::BufferPtr buffer =
>> QGst::Buffer::create(WIDTH*HEIGHT*3*sizeof(quint8*));
>>       quint8 *data = buffer->data();
>>       Mat frame = this->ImageBuffer->getFrame();
>>       data      = (quint8*)frame.data;
>>       QGst::FlowReturn ret = this->m_src.pushBuffer(buffer);
>>       if(ret!=QGst::FlowOk){
>>           qDebug()<<"Error pushing data";
>>       }
>>   }
>> }
>> StreamThread::~StreamThread(){
>>   this->pipeline->setState(QGst::StateNull);
>> }
>> 
>> When i run this app i don't get errors, just this warning (this doesn't
>> show if i don't execute StreamThread)
>> QObject::startTimer: QTimer can only be used with threads started with
>> QThread
>> So i suppose that QGstreamer use QTimer in some hidden part.
>> 
>> And... if i try to see the stream using
>> gst-launch tcpclientsrc host=127.0.0.1 port=5000 !  jpegdec ! queue !
>> video/x-raw-rgb,width=320,height=240 ! ffmpegcolorspace ! queue !
>> glimagesink sync=false
>> 
>> I get:
>> Estableciendo el conducto a PAUSA ?
>> El conducto est? PREPAR?NDOSE ?
>> 
>> But the pipeline never goes to PLAYING and nothing happen (i can't see the
>> result)...
>> 
>> Any idea??
>> 
>> Thanks in advance
>> 
>> 
>> El 08-04-2011, a las 4:34, Tim-Philipp Müller escribió:
>> 
>>> On Thu, 2011-04-07 at 18:20 -0400, Matias Hernandez Arellano wrote:
>>> 
>>>> It's possible to use gstreamer in a multithread app?
>>> 
>>> Yes.
>>> 
>>>> I have an Producer/Consumer application where i have two groups 1 with
>>>> 1 Producer and 1 Cosumer (Capture Video with Opencv/Process Image) and
>>>> other group equal ... Take de Process Frame  / Stream the frames...
>>> 
>>> Have you considered writing a simple source element that captures and
>>> processes the frame directly in a GStreamer pipeline?
>>> 
>>>> The communication between threads works good, but know i need to add
>>>> the stream part .. I have another app where i test the functionality of
>>>> AppSrc and works!!! .. So i need to add this (Read from the Buffer
>>>> where i have images, push into AppSrc and Stream over network) in a
>>>> thread.
>>>> 
>>>> is this possible??
>>> 
>>> Why would this not be possible? Have you tried it and run into problems?
>>> 
>>> GStreamer does (almost) all of its processing in dedicated threads of
>>> its own ("streaming threads"). You should be able to push buffers into
>>> appsrc from any application thread (they will get queued internally in
>>> appsrc and then processed in the GStreamer streaming thread).
>>> 
>>> Cheers
>>> -Tim
>>> 
>>> 
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> 
>> Matías Hernandez Arellano
>> Ingeniero de Software/Proyectos en VisionLabs S.A
>> CDA Archlinux-CL
>> www.msdark.archlinux.cl
>> 
>> 
>> 
>> 
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> 
> 
> 
> 
> -- 
> Your Sincerely
> Michal Joachimiak
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Matías Hernandez Arellano
Ingeniero de Software/Proyectos en VisionLabs S.A
CDA Archlinux-CL
www.msdark.archlinux.cl