Gstreamer with multithread app (Qt)
Matias Hernandez Arellano
msdark at archlinux.cl
Tue Apr 12 15:25:17 PDT 2011
(Sorry i write a lot of emails but i have a lot of ideas/doubts...)
I remove the SIGNAL/SLOT so in the StreamThread i use this:
void StreamThread::run(){
for(;;){
Mat frame = this->ImageBuffer->getFrame();
if(!this->m_src.element().isNull() && !frame.empty()){
this->pipeline->setState(QGst::StatePlaying);
QGst::BufferPtr buffer = QGst::Buffer::create(WIDTH*HEIGHT*3*sizeof(quint8*));
quint8 *data = buffer->data();
data = (quint8*)frame.data;
QGst::FlowReturn ret = this->m_src.pushBuffer(buffer);
switch(ret){
case QGst::FlowNotNegotiated:
qDebug()<<"Not Negotiated";
break;
case QGst::FlowNotSupported:
qDebug()<<"Not Supported";
break;
case QGst::FlowUnexpected:
qDebug()<<"Unexpected";
break;
case QGst::FlowWrongState:
qDebug()<<"Wrong State";
break;
}
}
}
}
the ImageBuffer->getFrame() pull data from the ImageBuffer .. this is a implementation of a Circular Buffer with WaitConditions to ensure that the buffer isn't empty when the pull operation occurs.
So, i check if the appsrc.element() is valid and if the frame obtained is valid and then push into Appsrc using pushBuffer method and check the return value of this operation.. and is QGst::FlowOk .. but in the other side when i try to receive the stream
gst-launch tcpclientsrc host=127.0.0.1 port=5000 ! jpegdec ! queue ! video/x-raw-rgb,width=320,height=240 ! ffmpegcolorspace ! queue ! glimagesink sync=false
i gen nothing .. the pipeline don't put into Playing State.
So the problem (maybe) is not the QTimer problem .. i don't use this and neither SIGNALS and SLOTS
Any idea or guideline will be very appreciate
Thanks in advance..
El 12-04-2011, a las 17:03, Michael Joachimiak escribió:
> Does your StreamThread inherits from QThread? If not you have the answer.
>
> I think generally you have a problem here because you are using singal/slot
> mechanism which was designed for GUI not for normal (fast) operation. I
> could not find it in the documentation (somebody please correct me if wrong)
> but my guess is that signal/slot mechanism is using qtimer.
> Excerpt from the documentation: "In general, emitting a signal that is
> connected to some slots, is approximately ten times slower than calling the
> receivers directly, with non-virtual function calls." From here
> http://doc.qt.nokia.com/latest/signalsandslots.html.
>
> So if you want some real-time processing to happen and you have a
> producer-consumer scenario I would suggest to use semaphores (QSemaphore )
> for that.
>
> Otherways (if qtimer is involved in multithreading) you will get performance
> issues sooner or later.
> Checked on my own skin.
>
>
>
> If not I guess it should
>
> 2011/4/13 Matias Hernandez Arellano <msdark at archlinux.cl>
>
>> (me again)...
>> Well i try it a lot of things and finally i get a proof of concept of the
>> final application semi functional using Qt+OpenCV+QGstreamer ...
>> but.. yet.. i have a problem..
>>
>> Here some snippets of code
>>
>> main.cpp
>> int main(int argc, char *argv[]){
>> QCoreApplication a(argc, argv);
>> QGst::init(&argc,&argv);
>> FrameBuffer *buffer = new FrameBuffer(3);
>> FrameBuffer *buffer2 = new FrameBuffer(3);
>> CaptureThread *capture = new CaptureThread(buffer,0);
>> ProcessingThread *processing = new
>> ProcessingThread(buffer,buffer2,capture->getSourceWidth(),capture->getSourceHeight());
>> StreamThread *stream = new StreamThread(buffer2);
>>
>> QCoreApplication::connect(processing,SIGNAL(newframe()),stream,SLOT(update()),Qt::QueuedConnection);
>> capture->start();
>> processing->start();
>> stream->start();
>> return a.exec();;
>> }
>>
>> buffer and buffer2 are Circular Buffers that hold cv::Mat (OpenCV Images).
>> buffer hold cv::Mat obtained with CaptureThread. ProcessingThread pull
>> images from here.
>> ProcessingThread push images into buffer2 and StreamThread pull images from
>> here.
>>
>> The cycle works good...
>> now the StreamThread
>>
>> StreamThread::StreamThread(FrameBuffer *b, QObject *parent):
>> ImageBuffer(b),QThread(parent){
>> QString pipe_str = QString("appsrc name=\"appsrc\"
>> caps=\"video/x-raw-rgb,width=320,height=240\" ! queue ! videoparse format=14
>> width=%1 height=%1 ! videorate "
>> " ! videoscale !
>> video/x-raw-rgb,width=320,height=240 "
>> " ! queue ! ffmpegcolorspace ! jpegenc !
>> queue "
>> " ! tcpserversink port=5000
>> ").arg(WIDTH,HEIGHT);
>> this->pipeline =
>> QGst::Parse::launch(pipe_str).dynamicCast<QGst::Pipeline>();
>>
>> QGlib::connect(this->pipeline->bus(),"message",this,&StreamThread::onBusMessage);
>> this->pipeline->bus()->addSignalWatch();
>> this->pipeline->setState(QGst::StatePlaying);
>> this->m_src.setElement(this->pipeline->getElementByName("appsrc"));
>> }
>> void StreamThread::onBusMessage(const QGst::MessagePtr &message){
>> switch (message->type()) {
>> case QGst::MessageEos:
>> quit();
>> break;
>> case QGst::MessageError:
>> qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
>> break;
>> default:
>> break;
>> }
>> }
>> void StreamThread::run(){
>> this->exec();
>> }
>>
>> //This SLOT fires up when ProcessingThread calls the newImage() SIGNAL ..
>> // this signal ocurrs when a new processing image was push into the buffer
>> void StreamThread::update(){
>> if(!this->m_src.element().isNull()){
>> this->pipeline->setState(QGst::StatePlaying);
>> QGst::BufferPtr buffer =
>> QGst::Buffer::create(WIDTH*HEIGHT*3*sizeof(quint8*));
>> quint8 *data = buffer->data();
>> Mat frame = this->ImageBuffer->getFrame();
>> data = (quint8*)frame.data;
>> QGst::FlowReturn ret = this->m_src.pushBuffer(buffer);
>> if(ret!=QGst::FlowOk){
>> qDebug()<<"Error pushing data";
>> }
>> }
>> }
>> StreamThread::~StreamThread(){
>> this->pipeline->setState(QGst::StateNull);
>> }
>>
>> When i run this app i don't get errors, just this warning (this doesn't
>> show if i don't execute StreamThread)
>> QObject::startTimer: QTimer can only be used with threads started with
>> QThread
>> So i suppose that QGstreamer use QTimer in some hidden part.
>>
>> And... if i try to see the stream using
>> gst-launch tcpclientsrc host=127.0.0.1 port=5000 ! jpegdec ! queue !
>> video/x-raw-rgb,width=320,height=240 ! ffmpegcolorspace ! queue !
>> glimagesink sync=false
>>
>> I get:
>> Estableciendo el conducto a PAUSA ?
>> El conducto est? PREPAR?NDOSE ?
>>
>> But the pipeline never goes to PLAYING and nothing happen (i can't see the
>> result)...
>>
>> Any idea??
>>
>> Thanks in advance
>>
>>
>> El 08-04-2011, a las 4:34, Tim-Philipp Müller escribió:
>>
>>> On Thu, 2011-04-07 at 18:20 -0400, Matias Hernandez Arellano wrote:
>>>
>>>> It's possible to use gstreamer in a multithread app?
>>>
>>> Yes.
>>>
>>>> I have an Producer/Consumer application where i have two groups 1 with
>>>> 1 Producer and 1 Cosumer (Capture Video with Opencv/Process Image) and
>>>> other group equal ... Take de Process Frame / Stream the frames...
>>>
>>> Have you considered writing a simple source element that captures and
>>> processes the frame directly in a GStreamer pipeline?
>>>
>>>> The communication between threads works good, but know i need to add
>>>> the stream part .. I have another app where i test the functionality of
>>>> AppSrc and works!!! .. So i need to add this (Read from the Buffer
>>>> where i have images, push into AppSrc and Stream over network) in a
>>>> thread.
>>>>
>>>> is this possible??
>>>
>>> Why would this not be possible? Have you tried it and run into problems?
>>>
>>> GStreamer does (almost) all of its processing in dedicated threads of
>>> its own ("streaming threads"). You should be able to push buffers into
>>> appsrc from any application thread (they will get queued internally in
>>> appsrc and then processed in the GStreamer streaming thread).
>>>
>>> Cheers
>>> -Tim
>>>
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>> Matías Hernandez Arellano
>> Ingeniero de Software/Proyectos en VisionLabs S.A
>> CDA Archlinux-CL
>> www.msdark.archlinux.cl
>>
>>
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>
>
>
> --
> Your Sincerely
> Michal Joachimiak
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Matías Hernandez Arellano
Ingeniero de Software/Proyectos en VisionLabs S.A
CDA Archlinux-CL
www.msdark.archlinux.cl
More information about the gstreamer-devel
mailing list