audio speech recognition and recording at the same time with tee

[Nicolas Dufresne]

Le jeu. 7 nov. 2019 03 h 40, kyle111 <purehavoc77 at gmail.com> a écrit :

> Nicolas Dufresne-5 wrote
> > Le lun. 4 nov. 2019 15 h 40, kyle111 <
>
> > purehavoc77@
>
> > > a écrit :
> >
> >> Hello all,
> >>
> >> I am using gstreamer plugin for kaldi
> >> <https://github.com/alumae/gst-kaldi-nnet2-online>   (speech
> >> recognition
> >> tool). This is the  pipeline
> >> <
> >>
> https://github.com/alumae/gst-kaldi-nnet2-online/blob/master/demo/gui-demo.py
> >
> >>
> >> for kaldi plugi to do speech-to-text ( microphone input -> text
> >> transcript):
> >>
> >>     self.pulsesrc.link(self.audioconvert)
> >>     self.audioconvert.link(self.audioresample)
> >>     self.audioresample.link(self.asr)
> >>     self.asr.link(self.fakesink)
> >>
> >>
> >> Now, I'd like to record the audio from microphone into .ogg file at the
> >> same
> >> time. I made the following changes to the pipeline codes:
> >>
> >>     # get audio from mic device
> >>     self.pulsesrc.link(self.audioconvert1)
> >>     self.audioconvert1.link(self.tee)
> >>
> >>     # save to ogg
> >>     self.audio_queue.link(self.vorbisenc)
> >>     self.vorbisenc.link(self.oggmux)
> >>     self.oggmux.link(self.filesink)
> >>
> >>     # send to asr
> >>     self.asr_queue.link(self.audioconvert)
> >>     self.audioconvert.link(self.audioresample)
> >>     self.audioresample.link(self.asr)
> >>     self.asr.link(self.fakesink)
> >>
> >>     # tied up tee and queues
> >>     self.tee.link(self.audio_queue)
> >>     self.tee.link(self.asr_queue)
> >>
> >> However, the pipeline doesn't work, There is no gst errors but audio is
> >> not
> >> saved and text doesn't come out.
> >> I suspect something is blocking the pipeline and I couldn't figure it
> >> out.
> >> Would like to hear suggestions !
> >>
> >
> > Check your return values. Something must be failing. The error is
> returned
> > by GstPad.link() call directly, and may not appear on the bus.
>
> Hello Nicolas, thanks again for lending a hand.
> Not sure what you mean by 'check return values'. You mean I should return
> the above pipelin w/ gst-launch in terminal instead of python lib? I could
> try that out.
>

No, I simply mean that function like link() have a return value, in this
case true or false, false would mean that the link failed.


> so I found 1 interesting thing.
> >     self.tee.link(self.audio_queue)
> >     self.tee.link(self.asr_queue)
>
> whenever, I disable of the one of 'tee' links to queue, the other queue
> works (can record audio or can get speech-2-text ) but not both at the same
> time.
>

If links didn't fail, check the queue level, maybe one end accumulate more
then the queue capacity, preventing preroll. You may also chose to disable
preroll using async property you let's say the speech to text sink.

>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20191107/591b9ab0/attachment-0001.html>