Advice needed on plugin state transitions

[Philippe Rouquier]

Hello,

I'm developping a plugin that does speech to text
(https://github.com/PhilippeRo/gst-vosk).

Before performing voice recognition the plugin needs to load large
language models (>1.6G). The plugin does this in a thread while
transitioning from READY to PAUSED.
When the pipeline is started (pulsesrc ! queue ! my STT plugin !
fakesink), there is the request for state change from READY to PAUSED,
the plugin returns GST_STATE_CHANGE_ASYNC, sends an ASYNC_START
message, starts the thread, loads the language model and when the
language model is loaded, sends an ASYNC_DONE message and calls
gst_element_continue_state().

That works well. The queue element right before the STT plugin stores
the buffers while the model is being loaded and, when ready, the STT
plugin can perform recognition on everything that has been recorded
since the pipeline started and "catch up".

Now, my problem is that if the STT plugin needs to load a new model,
while the pipeline is playing, I don't really know what to do.

My first idea was to set the STT plugin back to READY internally -
which cleans the current model during the transition from PAUSED to
READY - and then call gst_element_sync_state_with_parent() which
triggers a new call to set_state(PAUSED) that leads to the
transitioning described above. Except it does not work since after
loading the new model, sending the ASYNC_DONE message and calling
gst_element_continue_state(), the whole pipeline seems inactive and the
STT plugin's chain function is no longer called. Another "symptom" is
that the sinkpad no longer has any caps set.

Now, the other solution I'm using is to leave the STT plugin's state as
is (playing) and just perform the loading of the model as described
above (ASYNC_START message, thread, loading, ASYNC_DONE message,
continue_state()).
The pipeline notices the change since it goes to PAUSED ; but not the
other plugins (including the queue element) since this time the STT
plugin's chain function is called while the model is being loaded. That
forced me to add an internal queue to store the buffers received and
use them later. I'd rather have the queue element do this like in the
first case described.

I also, naively, tried to take the STREAM_LOCK,
gst_pad_set_active(element->sinkpad, FALSE/TRUE) but to no avail since
there are deadlocks (which is the expected behaviour I guess).

Would you have some advice for me on how to handle this case, that is a
plugin which, while it is playing, has to "pause" to perform another
task, leaving it to the queue element to store data until the plugin
can be back online again to use the stored buffers.

I'm doing something wrong here but I can't figure out what.

Thanks for your help.

Cheers,
Philippe Rouquier