Using appsink in an shmsrc pipeline

[Michiel Konstapel]

On 02-02-2023 23:36, Kyle Flores via gstreamer-devel wrote:
> Hi gst-devel,
>
> I'm working on an application where a single camera produces a stream 
> that I'd like multiple clients (such as a display, or network stream) 
> to use. One such client is an appsink that does some custom work, and 
> then stops after it has processed a fixed number of buffers. My goal 
> is to capture a "snapshot" of the camera stream with this, and am 
> trying to have an appsink pipeline run temporarily and then go back to 
> null, but I'm open to other ways to accomplish the same thing, as 
> something's incorrect about mine.
>
> Here are pipelines descriptions that simulate my configuration, but in 
> practice run inside an application rather than by gst-launch-1.0.
> - "camera" source: videotestsrc is-live=true pattern=ball ! 
> video/x-raw,format=I420,width=640,height=480,framerate=60/1 ! shmsink 
> name=source_shmsink socket-path=/tmp/shmdemo sync=false 
> wait-for-connection=false
> - display client: shmsrc name=watch_shmsrc socket-path=/tmp/shmdemo 
> is-live=true ! 
> video/x-raw,format=I420,width=640,height=480,framerate=60/1 ! 
> videoconvert ! autovideosink sync=false
> - appsink client: shmsrc name=app_shmsrc socket-path=/tmp/shmdemo 
> is-live=true ! 
> video/x-raw,format=I420,width=640,height=480,framerate=60/1 ! appsink 
> sink=false
>
> All three pipelines are set to playing, I see the display pop up, and 
> then another thread begins calling pull_sample on the appsink, 
> reporting each sample that arrives. The thread stops calling 
> pull_sample after handling a fixed number of samples, then tries to 
> set the state of the appsink client pipeline to NULL. In the logs I 
> see the appsink go to NULL, but a short time after this, the display 
> client also appears to stop. Additionally, I don't see that message 
> from shmsink that a client disconnected that appears when other shmsrc 
> clients go to NULL. I tried various combinations of properties on 
> shmsrc/sink and appsink but haven't seen anything change yet. Ideally, 
> I'd like the camera source and the display client to continue even 
> after the appsink client stops, and I don't understand how stopping 
> the appsink client affects the display client.
>
> I'd be curious to know if/how I'm misusing appsink or the shmsrc/sink 
> pair. Specifically for the shm plugins I haven't found much precise 
> info so my usage has just been based on observing their behavior.  
> I've seen my app do the same thing on different gst versions so I 
> don't think this is a bug with the plugins themselves.
>
> Thanks for any suggestions or pointers.

I have never used shmsink but I wonder if it is intended to support 
multiple sources "connecting" to it. The documentation says "Send data 
over shared memory to *the* matching source", emphasis mine. Can you 
have a tee in your camera pipeline and separate shmsinks for each 
intended consumer? Something like this:

videotestsrc is-live=true pattern=ball ! 
video/x-raw,format=I420,width=640,height=480,framerate=60/1 ! tee name=t \
t. ! queue ! shmsink name=display_shmsink 
socket-path=/tmp/shmdemo_display sync=false wait-for-connection=false \
t. ! queue ! shmsink name=appsink_shmsink 
socket-path=/tmp/shmdemo_appsink sync=false wait-for-connection=false

And then have each consumer read from its designated socket path.

Cheers,
Michiel