AW: AW: AW: Serial port interface on frame grabber

[James Cameron]

G'day vk_gst,

I'm finding this complex, as I've not needed to go to such depth with
GStreamer.  Maybe someone else would have some ideas.

To paraphrase, you have application data arriving at 60 frames per
second and you want to send that along the pipeline with the
corresponding video frame.  Also, ideally, you'd like to have the
frames from the v4l2src captured at about the same time as the
application data; synchronisation.

GStreamer is open source, so you can make whatever changes you like in
a custom system, but your maintenance burden will be lowered if you
can achieve your goals without changing GStreamer.

Several approximate solutions spring to mind; but I don't know which
will be lowest initial cost or lowest eventual cost.  I've never done
any of these myself; I'm speculating.  Here's two;

1.  a custom Gst.Clock instance that is used to drive the entire
pipeline, thus set the v4l2src capture times; with the clock in turn
driven by the arriving IMU data, ... but I don't know if v4l2src can
be driven like this,

2.  a custom v4l2src element that correlates the frames to IMU data,
and emits the IMU data timestamped on another source pad, which you
can later mux back into the stream,

Hope that helps.  Look through the API for Gst.Element, Gst.Bin,
Gst.Element, Gst.Clock, Gst.Message, and Gst.Bus.

On Thu, Jul 19, 2018 at 04:44:49AM -0500, vk_gst wrote:
> Hi james,
> 
> Thanks for the update. I am tackling this solution from the source, so the
> first part is synchronizing the IMU and video frame data. 
> 
> The IMU(runs at 60Hz) device gives me a trigger that I can use to trigger my
> pipeline at 30Hz. 
> So this means I have two sets of IMU data for each frame at a give point of
> time.  
>  I was considering if I can trigger the pipeline at 30Hz, so that the IMU
> and video data is synchronized. 
> 
>  For eg: consider this pipeline 
> gst-launch-1.0 -v v4l2src device=/dev/video1 !
> video/x-raw,width=100,height=50,framerate=30/1 ! filesink location=xyz.avi 
> 
> How could I trigger this based on the external signal. The easiest I can
> think is of writing a C program, that polls for the signal and plays the
> pipeline. But I feel this is not the right way to do it? 
> 
> Can you suggest me if the Gstreamer provides any elements, that can help me
> do this? 
>
> Else, the other way of synchronizing is to use the timestamp of each frame,
> and allot the timestamp to the IMU data at the imx6 side. Later at receiver,
> I need to match the timestamp again to relate each frame to IMU. Would that
> be a better option?

-- 
James Cameron
http://quozl.netrek.org/