<div dir="ltr"> <span style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Dear All,</span><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">I have been trying to put together a working example for an AppSrc based pipeline. The objective is to capture RGB frames through webcam using OpenCV's VideoCapture module. These frames are to be pushed into a Gstreamer pipeline using AppSrc (in live-mode), and then they are either displayed on screen using autovideosink/glimagesink, or they are saved in an AVI video.</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">This example is constructed to serve as a proof of concept for a different application, which continuously generates processed frames. I am successfully able to get the display properly with an error on console (as given below), repeated multiple times. Filesink however, also gives the same error, but does not write the AVI file properly. The file remains 1KB in size and does not open. The application freezes if I try to insert videorate into the pipeline.</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">Error repeated on console: (javaw.exe:10320): GStreamer-CRITICAL **: gst_segment_to_running_time: assertion 'segment->format == format' failed</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">Attached below is my code for your reference. Any help in sorting out this application will be very valuable. I really appreciate your time for giving an answer.</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">Thanks,</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">Manoj</div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial">//////////////////////////////////////////////////////////////////////////////////////////////</div><div style="text-decoration-style:initial;text-decoration-color:initial"><div><span style="font-size:12.8px">/*</span></div><div><span style="font-size:12.8px"> * Used jars: jna-4.4.0.jar; opencv-330.jar; gst1-java-core-0.9.3.jar</span></div><div><span style="font-size:12.8px"> * Gstreamer version - 1.12.4, opencv version - 3.3.0</span></div><div><span style="font-size:12.8px"> * </span></div><div><span style="font-size:12.8px"> */</span></div><div><span style="font-size:12.8px">import java.nio.ByteBuffer;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Buffer;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Bus;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Caps;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.ClockTime;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Element;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.ElementFactory;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Format;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Gst;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.GstObject;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Message;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.Pipeline;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.elements.AppSrc;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.elements.AppSrc.Type;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.event.EOSEvent;</span></div><div><span style="font-size:12.8px">import org.freedesktop.gstreamer.lowlevel.MainLoop;</span></div><div><span style="font-size:12.8px">import org.opencv.core.Core;</span></div><div><span style="font-size:12.8px">import org.opencv.core.Mat;</span></div><div><span style="font-size:12.8px">import org.opencv.videoio.VideoCapture;</span></div><div><span style="font-size:12.8px">import org.opencv.videoio.Videoio;</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">public class AppSrcToFileSinkExample {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>static {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>System.loadLibrary(Core.NATIVE_LIBRARY_NAME);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private AppSrc appSrc;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Pipeline pipe;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element queue;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element videoRate;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element convert;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element encoder;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element muxer;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element fileSink;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Element glSink;</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private MainLoop loop;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private static int count = 0;</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Mat matImage;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private VideoCapture capture;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private int sourceWidth;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private int sourceHeight;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private int fps = 15;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Caps videoCaps;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private long mPts = 0;</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>private Bus bus;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public AppSrcToFileSinkExample() {</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/****************** Creating Elements *****************************/</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>queue = ElementFactory.make("queue2", "queue");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>videoRate = ElementFactory.make("videorate", "videoRate");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>convert = ElementFactory.make("videoconvert", "convert");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>encoder = ElementFactory.make("jpegenc", "encoder");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>muxer = ElementFactory.make("avimux", "muxer");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>fileSink = ElementFactory.make("filesink", "fileSink");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>fileSink.set("location","C:/MvaWorkspace/AppSrcExample/test.avi");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>glSink = ElementFactory.make("glimagesink", "glSink");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/********************** End **************************************/</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/****************** OpenCV videocapture() and appSrc************************/</span></div><div><span style="white-space:pre;font-size:12.8px"> </span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture = new VideoCapture(0);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.set(Videoio.CAP_PROP_FRAME_WIDTH, 640);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.set(Videoio.CAP_PROP_FRAME_HEIGHT, 480);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.set(Videoio.CAP_PROP_FRAME_COUNT, fps);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.set(Videoio.CAP_PROP_FORMAT, Videoio.CAP_MODE_BGR);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>matImage = new Mat();</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>loop = new MainLoop();</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>sourceWidth = 640;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>sourceHeight = 480;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>videoCaps = Caps.fromString("video/x-raw, format=BGR, framerate=15/1, pixel-aspect-ratio=1/1, interlace-mode=progressive, width=" + 640 + ", height=" + 480);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc = (AppSrc) ElementFactory.make("appsrc", "appSrc");</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setLive(true);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setStreamType(Type.STREAM);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setFormat(Format.TIME);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setLatency(0, 1000*1000*1000);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setSize(-1);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setMaxBytes(sourceWidth * sourceHeight * 12);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setCaps(videoCaps);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.set("emit-signals", true);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.setTimestamp(true);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>mPts = appSrc.getBaseTime().toMicros();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc.connect(new AppSrc.NEED_DATA() {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>@Override</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public void needData(AppSrc appSrc1, int size) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>System.out.println("Count: "+count);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>byte[] imgBytes = getImageBytes();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>if(imgBytes==null) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>pipe.sendEvent(new EOSEvent());</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>try {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>Thread.sleep(5000);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>} catch (InterruptedException e) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>e.printStackTrace();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>input(appSrc1, imgBytes);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>});</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/********************** End **************************************/</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/****************** Launch pipeline *****************************/</span></div><div><span style="font-size:12.8px"> pipe = new Pipeline();</span></div><div><span style="font-size:12.8px"> </span></div><div><span style="font-size:12.8px"> //uncomment for writing to file</span></div><div><span style="font-size:12.8px"> pipe.addMany(appSrc, queue, convert, encoder, muxer, fileSink);</span></div><div><span style="font-size:12.8px"> Pipeline.linkMany(appSrc, queue, convert, encoder, muxer, fileSink);</span></div><div><span style="font-size:12.8px"> </span></div><div><span style="font-size:12.8px"> //uncomment to display video</span></div><div><span style="font-size:12.8px"> //pipe.addMany(appSrc, queue, convert, glSink);</span></div><div><span style="font-size:12.8px"> //Pipeline.linkMany(appSrc, queue, convert, glSink);</span></div><div><span style="font-size:12.8px"> </span></div><div><span style="font-size:12.8px"> bus = pipe.getBus();</span></div><div><span style="font-size:12.8px"> bus.connect(new Bus.MESSAGE() {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>@Override</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public void busMessage(Bus arg0, Message arg1) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>System.out.println(arg1.getStructure());</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>});</span></div><div><span style="font-size:12.8px"> bus.connect(new Bus.EOS() {</span></div><div><span style="font-size:12.8px"> @Override</span></div><div><span style="font-size:12.8px"> public void endOfStream(GstObject source) {</span></div><div><span style="font-size:12.8px"> System.out.println("Reached end of stream");</span></div><div><span style="font-size:12.8px"> //To allow End of Stream to settle down, I observed that a sleep of around 5 seconds is required</span></div><div><span style="font-size:12.8px"> try {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>Thread.sleep(5000);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>} catch (InterruptedException e) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>e.printStackTrace();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"> loop.quit();</span></div><div><span style="font-size:12.8px"> }</span></div><div><span style="font-size:12.8px"> });</span></div><div><span style="font-size:12.8px"> </span></div><div><span style="font-size:12.8px"> pipe.play();</span></div><div><span style="font-size:12.8px"> loop.run();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/********************** End **************************************/</span></div><div><span style="font-size:12.8px"> <span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public void input(AppSrc appSrc1, byte[] imageBytes) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>Buffer buf = new Buffer(imageBytes.length);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>ByteBuffer byteBuffer = ByteBuffer.wrap(imageBytes);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>buf.map(true).put(byteBuffer);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>buf.setDuration(ClockTime.fromMicros(1000000 / 15));</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>buf.setPresentationTimestamp(ClockTime.fromMicros(mPts+buf.getDuration().toMicros()));</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>mPts = buf.getPresentationTimestamp().toMicros();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>appSrc1.pushBuffer(buf);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>count++;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>/////////////////////</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>//Just wanted to experiment with only 500 frames, and the code below is written accordingly</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public byte[] getImageBytes() {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>byte[] imageBytes = null;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>if(count >=500) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.release();</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>return null;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>if (capture.isOpened()) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>capture.read(matImage);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>if (!matImage.empty()) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>System.out.println("matImage.channels(): " + matImage.channels());</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>imageBytes = new byte[matImage.channels() * matImage.cols() * matImage.rows()];</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>matImage.get(0, 0, imageBytes);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>System.out.println(imageBytes.length);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>return imageBytes;</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>}</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>public static void main(String[] args) {</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>Gst.init("appsrc", args);</span></div><div><span style="font-size:12.8px"><span style="white-space:pre"> </span>new AppSrcToFileSinkExample();</span></div><div><span style="font-size:12.8px"> }</span></div><div><span style="font-size:12.8px">}</span></div></div> <br></div>