You'll have to excuse my ignorance in my last email. I had the mixer in the gnloperation, but I had the src pad of the mixer attached to the final output segment rather than the composition src pad. I fixed that, but am still running into issues. So, currently, I have a pipeline which resembles this: +--------------------------------+ | gnlcomposition | | +----------------------------+ | | | Box 1 (gnlfilesource) | | +-------+ +-------------------------------+ | +----------------------------+ |->| queue |->| filesink (plus encoder stuff) | | | Box 2 (gnlfilesource) | | +-------+ +-------------------------------+ | +----------------------------+ | | | Video Mixer (gnloperation) | | | +----------------------------+ | +--------------------------------+ I hope that comes through ok.. Box 1, box 2 and video mixer all have the same duration and the input-priority-changed call is triggering the update to xpos and ypos on the video mixer pads. Ultimately there will be 6 boxes per frame, but that I'm trying to keep it simple at the moment. The issue I'm currently running into is with the video mixer. With the Box 1 and Box 2 being 240x240 px, the final frame size is 240x240 px. Is there a way to feed in a blank 720x480 frame into the mixer so it outputs the proper resolution or am I looking at this the wrong way? I've tried to input a 720x480 frame from a videotestsrc, but am having issues (structurally I believe) getting my test source to exist in a gnlsource at a 720x480 resolution. Again, any help is greatly, greatly appreciated as I'm running short on time to get this complete. Best, Tim <div class="gmail_quote">On Sun, Nov 28, 2010 at 9:06 PM, Timothy Braun <<a href="mailto:braunsquared@gmail.com">braunsquared@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"> Hi Edward, Thanks for the quick reply. It has been most informative. A couple questions if I may. For the video rescale/capsfilter part, what is this doing and how should it look in the pipeline? The input videos are all 240x240 px with a goal output frame size of 720x480px. For a very raw test, I am using this to build my pipeline: GstPipeline *pipeline = GST_PIPELINE(gst_pipeline_new("pipeline")); GstCaps *caps = gst_caps_from_string("video/x-raw-yuv;video/x-raw-rgb"); // Create our composition GstElement *comp = gst_element_factory_make("gnlcomposition", "composition"); g_object_set(G_OBJECT(comp), "caps", caps, NULL); GstElement *src1 = gst_element_factory_make("gnlfilesource", "source1"); g_object_set(G_OBJECT(src1), "location", "loop1.mp4", "start", 0, "duration", 30 * GST_SECOND, "media-start", 0, "media-duration", 30 * GST_SECOND, "priority", 1, "caps", caps, NULL); GstElement *src2 = gst_element_factory_make("gnlfilesource", "source2"); g_object_set(G_OBJECT(src2), "location", "loop2.mp4", "start", 0, "duration", 30 * GST_SECOND, "media-start", 0, "media-duration", 30 * GST_SECOND, "priority", 2, "caps", caps, NULL); GstElement *oper = gst_element_factory_make("gnloperation", NULL); g_object_set(G_OBJECT(oper), "caps", caps, "expandable", TRUE, NULL); GstElement *mixer = gst_element_factory_make("videomixer", NULL); gst_bin_add(GST_BIN(oper), mixer); // listen for the input priority change signal // to update the video mixer pad g_object_connect(oper, "signal::input-priority-changed", onPriorityChange, mixer, NULL ); // add the sources to the composition gst_bin_add(GST_BIN(comp), src1); gst_bin_add(GST_BIN(comp), src2); gst_bin_add(GST_BIN(comp), oper); // build the output stream GstElement *color = gst_element_factory_make("ffmpegcolorspace", "colorspace"); GstElement *identity = gst_element_factory_make("identity", "ident"); g_object_set(identity, "single-segment", TRUE, NULL); GstElement *enc = gst_element_factory_make("ffenc_mpeg1video", "encoder"); GstElement *mux = gst_element_factory_make("ffmux_mpeg", "mux"); GstElement *queue = gst_element_factory_make("queue", "queue"); GstElement *sink = gst_element_factory_make("filesink", "sink"); g_object_set(sink, "location", "output.mpg", NULL); gst_bin_add_many(GST_BIN(pipeline), comp, color, identity, enc, mux, queue, sink, NULL); /* g_object_connect (comp, "signal::pad-added", onPad, mixer, NULL); */ gst_element_link_many(mixer, queue, color, identity, enc, mux, sink, NULL); It seems to link up fine, but fails to stream as it gets stuck in the paused state. I've tried adding queues all around, but no luck. Here is the debug output: ** Message: Creating run loop... ** Message: Building pipeline... ** Message: Attaching to bus... ** Message: Setting state to PLAYING... 0:00:00.141504000 [334m91821 [00m 0x100609d30 [33;01mWARN [00m [00;01;34m gnlsource gnlsource.c:545:gnl_source_change_state:<source2> [00m Couldn't find a valid source pad 0:00:00.162296000 [334m91821 [00m 0x100609d30 [33;01mWARN [00m [00;01;35m GST_SCHEDULING gstpad.c:4692:gst_pad_get_range:<source:src> [00m getrange failed unexpected 0:00:00.191513000 [334m91821 [00m 0x100609d30 [33;01mWARN [00m [00;01;34m gnlsource gnlsource.c:545:gnl_source_change_state:<source1> [00m Couldn't find a valid source pad 0:00:00.199956000 [334m91821 [00m 0x100686b10 [33;01mWARN [00m [00m qtdemux qtdemux.c:5801:qtdemux_parse_trak:<qtdemux0> [00m unknown version 00000000 0:00:00.200693000 [334m91821 [00m 0x100609d30 [33;01mWARN [00m [00;01;35m GST_SCHEDULING gstpad.c:4692:gst_pad_get_range:<source:src> [00m getrange failed unexpected 0:00:00.210835000 [334m91821 [00m 0x10064f030 [33;01mWARN [00m [00m qtdemux qtdemux.c:5801:qtdemux_parse_trak:<qtdemux1> [00m unknown version 00000000 0:00:00.244706000 [334m91821 [00m 0x101879f60 [33;01mWARN [00m [00;01;34m gnlsource gnlsource.c:221:element_pad_added_cb:<source2> [00m We already have (pending) ghost-ed a valid source pad (ghostpad:'':'', pendingblock:1 0:00:00.244852000 [334m91821 [00m 0x1006501f0 [33;01mWARN [00m [00;01;34m gnlsource gnlsource.c:221:element_pad_added_cb:<source1> [00m We already have (pending) ghost-ed a valid source pad (ghostpad:'':'', pendingblock:1 ** Message: Priority Changed: composition::gnloperation0->gnloperation0::sink_1(videomixer0::sink_1) - 2 ** Message: Priority Changed: composition::gnloperation0->gnloperation0::sink_0(videomixer0::sink_0) - 1 I've been battling with this for a bit now, can't seem to make any headway. Any input is, again, much appreciated. Best, Tim<div><div></div><div class="h5"> <div class="gmail_quote">On Sun, Nov 28, 2010 at 3:58 AM, Edward Hervey <<a href="mailto:bilboed@gmail.com" target="_blank">bilboed@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi, <div> On Sat, 2010-11-27 at 13:50 -0500, Timothy Braun wrote: > Kapil, > Thanks for the suggestion, but with multifilesrc I would have to > have the files static with incrementing named files. A single box in > the 3x2 grid may contain the same file multiple times so I'm afraid it > won't be the best solution. > > I guess, ultimately, there's multiple ways to attack this one from > what I've been able to find. Here are the two that I've been looking > at, I'm just not sure which is the better solution at this point: > * A single gnonlin composition with a 4 gnlsources similar in > setup as the gst-launch text I have below. </div> Using one composition would actually be the 'proper' way. > * 6 gnonlin compositions, each feeding to a single videomixer <div><div></div><div>> which combines them into the final frame. > * This path I'm currently investigating. I have a test > written in C, but I'm having some difficulties with > pad linkage as I still don't have a complete > understanding of when certain things will exist and > how to get them. > * Here's currently whats happening: > * Create a new pipeline > * Create a videomixer > * Create 6 gnonlin compositions each with a > pad-added signal callback to connect > gnlcomposition pad to videomixer. > * ... (this is were it's going wrong) > * In the pad-added callback I have: > * static void onPad(GstElement *comp, GstPad > *pad, GstElement *sink) { > GstPad *v = gst_element_get_pad(sink, > "sink"); > gst_pad_link(pad, v); > gst_object_unref(v); > } > * gst_element_get_pad is not returning a pad > from the video mixer (sink) which leads me to > believe that I'm either not asking in the > right manner or the pad doesn't exist. (I'm > aware that gst_element_get_pad is deprecated, > I'm just looking to test at the moment) > * I noticed in one of the repositories under a > unit test, the videomixer was attached as a > gnloperation? Is this the better path to > take? > > This all leads me to a couple more questions as well: > * A video mixer pad has xpos and ypos properties. This would > let me shift the video around without needing a video box > which I believe may be more efficient? </div></div> Yes, it will be more efficient. <div> > * If I use the xpos and ypos properties, is the video mixer > smart enough to change the frame size appropriately or will it > simply crop the frame to the size of the largest input frame? > * If so, would it be better to add a videobox to do the > adjustments for me, or feed in a solid color > background of the required output size? </div> No, it won't change the size, but what you could do is mix the original sizes with original offsets and then downconvert the video later. Example for one 3x2 segment: Create a gnloperation with a videomixer in it with a gnl priority of 0. Create a gnlfilesource for each clip with increasing priorities (1->6) going from left-right and then top to bottom: 1 2 3 4 5 6 Connect to the gnloperation 'input-priority-changed' signal. When your callback is called, you will know which priority is being connected to which gnloperation ghostpad. You can get the videomixer sink pad by using the gst_ghost_pad_get_target() method and then setting the proper xpos/ypos property on that pad based on the priority of the feed being provided. Set 'video/x-raw-yuv;video/x-raw-rgb' as the caps property on all your sources. Set duration and media-duration of *all* gnlobjects to the same duration. If you want to add another segment of 3x2 clips, you'll need to re-add all those 7 objects with a modified 'start' property. First connect your composition to an imagesink to make sure the result is what you want. When it is, insert a videoscale element followed with a capsfilter with your target resolution. Hope this helps. <div><div></div><div> > > Thanks again for the time. I know there's a lot of questions above, > but any help of any kind is greatly appreciated. > > All the best, > Tim > > > On Fri, Nov 26, 2010 at 1:04 AM, Kapil Agrawal <<a href="mailto:kapil.agl@gmail.com" target="_blank">kapil.agl@gmail.com</a>> > wrote: > Just a quick clue that might help, try using multifilesrc ? > > > On Thu, Nov 25, 2010 at 9:47 PM, Timothy Braun > <<a href="mailto:braunsquared@gmail.com" target="_blank">braunsquared@gmail.com</a>> wrote: > > > Hello Everyone, > I'm fairly new to G-Streamer so any input you can > provide is much appreciated. I'm working on a project > where we need to generate a 2 minute video which is a > composite of a total of 24 input videos. The output > video will have 4 different 30 second sections, each > containing a 3x2 grid of the smaller input videos. > The input videos are all naturally at 240x240 with the > goal of having a final output frame size of 720x480. > > Using gst-launch, I've been able to construct a > sample 30 second clip using a combination of inputs, > videoboxes and a videomixer. Here is what I've come > up with so far: > > videomixer name=mix ! ffmpegcolorspace ! > ffenc_mpeg1video ! ffmux_mpeg name=mux ! queue ! > filesink location=output.mpg > adder name=adder ! audioconvert ! ffenc_mp2 ! mux. > filesrc location=loop1.mp4 ! decodebin name=decode1 > decode1. ! videobox border-alpha=0 top=-240 left=0 ! > queue ! mix. > decode1. ! adder. > filesrc location=loop2.mp4 ! decodebin name=decode2 > decode2. ! videobox border-alpha=0 top=-240 > left=-240 ! queue ! mix. > decode2. ! adder. > filesrc location=loop3.mp4 ! decodebin name=decode3 > decode3. ! videobox border-alpha=0 top=-240 > left=-480 ! queue ! mix. > decode3. ! adder. > filesrc location=loop4.mp4 ! decodebin name=decode4 > decode4. ! videobox border-alpha=0 top=0 left=0 ! > queue ! mix. > decode4. ! adder. > filesrc location=loop5.mp4 ! decodebin name=decode5 > decode5. ! videobox border-alpha=0 top=0 left=-240 ! > queue ! mix. > decode5. ! adder. > filesrc location=loop6.mp4 ! decodebin name=decode6 > decode6. ! videobox border-alpha=0 top=0 left=-480 ! > queue ! mix. > decode6. ! adder. > > Now I need to do this 4 times, each time with a > potentially different video in each box. I've started > looking into C interfaces as there's other pieces of > the puzzle which need to be tied into this, and I am > trying to determine the best way to tackle this. I > originally was looking at Gnonlin, but the > documentation is lacking in regards to how > gnloperations work. I also recently stumbled upon the > GES library by Edward Hervey, this looks promising as > well, but I haven't been able to spend much time on > it. > > If I go the Gnonlin route, I believe I would need 6 > compositions, one for each box. At the 30 second > marker, I would swap the filesource to a new one using > dynamic pads and listening for messages on the > pipeline bus. Am I far off on this? Any suggestions? > > As for the GES library, it looks very promising and > powerful from the little I read on it. Would this be > the smarter route to take? If so, does anyone have > any suggestions for how the pipeline would be > structured? > > Thank you in advance for your time on this and I > truly appreciate any information you are willing to > share with me. > > Happy Thanksgiving, > Tim > > > ------------------------------------------------------------------------------ > Increase Visibility of Your 3D Game App & Earn a > Chance To Win $500! > Tap into the largest installed PC base & get more eyes > on your game by > optimizing for Intel(R) Graphics Technology. Get > started today with the > Intel(R) Software Partner Program. Five $500 cash > prizes are up for grabs. > <a href="http://p.sf.net/sfu/intelisp-dev2dev" target="_blank">http://p.sf.net/sfu/intelisp-dev2dev</a> > _______________________________________________ > gstreamer-devel mailing list > <a href="mailto:gstreamer-devel@lists.sourceforge.net" target="_blank">gstreamer-devel@lists.sourceforge.net</a> > <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> > > > > > -- > <a href="http://www.mediamagictechnologies.com" target="_blank">www.mediamagictechnologies.com</a> (Gstreamer, ffmpeg, Red5, > Streaming) > twitter handle: @gst_kaps > <a href="http://www.linkedin.com/in/kapilagrawal" target="_blank">http://www.linkedin.com/in/kapilagrawal</a> > > ------------------------------------------------------------------------------ > Increase Visibility of Your 3D Game App & Earn a Chance To Win > $500! > Tap into the largest installed PC base & get more eyes on your > game by > optimizing for Intel(R) Graphics Technology. Get started today > with the > Intel(R) Software Partner Program. Five $500 cash prizes are > up for grabs. > <a href="http://p.sf.net/sfu/intelisp-dev2dev" target="_blank">http://p.sf.net/sfu/intelisp-dev2dev</a> > _______________________________________________ > gstreamer-devel mailing list > <a href="mailto:gstreamer-devel@lists.sourceforge.net" target="_blank">gstreamer-devel@lists.sourceforge.net</a> > <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> > > > ------------------------------------------------------------------------------ > Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! > Tap into the largest installed PC base & get more eyes on your game by > optimizing for Intel(R) Graphics Technology. Get started today with the > Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. > <a href="http://p.sf.net/sfu/intelisp-dev2dev" target="_blank">http://p.sf.net/sfu/intelisp-dev2dev</a> > _______________________________________________ gstreamer-devel mailing list <a href="mailto:gstreamer-devel@lists.sourceforge.net" target="_blank">gstreamer-devel@lists.sourceforge.net</a> <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> ------------------------------------------------------------------------------ Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! Tap into the largest installed PC base & get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. <a href="http://p.sf.net/sfu/intelisp-dev2dev" target="_blank">http://p.sf.net/sfu/intelisp-dev2dev</a> _______________________________________________ gstreamer-devel mailing list <a href="mailto:gstreamer-devel@lists.sourceforge.net" target="_blank">gstreamer-devel@lists.sourceforge.net</a> <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> </div></div></blockquote></div> </div></div></blockquote></div>