[gst-devel] [RFC] Encoding and Profiles

[Edward Hervey]

On Tue, 2009-10-20 at 13:12 +0100, Robert Swain wrote:
> Hello,
> 
> 2009/10/19 Edward Hervey <edward.hervey at collabora.co.uk>:
> >  I have been working lately on researching ways to make the whole
> > encoding experience better and more streamlined for applications using
> > GStreamer, and have come up with a proposal.
> >
> >  You will find the proposal below, and attached to the mail the
> > research and proposed C API.
> >
> >  Comments and feedback are most welcome
> 
> Great! Looks good. :)
> 
> A few notes:
> 
> - It's not clear if it's possible to derive target profiles from other
> but it seems like it's been considered.

  I'm not 100% certain about this. The main issue that comes to mind is
that if we start making a hierarchy of profiles, we'll have to make sure
that any change in a parent profile doesn't have any ill-effect on any
of the sub-profiles... including those you don't control.

>  My thought is that encoder
> quality presets are usually orthogonal to profile/level and
> device-specific restrictions. I think something like the following
> might be useful:
> 
> * Quality-related
> ** Hypothetically, say we have qth264enc and x264enc. They can have
> presets created for options which affect the encoding time/quality.
> ** Quality profiles can use these presets and add to them suggestions
> for the available rate control methods (e.g. quantiser, bit rate or
> so)

  I don't really understand this part :(

> 
> * Restriction-related
> ** Profile/level restrictions
> ** Device-specific restrictions
> 
> My point is that codec elements always have rather specific
> configuration options and it would be good to maintain some kind of
> range of quality option presets for each encoder that trade off speed
> and quality for a given bit rate. If these options are included in
> each target preset it will greatly increase maintenance and it's a big
> enough job as it is. The whole set up lends itself well to this as
> target devices could just override options as necessary.

  Profile/Level are clear restrictions. If you choose those, you're
already limiting the rest of the choices (1)

  Then amongst the remaining choices I can only think three ways of
going:
  * Expose all remaining element-specific options... 
  * Having a Quality/Speed setting ranging from Low-Quality/Fast to
HighestQuality/Sloww
  * A mix of the two :(

(1): that reminds me... if you select a certain profile that limits the
range of some option... how do we report that ?

> 
> - System-wide versus application-specific versus user-specific
> profiles can be manageable through the API without too much difficulty
> I think. If scope is added in the API for providing a path from which
> one can load a profile then applications can use their own profiles.
> Similarly if scope is added for creating non-stored profiles in memory
> and just passing a pointer, this allows users/application developers
> to create profiles however they like.

  As stated before, all profiles will be available a C
structures/objects. Different backends can be written to support various
storage formats.

> 
> - To manage stream copying it should be simple enough to probe the
> caps of the target profile to check if an input stream is supported
> and if so flag somehow that it can be copied so that this can be
> checked and specified whether to copy or transcode in the application.

  For that you would need to know the available streams in the file you
wish to convert. I'm working on that part, but should go along with
this.

> 
> - I think the API should allow an application to probe the profile to
> find what ranges are supported and allow customisation within those
> ranges.

  Ranges of what ?

> 
> Michael's use case about the device being able to play up to 720p but
> to save space on the low capacity storage device he wants to encode at
> a lower resolution/bit rate is sound.

  Agreed.

> 
> Also, one may have a device that can play a higher resolution and has
> some video output to hook up to a higher resolution display device,
> but the playback device itself only has a small display. If one never
> uses the video output functionality, one might want to restrict the
> resolution.

  That could just be another target.
  Ex : "N1234/H264" and "N1234/H264 External viewing"

> 
> Similarly one might want to downmix audio or so even though 5.1 is
> supported e.g. a PS3 hooked up to a stereo amplifier.

  Can be overridden in the profile once you've loaded it.

> 
> 
> In short: target profiles should specify the range of operation of a
> target device, essentially being a device caps specification plus any
> necessary encoder/muxer options (e.g. setting 0 b-frames or the mux
> rate) to make things work on the device. Encoder element presets
> should consider an unrestricted playback service and be focused on
> speed/quality trade-off.
> 
> 
> - Finally with regard to Michael's concerns about tying a target
> profile to specific elements - how about one being able to specify
> multiple possible elements for encode/mux a particular mime-type and
> having some hierarchy for selection based on which is deemed best like
> the rest of the GStreamer system works (I think... I'm new here. :))

  That's why I'm only specifying GstCaps in the profile. I'm figuring
out some extra signals on EncodeBin so that one can override/reorder the
selected muxers/encoders.

     Edward

> 
> Best regards,
> Rob
> 
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay 
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel