[gst-devel] [RFC] Encoding and Profiles

[Robert Swain]

Hello,

2009/10/19 Edward Hervey <edward.hervey at collabora.co.uk>:
>  I have been working lately on researching ways to make the whole
> encoding experience better and more streamlined for applications using
> GStreamer, and have come up with a proposal.
>
>  You will find the proposal below, and attached to the mail the
> research and proposed C API.
>
>  Comments and feedback are most welcome

Great! Looks good. :)

A few notes:

- It's not clear if it's possible to derive target profiles from other
but it seems like it's been considered. My thought is that encoder
quality presets are usually orthogonal to profile/level and
device-specific restrictions. I think something like the following
might be useful:

* Quality-related
** Hypothetically, say we have qth264enc and x264enc. They can have
presets created for options which affect the encoding time/quality.
** Quality profiles can use these presets and add to them suggestions
for the available rate control methods (e.g. quantiser, bit rate or
so)

* Restriction-related
** Profile/level restrictions
** Device-specific restrictions

My point is that codec elements always have rather specific
configuration options and it would be good to maintain some kind of
range of quality option presets for each encoder that trade off speed
and quality for a given bit rate. If these options are included in
each target preset it will greatly increase maintenance and it's a big
enough job as it is. The whole set up lends itself well to this as
target devices could just override options as necessary.

- System-wide versus application-specific versus user-specific
profiles can be manageable through the API without too much difficulty
I think. If scope is added in the API for providing a path from which
one can load a profile then applications can use their own profiles.
Similarly if scope is added for creating non-stored profiles in memory
and just passing a pointer, this allows users/application developers
to create profiles however they like.

- To manage stream copying it should be simple enough to probe the
caps of the target profile to check if an input stream is supported
and if so flag somehow that it can be copied so that this can be
checked and specified whether to copy or transcode in the application.

- I think the API should allow an application to probe the profile to
find what ranges are supported and allow customisation within those
ranges.

Michael's use case about the device being able to play up to 720p but
to save space on the low capacity storage device he wants to encode at
a lower resolution/bit rate is sound.

Also, one may have a device that can play a higher resolution and has
some video output to hook up to a higher resolution display device,
but the playback device itself only has a small display. If one never
uses the video output functionality, one might want to restrict the
resolution.

Similarly one might want to downmix audio or so even though 5.1 is
supported e.g. a PS3 hooked up to a stereo amplifier.


In short: target profiles should specify the range of operation of a
target device, essentially being a device caps specification plus any
necessary encoder/muxer options (e.g. setting 0 b-frames or the mux
rate) to make things work on the device. Encoder element presets
should consider an unrestricted playback service and be focused on
speed/quality trade-off.


- Finally with regard to Michael's concerns about tying a target
profile to specific elements - how about one being able to specify
multiple possible elements for encode/mux a particular mime-type and
having some hierarchy for selection based on which is deemed best like
the rest of the GStreamer system works (I think... I'm new here. :))

Best regards,
Rob