[RFC] wl_surface video protocol extension

[James Courtier-Dutton]

On 21 October 2013 20:45, Daniel Stone <daniel at fooishbar.org> wrote:
> Hi,
>
> On 18 October 2013 11:52, James Courtier-Dutton <james.dutton at gmail.com> wrote:
>>
>> I think how wayland can help provide the "deterministic" angle is not yet
>> clear to me.
>
> Nothing is deterministic.  Is your kernel scheduling your application
> in time? And the compositor? Is your application actually in memory,
> or are you frantically paging everything in? Has someone just blown
> out the cache? Is your data actually streaming OK, or are you waiting
> on I/O? Has someone just submitted a huge job to the GPU (which can be
> as simple as texture setup) which is going to grief all its load? Did
> a notification with an overlay just trigger and demote you from an
> overlay to the GPU? Did the display controller hiccup for a frame
> during a voltage/clock change? Did your constant frame submission
> earlier run the system so hot it's throttled its clock down so far you
> can no longer meet your frame deadlines?
>
> There's no plan to provide an absolute hard ironclad guarantee on
> scheduling, because all of these things are out of our control, and
> most of them are fundamentally unsolvable.  What we're suggesting is a
> two-part solution: before submission, allow applications to queue
> (multiple) frames along with a list of target times which the
> compositor will attempt (not guarantee) to meet; after submission,
> tell the application when the frame was actually displayed, and allow
> them to adjust their submission times accordingly.
>
> Cheers,
> Daniel

Thank you for that clarification.
We have deterministic features for Audio output. It is probably easier
for Audio processing because that hardware and audio DSPs can have
fixed delay pipelines.
You have made it clear that this is not possible for GPUs.
I suggested what would be the best situation to have, we don't have it
and apparently cannot.
So, the  "requested timestamp" and "actually presented timestamp" is
currently the best solution.
This will be OK. It lets the APP play with 3 data items:
1) When the APP submitted the frame
2) The requested timestamp the APP put on the frame.
3) The actually presented timestamp sent to the APP from the GPU.

Maybe an option to have other GPU processing suppressed while
displaying video full screen might help to make things a little less
variable.

Kind Regards
James