[Bug 685282] dvdspu: figure out how to make it work with hardware decoders and subpicture overlays

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=685282

--- Comment #37 from Arnaud Vrac <rawoul at gmail.com> ---
Hi Jan,

Thanks for taking a look. The patches on the github branch are more recent,
mostly fixing issues with PGS rendering introduced by my previous patches. Now
everything works fine, and I also handle the cropping window properly. Since
1.6 has been released, I'm now trying to make the individual patches easier to
review (separating overlay meta negotiation and rendering in separate patches).
I'll try to finish this work tonight.


(In reply to Jan Schmidt from comment #36)
> I think somewhere down the road, it would be nice to have more generic SPU
> interface than sending all these events with application/x-gst-dvd prefix,
> but I'm not sure what that API should be.
> 
> So far these patches look good.
> 
> I'm still a little worried about the performance impact blending an entire
> display-area sized region. My memory is that it was a non-trivial impact in
> 2004-2005 when I wrote the original code. It's something we could optimise
> later though - perhaps keep an auxilliary array of "height" entries that
> stores the X position of the first non-transparent pixel for each line so it
> can skip blending.

I agree, and the blending performance could be better if the overlay format
could be dynamic instead of hardcoding ARGB or AYUV. We would use AYUV when
blending to avoid conversions, and ARGB in other cases since we have no way yet
to negotiate the overlay format with downstream, and most hardware works in
ARGB.

The rendering code does skip transparent pixels so I hope the impact is not too
big. Clearing the full frame should be pretty fast.

> 
> Instead of only scaling down to fit, is it worth considering using the
> set-frame-size event to allocate a larger downstream frame as the target for
> blending into? I think in any case a mechanism for communicating the
> intended display size to the sinks would be useful when using
> GstVideoOverlayComposition.  I have a rip here where the video was
> originally 1080p, and they've trimmed the black-bars to enode 1920x800. When
> the black bars are added back in for display, the PGS titles should appear
> there - starting below Y=870 or so.

I'm not sure what to do here. Right now I have the exact same result as VLC for
vobsub and PGS. mpv renders PGS slightly differently, it doesn't scale them.

> 
> Between dvbsuboverlay, dvdspu and textoverlay, we probably have enough
> experience to start factoring out a set of common overlay/SPU behaviours
> into a base class and have features like negotiation, colour conversion and
> software blending fallbacks in there.

Yes, the overlay meta negotiation is not easy to get right and is a big copy
paste in all the subtitle renderers.

> 
> Besides all that, there is something wrong with these patches, but I don't
> know what yet. On my Back To The Future DVD, they make it not display
> certain subpicture entries in the bonus 'Animated Anecdotes' track, but only
> some.

The issue you face might be related to the overlay cache not being cleared in
some cases, which I might have overlooked. Can you check if reverting the
'dvdspu: cache overlay composition' helps ?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.