[Mesa-dev] GSoC : Video decoding state tracker for Gallium3d

[Christian König]

Hi,

> Hi everyone,
> My name is Emeric, I am a?22 years old?french student, and I am
> currently looking to apply to the google summer of code 2011.
> I saw the "Gallium H.264 decoding" idea on the X.Org GSoC page, and I
> am particularly interested by this project.
> 
> The idea would be to use shaders to do some work, in order to write a
> generic video decoding solution, targeting r300g, r600g, nouveau, and
> basically all Gallium3d drivers.
> The project would be to write?a state tracker wich expose some of the
> most shaders-friendly decoding operations (like motion compensation,
> idct, intra-predictions, deblocking filter and maybe vlc decoding)
> through a common API like VDPAU or VA-API.
> These APIs can be used to decode mpeg2, mpeg 4 asp/avc, vc1 and
> others, but at first I intend to focus on the h264 decoding to save
> time, because I know it better and it is currently widely in use, but
> again the goal of the project is to be generic.
> 
> * I am well familiar with the h264 specification and implementation, I
> can understand the amount of work required on that side.
> * I have some understanding of how a graphic card works, what could be
> and what should not be done with it, but I am not used to write
> shaders programs. But hey, this is the fun part, and I am?definitively
> here to learn.
> * As I see it there is no easy or magical gains when using shaders to
> compute stuff like motion compensation or idct, but there are probably
> some things that can be done to smooth over the decoding of high
> bit-rates?videos.
> Beside with the emergence of h265?in the next years, it seems that
> there will be more and more computational power involved (bigger
> transform size, much more intra and inter prediction schemes, insane
> resolutions, ...), so I think spending some time preparing to that
> could be a good idea.

Sounds like I finally get some more help with the pipe-video branch
here. Having a good idea how h264 works is a good start, but keep in
mind that this is quite a bunch of work todo.

Now to your questions:
> - First of all, does this project seems to be a good idea, would you
> be interested in it ?
Yes, I think it's quite a good idea to have an open source
implementation of the different video APIs as state trackers in mesa.
Beside the obviously value of the shader based decoding the mesa
infrastructure seems to be the right place to even support the different
hardware decoders.

> - As the time to realize this project will be quite limited, what
> would be the tasks to achieve in priority ?
It generally seems to be a good Idea to code the pipeline in reverse
order, e.g. start with a pure CPU based implementation and then
implement the step nearest to displaying a frame (usually MC), then the
next stage and so on.

> - What API would be better suited for the state tracker, vaapi or
vdpau ?
Actually I don't really know, vdpau and vaapi doesn't seems to have so
much differences. I'm currently in the process to clean the stuff up and
get at least the presentation part of vdpau working, so I would suggest
to start there.

> - I understand my entry point into this project should be the?g3dvl
> state tracker and the mesa pipe-video branch, I'd like to learn more
> about the state of these work.
A good start would be to checkout and compile the stuff and get a
working development environment for testing up and running.
Then start reading the code and get familiar with the coding style and
the different ideas behind it. If you have any questions about g3dvl
feel free to ask me.

> - Should I cross-post this on xorg or xorg-devel maillist ?
No, I don't think that's necessary. X is just used for outputting the
final rendering result these days (hopefully the X devs won't hurt me to
hard for this).

> Thanks for reading me, and I want to apologies if some parts of this
> mail are hard to understand, I am not an english native speaker,
> mistakes may have been made :-)
> If there is anything that I forgot or that I should detailed do not
> hesitate to tell me.
I'm not a native speaker myself, and to be honest my English speaking
ability could also need some improvements, but I think that English is
one of the easiest languages to learn and most of the computer specific
vocabulary is based on English any way.

Regards,
Christian.