[PATCH v2 0/7] Host1x IOMMU support + VIC support

[Mikko Perttunen]

On 14.12.2016 15:05, Daniel Vetter wrote:
> On Wed, Dec 14, 2016 at 02:41:28PM +0200, Mikko Perttunen wrote:
>> On 14.12.2016 14:30, Daniel Vetter wrote:
>>> On Wed, Dec 14, 2016 at 01:16:10PM +0200, Mikko Perttunen wrote:
>>>> This series adds IOMMU support to Host1x and TegraDRM
>>>> and adds support for the VIC (Video Image Compositor)
>>>> host1x client. The series is available as a git repository at
>>>> git://github.com/cyndis/linux.git; branch vic-2.
>>>>
>>>> A userspace test case for VIC can be found at
>>>> https://github.com/cyndis/drm/tree/work/tegra.
>>>> The testcase is in tests/tegra and is called submit_vic.
>>>> The testcase/TRM include full headers and documentation
>>>> to program the unit. The unit by itself, however, does not
>>>> readily map to existing userspace library interfaces, so
>>>> implementations for those are not provided.
>>>
>>> Afaik libva has an entire pile of post-processing support. Pretty sure
>>> other video transcode libraries have similar interfaces, so should all be
>>> possible to implement this.
>>
>> We don't have any actual video transcoding support though, so unless it's
>> possible to just implement a part of libva and defer the rest to some CPU
>> implementation, I don't see how this is useful. I suppose I could implement
>> a GStreamer plugin for colorspace conversion or resizing, since those are
>> very modular.
>
> Hm, I guess the question then is, how did that get enabled?

What is "that"? I'm not exactly sure.

Our architecture is such that there's the VIC that handles colorspace 
conversion, rescaling, blitting and can do some 2d postprocessing 
effects as well.

Then there's the separate NVDEC that is a video bitstream decoder. 
There's no support for that at the moment. I am working on the IP side 
of that.

The video processing pipeline is then such that NVDEC is fed the 
bitstream; NVDEC outputs a YUV picture in a specific format; VIC takes 
that YUV picture and converts/rescales it into the desired format. Or if 
we are encoding video, VIC takes your RGB image, converts it into a 
format that NVENC understands, and so on.

So with just VIC support, I could implement some simple 2D things. I 
don't know if anyone would want to specifically use the VIC for those 
since applications already have fast CPU algorithms. For the video 
pipeline using VIC is nice since these units can synchronize work 
without CPU involvement and when you're already using NVDEC or NVENC 
it's barely any extra effort to involve VIC as well. It can also be 
useful in power usage sensitive situations, but we aren't really fit for 
those situations with the upstream kernel anyway :)

Thanks;
Mikko.

>
>>> Until that exists I really think that the VIC part (and only that, since
>>> tk1/tx1 in general seems to work with nouveau and all that) should stay
>>> out of tree.
>>
>> Ok. I'll try to whip up something.
>
> But yeah some gstreamer thing might be doable, but not sure they take that
> (I don't have any contacts there). Video is a bit a mess, there's no clear
> go-to place for dropping some new hw support.
> -Daniel
>