[Spice-devel] [virtio-dev] [PATCH RFC v4 0/1] Virtio Video Device Specification

[Keiichi Watanabe]

On Fri, Jan 15, 2021 at 3:18 AM Alex Bennée <alex.bennee at linaro.org> wrote:

>
> Keiichi Watanabe <keiichiw at chromium.org> writes:
>
> > This is the v4 RFC of virtio video device specification.
> > PDF versions are available at [1, 2].
> >
> > Note that this patch depends on a recent patchset "Cross-device resource
> > sharing" [3].
> >
> > Here is a list of major changes from v3:
> > * Redesingned struct definitions for each command and request based on
> >   discussions at [4].
> > * Renamed commands/structs/enums to more descriptive names.
> > * Had different structs and fields for image formats and bitstream
> formats.
> > * Added more detailed specification for supported video codecs.
> > * Made stream_id be allocated by the device.
> > * Had a single parameter struct per-stream instead of per-queue
> parameters and
> >   controls.
> > * Allowed the driver to specify the number of buffers to use via
> >   "cur_{image,bitstream}_buffers".
> > * Renamed RESOURCE_CREATE command to RESOURCE_ATTACH command and allow
> the
> >   driver to use this command when replacing backing memories as well.
> >
> > [5] is the diff of the header file from v3. Note that it only contains
> changes
> > in the header. We haven't updated the driver nor device implementation
> to focus
> > on protocol design discussion first.
> >
> > While it may appear that many parts have been changed since the previous
> > revision, these changes are to address the issues raised in previous
> discussions
> > or/and to make the protocol simpler and easier to prevent misuse.
> > I'd appreciate any types of feedback.
> >
> > Best regards,
> > Keiichi
> >
> > [1] (full):
> https://drive.google.com/file/d/1DiOJZfUJ5wvFtnNFQicxt0zkp4Ob1o9C/view?usp=sharing
> > [2] (only video section):
> https://drive.google.com/file/d/188uAkIWE0BsXETECez98y5fJKw8rslj3/view?usp=sharing
> > [3]
> https://lists.oasis-open.org/archives/virtio-comment/202003/msg00035.html
> > [4] https://markmail.org/thread/c6h3e3zn647qli3w
> > [5]
> >
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2164411
>
> Hi Keiichi,
>
> I wanted to ask what the current status of this spec was. Are you
> planning to submit a new revision of the specification due or are things
> fairly stable now?
>
> We are starting to think about next steps for virtualised video as part
> of Linaro's Stratos work. Specifically we are thinking about
> implementing backends and getting a stack up and running which we can
> use to experiment with multiple hypervisors and VM deployment
> approaches.
>
> Longer term goals included looking at how to integrate virtio-video with
> a secure world on ARM (e.g. feed video data to a secure world device for
> playback via virtio). As part of that we will also be looking at how to
> minimise the memory profile of the backend to do this.
>
> Looking at the virtio-spec repo it looks like the cross-device resource
> sharing is now merged:
>
>   87fa6b5 * virtio-gpu: add support for mapping/unmapping blob resources
>   89e7eb5 * virtio-gpu: add resource create blob
>   162578b * virtio-gpu: add the ability to export resources
>   68f66ff * content: define what an exported object is
>
> are there any other prerequisites?
>
> From a backend implementation point of view it makes sense to wait until
> there is a working frontend driver up-streamed into the kernel. I guess
> that is blocked on the final call for vote on the virtio spec?
>
> I'm sure there is scope for parallelism here but I wanted to get a sense
> of the current direction before embarking on work that would require a
> big re-write down the line.
>

Hi Alex,

Thanks for reaching out to me. I'm really happy to hear your team is
looking at virtio-video.

For the specification, we've been preparing the v5 spec by addressing
previous review comments and simplifying rules. I think we'll be able to
share it for reviewing soon. It won't require any additional mechanism
other than the resource sharing feature.

For the implementation, unfortunately, we haven't started implementing a
driver using the v5 protocol yet.
While we have the driver and device using the v3 protocol in Chrome OS
repository, some efforts are required to update them to support the v5
protocol because there are some gaps between the v3 protocol and the v4/v5
protocol.
I think the driver implementation is necessary for the spec to be merged,
but it's not yet clear when we can spend time implementing drivers. It's
likely to be after April or so.

IIRC, OpenSynergy folks, who implemented the v3 driver, also had some plan
to implement the driver with the v5 spec.
Matti, do you have any update on it? I'd really appreciate it if we could
keep working for upstream together.

Regards,
Keiichi




>
> Regards,
>
> --
> Alex Bennée
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/spice-devel/attachments/20210115/3d7f55c8/attachment.htm>