[Intel-gfx] [ANNOUNCE][RFC] KVMGT - the implementation of Intel GVT-g(full GPU virtualization) for KVM

[Jike Song]

Hi all,

We are pleased to announce another update of Intel GVT-g for KVM.

Intel GVT-g is a full GPU virtualization solution with mediated pass-through, starting from 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. KVM is supported by Intel GVT-g(a.k.a. KVMGT).


Repositories

     Kernel: https://github.com/01org/igvtg-kernel (2015q3-3.18.0 branch)
     Qemu: https://github.com/01org/igvtg-qemu (kvmgt_public2015q3 branch)


This update consists of:

     - KVMGT is now merged with XenGT in unified repositories(kernel and qemu), but currently
       different branches for qemu.  KVMGT and XenGT share same iGVT-g core logic.
     - PPGTT supported, hence the Windows guest support
     - KVMGT now supports both 4th generation (Haswell) and 5th generation (Broadwell) Intel Core(TM) processors
     - 2D/3D/Media decoding have been validated on Ubuntu 14.04 and Windows7/Windows 8.1

Next update will be around early Jan, 2016.

Known issues:

     - At least 2GB memory is suggested for VM to run most 3D workloads.
     - 3Dmark06 running in Windows VM may have some stability issue.
     - Using VLC to play .ogg file may cause mosaic or slow response.


Please subscribe the mailing list to report BUGs, discuss, and/or contribute:

     https://lists.01.org/mailman/listinfo/igvt-g

More information about Intel GVT-g background, architecture, etc can be found at(may not be up-to-date):

     https://01.org/igvt-g
     http://www.linux-kvm.org/images/f/f3/01x08b-KVMGT-a.pdf
     https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian


Note:

The KVMGT project should be considered a work in progress. As such it is not a complete product nor should it be considered one. Extra care should be taken when testing and configuring a system to use the KVMGT project.


--
Thanks,
Jike

On 12/04/2014 10:24 AM, Jike Song wrote:
> Hi all,
>
>    We are pleased to announce the first release of KVMGT project. KVMGT is the implementation of Intel GVT-g technology, a full GPU virtualization solution. Under Intel GVT-g, a virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance of performance, feature, and sharing capability.
>
>
>    KVMGT is still in the early stage:
>
>     - Basic functions of full GPU virtualization works, guest can see a full-featured vGPU.
>       We ran several 3D workloads such as lightsmark, nexuiz, urbanterror and warsow.
>
>     - Only Linux guest supported so far, and PPGTT must be disabled in guest through a
>       kernel parameter(see README.kvmgt in QEMU).
>
>     - This drop also includes some Xen specific changes, which will be cleaned up later.
>
>     - Our end goal is to upstream both XenGT and KVMGT, which shares ~90% logic for vGPU
>       device model (will be part of i915 driver), with only difference in hypervisor
>       specific services
>
>     - insufficient test coverage, so please bear with stability issues :)
>
>
>
>    There are things need to be improved, esp. the KVM interfacing part:
>
> 	1	a domid was added to each KVMGT guest
>
> 		An ID is needed for foreground OS switching, e.g.
>
> 			# echo <domid>	>	/sys/kernel/vgt/control/foreground_vm
>
> 		domid 0 is reserved for host OS.
>
>
>    	2	SRCU workarounds.
>
> 		Some KVM functions, such as:
>
> 				kvm_io_bus_register_dev
> 				install_new_memslots
>
> 		must be called *without* &kvm->srcu read-locked. Otherwise it hangs.
>
> 		In KVMGT, we need to register an iodev only *after* BAR registers are
> 		written by guest. That means, we already have &kvm->srcu hold -
> 		trapping/emulating PIO(BAR registers) makes us in such a condition.
> 		That will make kvm_io_bus_register_dev hangs.
>
> 		Currently we have to disable rcu_assign_pointer() in such functions.
>
> 		These were dirty workarounds, your suggestions are high welcome!
>
>
> 	3	syscalls were called to access "/dev/mem" from kernel
>
> 		An in-kernel memslot was added for aperture, but using syscalls like
> 		open and mmap to open and access the character device "/dev/mem",
> 		for pass-through.
>
>
>
>
> The source codes(kernel, qemu as well as seabios) are available at github:
>
> 	git://github.com/01org/KVMGT-kernel
> 	git://github.com/01org/KVMGT-qemu
> 	git://github.com/01org/KVMGT-seabios
>
> In the KVMGT-qemu repository, there is a "README.kvmgt" to be referred.
>
>
>
> More information about Intel GVT-g and KVMGT can be found at:
>
> 	https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian
> 	http://events.linuxfoundation.org/sites/events/files/slides/KVMGT-a%20Full%20GPU%20Virtualization%20Solution_1.pdf
>
>
> Appreciate your comments, BUG reports, and contributions!
>
>
>
>
> --
> Thanks,
> Jike
>