[RFC]: shmem fd for non-DMA buffer sharing cross drivers

[Hsia-Jun Li]

On 8/23/23 03:55, Nicolas Dufresne wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> Hi,
> 
> Le mardi 22 août 2023 à 19:14 +0800, Hsia-Jun Li a écrit :
>> Hello
>>
>> I would like to introduce a usage of SHMEM slimier to DMA-buf, the major
>> purpose of that is sharing metadata or just a pure container for cross
>> drivers.
>>
>> We need to exchange some sort of metadata between drivers, likes dynamic
>> HDR data between video4linux2 and DRM. Or the graphics frame buffer is
>> too complex to be described with plain plane's DMA-buf fd.
>> An issue between DRM and V4L2 is that DRM could only support 4 planes
>> while it is 8 for V4L2. It would be pretty hard for DRM to expend its
>> interface to support that 4 more planes which would lead to revision of
>> many standard likes Vulkan, EGL.
>>
>> Also, there is no reason to consume a device's memory for the content
>> that device can't read it, or wasting an entry of IOMMU for such data.
>> Usually, such a metadata would be the value should be written to a
>> hardware's registers, a 4KiB page would be 1024 items of 32 bits registers.
>>
>> Still, I have some problems with SHMEM:
>> 1. I don't want thhe userspace modify the context of the SHMEM allocated
>> by the kernel, is there a way to do so?
>> 2. Should I create a helper function for installing the SHMEM file as a fd?
> 
> Please have a look at memfd and the seal feature, it does cover the reason why

That is the implement I need, it would affact the userspace not the 
kernel space. Should I expand a kAPI for memfd or just take the 
implement for the SHMEM?
This interfaces need to offer three things:
1. a fd for userspace to exchange between drivers
2. a kernel virtual address for accessing
3. userspace SEAL

Meanwhile, I am thinking whether we should offer a generic context 
header for such usage. Or we need another fields in a driver to describe it.
struct shmem_generic_container {
	u64 format; /* use DRM modifier vendor bits but */
	u32 size; /* size of the payload */
	u8 payload[];
};
/* format linear for nesting dolls context */
struct shmem_nesting_container {
	u32 num;
	u64 formats[num];
	u32 sizes[num];
	u32 offsets[num]; /* offset from the payload below */
	u8 payload[];
};
> unsealed shared memory require full trust. For controls, the SEAL_WRITE is even
> needed, as with appropriate timing, a malicous process can modify the data in-
> between validation and allocation, causing possible memory overflow.
> 
> https://man7.org/linux/man-pages/man2/memfd_create.2.html
> File sealing
>         In the absence of file sealing, processes that communicate via
>         shared memory must either trust each other, or take measures to
>         deal with the possibility that an untrusted peer may manipulate
>         the shared memory region in problematic ways.  For example, an
>         untrusted peer might modify the contents of the shared memory at
>         any time, or shrink the shared memory region.  The former
>         possibility leaves the local process vulnerable to time-of-check-
>         to-time-of-use race conditions (typically dealt with by copying
>         data from the shared memory region before checking and using it).
>         The latter possibility leaves the local process vulnerable to
>         SIGBUS signals when an attempt is made to access a now-
>         nonexistent location in the shared memory region.  (Dealing with
>         this possibility necessitates the use of a handler for the SIGBUS
>         signal.)
> 
>         Dealing with untrusted peers imposes extra complexity on code
>         that employs shared memory.  Memory sealing enables that extra
>         complexity to be eliminated, by allowing a process to operate
>         secure in the knowledge that its peer can't modify the shared
>         memory in an undesired fashion.
> 
>         [...]
> 
> regards,
> Nicolas

-- 
Hsia-Jun(Randy) Li