[PATCH] nouveau/dmem: handle kcalloc() allocation failure

[duoming at zju.edu.cn]

On Mon, 04 Mar 2024 14:14:52 +0200 Jani Nikula wrote:
> >> The kcalloc() in nouveau_dmem_evict_chunk() will return null if
> >> the physical memory has run out. As a result, if we dereference
> >> src_pfns, dst_pfns or dma_addrs, the null pointer dereference bugs
> >> will happen.
> >>
> >> This patch uses stack variables to replace the kcalloc().
> >
> > Won't this blow the stack?  And why not just test the return value of
> > kcalloc?
> 
> VLAs should not be used in the kernel anymore. Building this results in
> a warning due to -Wvla. See 0bb95f80a38f ("Makefile: Globally enable VLA
> warning").
> 
> Error checking and propagation is the way to go.

The GPU is going away. If the kcalloc() in nouveau_dmem_evict_chunk() fail,
we could not evict all pages mapping a chunk. Do you think we should add a
__GFP_NOFAIL flag in kcalloc()? I see the __GFP_NOFAIL flag is used in the
following code:

/*
* _GFP_NOFAIL because the GPU is going away and there
* is nothing sensible we can do if we can't copy the
* data back.
*/
dpage = alloc_page(GFP_HIGHUSER | __GFP_NOFAIL);

Best regards,
Duoming Zhou