[PATCH] drm/amdgpu: fix modprobe failure of the 2nd GPU when GDDR6 training enabled

[Alex Deucher]

On Wed, Jan 8, 2020 at 7:52 AM Christian König <christian.koenig at amd.com> wrote:
>
> Am 08.01.20 um 13:36 schrieb Tianci Yin:
> > From: "Tianci.Yin" <tianci.yin at amd.com>
> >
> > [why]
> > In dual GPUs scenario, stolen_size is assigned to zero on the 2nd GPU,
> > then the bottom region of VRAM was allocated as GTT, unfortunately
> > a small region of bottom VRAM was encroached by UMC firmware during
> > GDDR6 BIST training, this cause pagefault.
>
> What I'm missing here is why is the stolen size zero on the 2nd GPU?
>
> Maybe we need to read the stolen size after posting the GPU instead?

There is no stolen memory on secondary GPUs because there is no pre-OS
console using that memory.

Alex

>
> Regards,
> Christian.
>
> >
> > [how]
> > Forcing stolen_size to 3MB, then the bottom region of VRAM was
> > allocated as stolen memory, GTT corruption avoid.
> > The stolen memory of the 2nd GPU will be free in late_init phase,
> > no memory wasted.
> >
> > Change-Id: Icd0ad7de41333282949bb1e3e676c6c307ddd081
> > Signed-off-by: Tianci.Yin <tianci.yin at amd.com>
> > ---
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |  6 ++++++
> >   drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c  | 21 +++++++++++++++++++++
> >   2 files changed, 27 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> > index c91dd602d5f1..440b793316df 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> > @@ -60,6 +60,11 @@
> >    */
> >   #define AMDGPU_GMC_FAULT_TIMEOUT    5000ULL
> >
> > +/*
> > + * Default stolen memory size, 1024 * 768 * 4
> > + */
> > +#define AMDGPU_STOLEN_VGA_DEFAULT_SIZE       0x300000
> > +
> >   struct firmware;
> >
> >   /*
> > @@ -192,6 +197,7 @@ struct amdgpu_gmc {
> >       uint32_t                srbm_soft_reset;
> >       bool                    prt_warning;
> >       uint64_t                stolen_size;
> > +     bool                    stolen_temp_reserved;
> >       /* apertures */
> >       u64                     shared_aperture_start;
> >       u64                     shared_aperture_end;
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> > index 7dc8c068c62a..0c96b67d6ca7 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> > @@ -566,6 +566,11 @@ static int gmc_v10_0_late_init(void *handle)
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >       int r;
> >
> > +     if (adev->gmc.stolen_temp_reserved) {
> > +             amdgpu_bo_late_init(adev);
> > +             adev->gmc.stolen_temp_reserved = false;
> > +     }
> > +
> >       r = amdgpu_gmc_allocate_vm_inv_eng(adev);
> >       if (r)
> >               return r;
> > @@ -756,6 +761,22 @@ static int gmc_v10_0_sw_init(void *handle)
> >               return r;
> >
> >       adev->gmc.stolen_size = gmc_v10_0_get_vbios_fb_size(adev);
> > +     /*
> > +      * In dual GPUs scenario, stolen_size is assigned to zero on the 2nd GPU,
> > +      * then the bottom region of VRAM was allocated as GTT, unfortunately
> > +      * a small region of bottom VRAM was encroached by UMC firmware during
> > +      * GDDR6 BIST training, this cause pagefault.
> > +      * The page fault can be fixed by forcing stolen_size to 3MB, then the bottom
> > +      * region of VRAM was allocated as stolen memory, GTT corruption avoid.
> > +      * The stolen memory of the 2nd GPU will be free in late_init phase,
> > +      * no memory wasted.
> > +      */
> > +     if (adev->fw_vram_usage.mem_train_support &&
> > +             adev->gmc.stolen_size == 0) {
> > +             adev->gmc.stolen_size = AMDGPU_STOLEN_VGA_DEFAULT_SIZE;
> > +             adev->gmc.stolen_temp_reserved = true;
> > +     } else
> > +             adev->gmc.stolen_temp_reserved = false;
> >
> >       /* Memory manager */
> >       r = amdgpu_bo_init(adev);
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx