[Nouveau] Documentation request for MP warp error 0x10

[Ilia Mirkin]

On Fri, Nov 6, 2015 at 3:59 PM, Robert Morell <rmorell at nvidia.com> wrote:
> On Fri, Oct 02, 2015 at 06:05:21PM -0400, Ilia Mirkin wrote:
>> Could you advise what the proper way of indicating
>> that the memory is "global" to the op? I'm sure I'm just missing
>> something simple. If you show me what to look for in SM35 I can
>> probably find it on my own for SM20/SM30/SM50.
>
> Sorry again for the delay.  Here's what I've been able to find out about
> the generic thread address space (used by the SMs) and what types of
> memory it contains.  Hopefully this clears things up.
>
>
> Local memory is a per-thread space.
> Shared memory is a per-CTA space (compute shaders only).
>
> LDL and STL instructions access local memory with a zero offset.
> LDS, LSDLK, STS, and STSCUL instructions access shared memory with a zero
> offset.
>
> LD, ST, RED, ATOM, and CCTL.D instructions access the generic thread address
> space, which is layered on top of the channel's virtual address space.
>
> In the generic thread address space, there are 16MB windows into local and
> shared memory; everything not in a Local or Shared address window accesses
> global virtual memory.
>
> The local window offset within the generic thread address space is determined
> by the SetShaderLocalMemoryWindow class method (offset 0x77c in classes *97 and
> *c0).
>
> The shared window offset within the generic thread address space is determined
> by the SetShaderSharedMemoryWindow class method (offset 0x214 in classes *c0).
>
> For both methods, the offset is in bytes, but the window must be aligned to a
> 16MB boundary (so the lower 24 bits of the data must be zero). The upper 32
> bits of the windows are hard-coded to 0 (so they must be placed within the
> lower 4GB of address space).
>
> Generally, it is expected that software will reserve ranges in the global
> virtual address space where these windows will be placed. (Otherwise anything
> mapped there will be inaccessible to shaders.)
>
> For graphics shaders, the shared address space logic does not exist, so there
> is no need to reserve virtual memory for it.

Hi Robert, thanks so much for getting back to me. I believe I've
understood what you've said, but please confirm:

In order for ATOM.*/RED.* to work, the addresses in question must
*NOT* be inside of the 16MB local/shared windows. So if I'm getting
that error, the address must be inside.

If so, this may be a reasonable explanation for what I'm seeing --
while I knew about the local/shared windows, I didn't realize that the
windows were 16MB-sized.

Thanks again,

  -ilia