[PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

Hi Jim, sorry I'm a little late to the party, but was on vacation.

On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> <natechancellor at gmail.com> wrote:
> > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > 
> > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > [snip]
> > > > > Hello Nathan,
> > > > > 
> > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > 
> > > > This is the 4GB version.
> > > > 
> > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > there?
> > > > 
> > > > Unfortunately, I am not familiar with how to get this information. If
> > > > you could provide some instructions for how to do so, I am more than
> > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > cause problems on real devices.
> > > 
> > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > you run the following:
> > > 
> > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > 
> > Sure, the result is attached.
> > 
> > > or cat /sys/firmware/fdt > device.dtb
> > > 
> > > and attach the resulting file?
> > > 
> > > > > Finally, can you attach the text of the full boot log?
> > > > 
> > > > I have attached a working and broken boot log. Thank you for the quick
> > > > response!
> > > 
> > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > chance?
> > 
> > Of course. A new log is attached with the debug output from that config.
> 
> Hi Nicolas,
> 
> Can you please help us out here?  It appears that your commit

It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
dma regions and, if no region matches the phys address range, it returns 0.
This isn't acceptable as is. In the past, we used to pass the offset
nonetheless, which would make the phys address grow out of the dma memory area
and fail the dma_capable() test.

For example, RPi4 devices attached to the old interconnect see phys [0x0
0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
(since your dma range only covers the first GB) to then test if 0xfa000000 is
dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
further down the line.

I don't have a proper suggestion on how to solve this but here's the solution I
used:

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afe..40fe3c97f2be 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
 {
        const struct bus_dma_region *m = dev->dma_range_map;
 
-       if (m)
+       if (m) {
                for (; m->size; m++)
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
+
+               /*
+                * The physical address doesn't fit any of the DMA regions,
+                * return an impossible to fulfill offset.
+                */
+               return -(1ULL << 46);
+       }
+
        return 0;
 }

I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
to 30 bit).

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/dri-devel/attachments/20200907/421d9c4e/attachment-0001.sig>