Why is Thunderbolt 3 limited to 2.5 GT/s on Linux?

[Timur Kristóf]

> > Thanks Marek, I didn't know about that option.
> > Tried it, here is the output: https://pastebin.com/raw/9SAAbbAA
> > 
> > I'm not quite sure how to interpret the numbers, they are
> > inconsistent
> > with the results from both pcie_bw and amdgpu.benchmark, for
> > example
> > GTT->VRAM at a 128 KB is around 1400 MB/s (I assume that is
> > megabytes /
> > sec, right?).
> 
> Based on the SDMA results, you have 2.4 GB/s. For 128KB, it's 2.2
> GB/s for GTT->VRAM copies.
> 

In the meantime I had a chat with Michel on IRC and he suggested that
maybe amdgpu.benchmark=3 gives lower results because it uses a less
than optimal way to do the benchmark.

Looking at the results from the mesa benchmark a bit more closely, I
see that the SDMA can do:
VRAM->GTT: 3087 MB/s = 24 Gbit/sec
GTT->VRAM: 2433 MB/s = 19 Gbit/sec

So on Polaris at least, the SDMA is the fastest, and the other transfer
methods can't match it. I also did the same test on Navi, where it's
different: all other transfer methods are much closer to the SDMA, but
the max speed is still around 20-24 Gbit / sec.

I still have a few questions:

1. Why is the GTT->VRAM copy so much slower than the VRAM->GTT copy?

2. Why is the bus limited to 24 Gbit/sec? I would expect the
Thunderbolt port to give me at least 32 Gbit/sec for PCIe traffic.
Mika, do you maybe have an explanation?

Thanks & best regards,
Tim