Why is Thunderbolt 3 limited to 2.5 GT/s on Linux?

[Michel Dänzer]

On 2019-07-01 6:01 p.m., Timur Kristóf wrote:
> On Mon, 2019-07-01 at 16:54 +0200, Michel Dänzer wrote:
>> On 2019-06-28 2:21 p.m., Timur Kristóf wrote:
>>> I haven't found a good way to measure the maximum PCIe throughput
>>> between the CPU and GPU,
>>
>> amdgpu.benchmark=3
>>
>> on the kernel command line will measure throughput for various
>> transfer
>> sizes during driver initialization.
> 
> Thanks, I will definitely try that.
> Is this the only way to do this, or is there a way to benchmark it
> after it already booted?

The former. At least in theory, it's possible to unload the amdgpu
module while nothing is using it, then load it again.


>>> but I did take a look at AMD's sysfs interface at
>>> /sys/class/drm/card1/device/pcie_bw which while running the
>>> bottlenecked
>>> game. The highest throughput I saw there was only 2.43 Gbit /sec.
>>
>> PCIe bandwidth generally isn't a bottleneck for games, since they
>> don't
>> constantly transfer large data volumes across PCIe, but store them in
>> the GPU's local VRAM, which is connected at much higher bandwidth.
> 
> There are reasons why I think the problem is the bandwidth:
> 1. The same issues don't happen when the GPU is not used with a TB3
> enclosure.
> 2. In case of radeonsi, the problem was mitigated once Marek's SDMA
> patch was merged, which hugely reduces the PCIe bandwidth use.
> 3. In less optimized cases (for example D9VK), the problem is still
> very noticable.

However, since you saw as much as ~20 Gbit/s under different
circumstances, the 2.43 Gbit/s used by this game clearly isn't a hard
limit; there must be other limiting factors.


-- 
Earthling Michel Dänzer               |              https://www.amd.com
Libre software enthusiast             |             Mesa and X developer