Why is Thunderbolt 3 limited to 2.5 GT/s on Linux?

[Alex Deucher]

On Sun, Jun 30, 2019 at 2:27 PM Timur Kristóf <timur.kristof at gmail.com> wrote:
>
>
> > > Sure, though in this case 3 of those downstream ports are not
> > > exposed
> > > by the hardware, so it's a bit surprising to see them there.
> >
> > They lead to other peripherals on the TBT host router such as the TBT
> > controller and xHCI. Also there are two downstream ports for
> > extension
> > from which you eGPU is using one.
>
> If you look at the device tree from my first email, you can see that
> both the GPU and the XHCI uses the same port: 04:04.0 - in fact I can
> even remove the other 3 ports from the system without any consequences.
>
> > > Like I said the device really is limited to 2.5 GT/s even though it
> > > should be able to do 8 GT/s.
> >
> > There is Thunderbolt link between the host router (your host system)
> > and
> > the eGPU box. That link is not limited to 2.5 GT/s so even if the
> > slot
> > claims it is PCI gen1 the actual bandwidth can be much higher because
> > of
> > the virtual link.
>
> Not sure I understand correctly, are you saying that TB3 can do 40
> Gbit/sec even though the kernel thinks it can only do 8 Gbit / sec?
>
> I haven't found a good way to measure the maximum PCIe throughput
> between the CPU and GPU, but I did take a look at AMD's sysfs interface
> at /sys/class/drm/card1/device/pcie_bw which while running the
> bottlenecked game. The highest throughput I saw there was only 2.43
> Gbit /sec.
>
> One more thought. I've also looked at
> /sys/class/drm/card1/device/pp_dpm_pcie - which tells me that amdgpu
> thinks it is running on a 2.5GT/s x8 link (as opposed to the expected 8
> GT/s x4). Can this be a problem?

We limit the speed of the link the the driver to the max speed of any
upstream links.  So if there are any links upstream limited to 2.5
GT/s, it doesn't make sense to clock the local link up to faster
speeds.

Alex

>
> >
> > > > > 3. Is it possible to manually set them to 8 GT/s?
> > > >
> > > > No idea.
> > > >
> > > > Are you actually seeing some performance issue because of this or
> > > > are
> > > > you just curious?
> > >
> > > Yes, I see a noticable performance hit: some games have very low
> > > frame
> > > rate while neither the CPU nor the GPU are fully utilized.
> >
> > Is that problem in Linux only or do you see the same issue in Windows
> > as
> > well?
>
>
> I admit I don't have Windows on this computer now and it has been some
> time since I last tried it, but when I did, I didn't see this problem.
>
> Best regards,
> Tim
>
> _______________________________________________
> dri-devel mailing list
> dri-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel