Why is Thunderbolt 3 limited to 2.5 GT/s on Linux?

[Mika Westerberg]

On Fri, Jun 28, 2019 at 02:21:36PM +0200, Timur Kristóf wrote:
> 
> > > Sure, though in this case 3 of those downstream ports are not
> > > exposed
> > > by the hardware, so it's a bit surprising to see them there.
> > 
> > They lead to other peripherals on the TBT host router such as the TBT
> > controller and xHCI. Also there are two downstream ports for
> > extension
> > from which you eGPU is using one.
> 
> If you look at the device tree from my first email, you can see that
> both the GPU and the XHCI uses the same port: 04:04.0 - in fact I can
> even remove the other 3 ports from the system without any consequences.

Well that's the extension PCIe downstream port. The other one is
04:01.0.

Typically 04:00.0 and 04:00.2 are used to connect TBT (05:00.0) and xHCI
(39:00.0) but in your case you don't seem to have USB 3 devices
connected to that so it is not present. If you plug in USB-C device
(non-TBT) you should see the host router xHCI appearing as well.

This is pretty standard topology.

> > > Like I said the device really is limited to 2.5 GT/s even though it
> > > should be able to do 8 GT/s.
> > 
> > There is Thunderbolt link between the host router (your host system)
> > and
> > the eGPU box. That link is not limited to 2.5 GT/s so even if the
> > slot
> > claims it is PCI gen1 the actual bandwidth can be much higher because
> > of
> > the virtual link.
> 
> Not sure I understand correctly, are you saying that TB3 can do 40
> Gbit/sec even though the kernel thinks it can only do 8 Gbit / sec?

Yes the PCIe switch upstream port (3a:00.0) is connected back to the
host router over virtual Thunderbolt 40Gb/s link so the PCIe gen1 speeds
it reports do not really matter here (same goes for the downstream).

The topology looks like bellow if I got it right from the lspci output:

  00:1c.4 (root port) 8 GT/s x 4
    ^
    | real PCIe link
    v
  03:00.0 (upstream port) 8 GT/s x 4
  04:04.0 (downstream port) 2.5 GT/s x 4
    ^
    |  virtual link 40 Gb/s
    v
  3a:00.0 (upstream port) 2.5 GT/s x 4
  3b:01.0 (downstream port) 8 GT/s x 4
    ^
    | real PCIe link
    v
  3c:00.0 (eGPU) 8 GT/s x 4

In other words all the real PCIe links run at full 8 GT/s x 4 which is
what is expected, I think.