Raspberry Pi 3 Model B+ hangs in vc4_hdmi_runtime_resume()

Thu Sep 29 13:46:01 UTC 2022

Am 29.09.22 um 11:35 schrieb Maxime Ripard:
> On Tue, Sep 27, 2022 at 03:15:17PM +0200, Maxime Ripard wrote:
>> On Tue, Sep 27, 2022 at 02:25:12PM +0200, Maxime Ripard wrote:
>>> On Tue, Sep 27, 2022 at 01:42:40PM +0200, Maxime Ripard wrote:
>>>> On Tue, Sep 27, 2022 at 01:12:35PM +0200, Stefan Wahren wrote:
>>>>> Am 27.09.22 um 11:42 schrieb Maxime Ripard:
>>>>>> On Tue, Sep 27, 2022 at 09:25:54AM +0200, Maxime Ripard wrote:
>>>>>>> Hi Stefan,
>>>>>>>
>>>>>>> On Mon, Sep 26, 2022 at 08:50:12PM +0200, Stefan Wahren wrote:
>>>>>>>> Am 26.09.22 um 14:47 schrieb Maxime Ripard:
>>>>>>>>> On Mon, Sep 26, 2022 at 02:40:48PM +0200, Marc Kleine-Budde wrote:
>>>>>>>>>> On 26.09.2022 14:08:04, Stefan Wahren wrote:
>>>>>>>>>>> Hi Marc,
>>>>>>>>>>>
>>>>>>>>>>> Am 26.09.22 um 12:21 schrieb Marc Kleine-Budde:
>>>>>>>>>>>> On 22.09.2022 17:06:00, Maxime Ripard wrote:
>>>>>>>>>>>>>> I'm on a Raspberry Pi 3 Model B+ running current Debian testing ARM64,
>>>>>>>>>>>>>> using Debian's v5.19 kernel (Debian's v5.18 was working flawless).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> | [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
>>>>>>>>>>>>>> | [    0.000000] Linux version 5.19.0-1-arm64 (debian-kernel at lists.debian.org) (gcc-11 (Debian 11.3.0-5) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38.90.20220713) #1 SMP Debian 5.19.6-1 (2022-0
>>>>>>>>>>>>>> 9-01)
>>>>>>>>>>>>>> | [    0.000000] Machine model: Raspberry Pi 3 Model B+
>>>>>>>>>>>>>> | [    3.747500] raspberrypi-firmware soc:firmware: Attached to firmware from 2022-03-24T13:21:11
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As soon a the vc4 module is loaded the following warnings hits 4
>>>>>>>>>>>>>> times, then the machine stops.
>>>>>>>>>>>> [...]
>>>>>>>>>>>>
>>>>>>>>>>>>> The warning itself is fixed, both upstream and in stable (5.19.7).
>>>>>>>>>>>> Ok. Debian is using 5.19.6
>>>>>>>>>>>>
>>>>>>>>>>>>> It shouldn't have any relation to the hang though. Can you share your
>>>>>>>>>>>>> setup?
>>>>>>>>>>>> - config.txt:
>>>>>>>>>>>>
>>>>>>>>>>>> -------->8-------->8-------->8-------->8--------
>>>>>>>>>>>> gpu_mem=16
>>>>>>>>>>>> disable_splash=1
>>>>>>>>>>>>
>>>>>>>>>>>> arm_64bit=1
>>>>>>>>>>>> enable_uart=1
>>>>>>>>>>>> uart_2ndstage=1
>>>>>>>>>>>>
>>>>>>>>>>>> os_prefix=/u-boot/
>>>>>>>>>>>>
>>>>>>>>>>>> [pi3]
>>>>>>>>>>>> force_turbo=1
>>>>>>>>>>>> -------->8-------->8-------->8-------->8--------
>>>>>>>>>>>>
>>>>>>>>>>>> - Raspberry Pi 3 Model B+
>>>>>>>>>>>> - no HDMI connected
>>>>>>>>>>> Does it mean, the issue only occurs without HDMI connected?
>>>>>>>>>>> If you didn't test with HDMI yet, could you please do?
>>>>>>>>>> The error occurs with HDMI not connected, as vc4 is the gfx driver I
>>>>>>>>>> thought this might be of interest. :)
>>>>>>>>>>
>>>>>>>>>> I don't have a HDMI monitor here, but I'll come back to you as soon as I
>>>>>>>>>> get access to one (might take some time).
>>>>>>>>> It's not the first time an issue like this one would occur. I'm trying
>>>>>>>>> to make my Pi3 boot again, and will try to bisect the issue.
>>>>>>>> yes the issue is only triggered without HDMI connected. I was able to
>>>>>>>> reproduce with an older vc4 firmware from 2020 (don't want to upgrade yet).
>>>>>>>> Kernel was also an arm64 build with defconfig.
>>>>>>>>
>>>>>>>> Here some rough starting point for bisection:
>>>>>>>>
>>>>>>>> 5.18.0 good
>>>>>>>> 5.19.0 bad
>>>>>>>> 5.19.6 bad
>>>>>>> Sorry it took a bit of time, it looks like I found another bug while
>>>>>>> trying to test this yesterday.
>>>>>>>
>>>>>>> Your datapoints are interesting though. I have a custom configuration
>>>>>>> and it does boot 5.19 without an HDMI connected.
>>>>>>>
>>>>>>> So I guess it leaves us with either the firmware version being different
>>>>>>> (I'm using a newer version, from March 2022), or the configuration. I'll
>>>>>>> test with defconfig.
>>>>>> So it turns out compiling vc4 as a module is the culprit.
>>>>> Do you mean regardless of the kernel version in your case?
>>>> No, I mean that, with vc4 as a module, 5.18 works but 5.19 doesn't, like
>>>> Marc said. But if vc4 is built in, both work.
>>>>
>>>>> In my test cases i build vc4 always as module.
>>>>>
>>>>>> It's not clear to me why at this point, but the first register write in
>>>>>> vc4_hdmi_reset stalls.
>>>>> Sounds like timing issue or a missing dependency (clock or power domain)
>>>> It felt like a clock or power domain issue to me indeed, but adding
>>>> clk_ignore_unused and pd_ignore_unused isn't enough, so it's probably
>>>> something a bit more complicated than just the clock / PD being
>>>> disabled.
>>> I found the offending patch:
>>> https://lore.kernel.org/dri-devel/20220225143534.405820-13-maxime@cerno.tech/
>>>
>>> That code was removed because it was made irrelevant by that earlier patch:
>>> https://lore.kernel.org/dri-devel/20220225143534.405820-10-maxime@cerno.tech/
>>>
>>> But it turns out that while it works when the driver is built-in, it
>>> doesn't when it's a module. If we add a clk_hw_get_rate() call right
>>> after that call to raspberrypi_fw_set_rate(), the rate returned is 0.
>>>
>>> I'm not entirely sure why, but I wonder if it's related to:
>>> https://github.com/raspberrypi/linux/issues/4962#issuecomment-1228593439
>> Turns out it's not, since the Pi3 is using the clk-bcm2835 driver.
>>
>> However, even reverting that patch fails. clk_set_min_rate fails because
>> the rate is protected, but it doesn't look like it is anywhere for that
>> clock, so I'm a bit confused.
>>
>> Even if we do remove the clock protection check in
>> clk_core_set_rate_nolock(), clk_calc_new_rates() will then fail because
>> the bcm2835 driver will round the clock rate below the minimum, which is
>> rejected.
>>
>> I'm not entirely sure what to do at this point. I guess the proper fix
>> would be to:
>>    - Figure out why it's considered protected when it's not (or shouldn't be)
>>    - Make the driver compute an acceptable rate for that clock
> This was due to 5.19.7 missing 88110a9f6209

Do you really mean this commit ("clk: bcm2835: fix 
bcm2835_clock_choose_div")?

I think this is applied in 5.19.7:

https://elixir.bootlin.com/linux/v5.19.7/source/drivers/clk/bcm/clk-bcm2835.c#L944

>
>>    - Reintroduce the clk_set_min_rate call to HDMI's runtime_resume, or
>>      some other equivalent code
> I just sent a series addressing this here:
> https://lore.kernel.org/r/20220929-rpi-pi3-unplugged-fixes-v1-0-cd22e962296c@cerno.tech
Thanks
>
> Thanks for the report and the help :)
> Maxime