Commit f5d9b7f0f9 (fix r600_enable_sclk_control()) causes kexec issues

[Alex Deucher]

On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf
<markus at trippelsdorf.de> wrote:
> On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
>> On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
>> <ebiederm at xmission.com> wrote:
>> > Alex Deucher <alexdeucher at gmail.com> writes:
>> >
>> >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
>> >> <ebiederm at xmission.com> wrote:
>> >>>
>> >>>
>> >>> Alex Deucher <alexdeucher at gmail.com> wrote:
>> >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>> >>>><markus at trippelsdorf.de> wrote:
>> >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>> >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>> >>>>>> <markus at trippelsdorf.de> wrote:
>> >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>> >>>>>> > 3.11.0-rc3 kernel.
>> >>>>>>
>> >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
>> >>>>>> bad hardware state when the driver loads again.  Does the attached
>> >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>> >>>>>> torn down properly previously.
>> >>>>>
>> >>>>> dpm initialization now works, but unfortunately GPU acceleration
>> >>>>still gets
>> >>>>> disabled:
>> >>>>
>> >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
>> >>>>down before we init the rest of the GPU, but dpm needs get initialized
>> >>>>later in the init process since it depends on certain other state from
>> >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
>> >>>>good way to handle this.
>> >>>
>> >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
>> >>>
>> >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
>> >>
>> >> Does kexec actually call this shutdown method?  The driver implements
>> >> appropriate clean-up measures if it's shutdown properly.
>> >
>> > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
>> > kexec.
>> >
>> > You don't get a device remove/hotunplug but unless this is a kexec on
>> > panic ->shutdown is most definitely called.  Now I am talking about the
>> > device layer/pci layer shutdown method I don't know how gpu drivers are
>> > wired up.  GPU land was a little strange last I looked.  Hopefully it
>> > isn't so strange that there is a method named shutdown that is not wired
>> > up.
>>
>> It doesn't look like the drm infrastructure has a shutdown callback.
>> The drm drivers register a drm_driver callback struct that includes an
>> unload callback which takes care of all the device teardown (if you
>> unload the module for example).  I don't know that it actually gets
>> called at kexec time however.  I don't know enough about how kexec
>> works.
>
> BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly
> handles the kexec case during init. So maybe an r600_restore_sanity()
> function is needed?
>
> (One of the advantages of using kexec (besides the much quicker boot
> time) is that the monitor is not switched off and then on during boot.
> I guess that advantage would be lost if the unload callback would be
> called.)

r100_restore_sanity() is basically a set of hacks (that gets called at
driver startup) to work around the fact that with kexec the drm driver
is not torn down correctly.  So we could add a bunch more asic
specific tear down sequences to deal with dpm (and all the other
engines on the GPU that may potentially cause problems if they are not
torn down properly), but that will just turn into a mess.  All of
these hacks also add latency to the driver load.  I think the best
solution would probably be to figure how to hook up the drm unload
callback to the shutdown method that kexec uses.

Alex