Commit f5d9b7f0f9 (fix r600_enable_sclk_control()) causes kexec issues

[Markus Trippelsdorf]

On 2013.07.30 at 20:46 +0200, Markus Trippelsdorf wrote:
> On 2013.07.30 at 10:53 -0400, Alex Deucher wrote:
> > On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf
> > <markus at trippelsdorf.de> wrote:
> > > On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
> > >> On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
> > >> <ebiederm at xmission.com> wrote:
> > >> > Alex Deucher <alexdeucher at gmail.com> writes:
> > >> >
> > >> >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
> > >> >> <ebiederm at xmission.com> wrote:
> > >> >>>
> > >> >>>
> > >> >>> Alex Deucher <alexdeucher at gmail.com> wrote:
> > >> >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
> > >> >>>><markus at trippelsdorf.de> wrote:
> > >> >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
> > >> >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
> > >> >>>>>> <markus at trippelsdorf.de> wrote:
> > >> >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
> > >> >>>>>> > 3.11.0-rc3 kernel.
> > >> >>>>>>
> > >> >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
> > >> >>>>>> bad hardware state when the driver loads again.  Does the attached
> > >> >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
> > >> >>>>>> torn down properly previously.
> > >> >>>>>
> > >> >>>>> dpm initialization now works, but unfortunately GPU acceleration
> > >> >>>>still gets
> > >> >>>>> disabled:
> > >> >>>>
> > >> >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
> > >> >>>>down before we init the rest of the GPU, but dpm needs get initialized
> > >> >>>>later in the init process since it depends on certain other state from
> > >> >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
> > >> >>>>good way to handle this.
> > >> >>>
> > >> >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
> > >> >>>
> > >> >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
> > >> >>
> > >> >> Does kexec actually call this shutdown method?  The driver implements
> > >> >> appropriate clean-up measures if it's shutdown properly.
> > >> >
> > >> > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
> > >> > kexec.
> > >> >
> > >> > You don't get a device remove/hotunplug but unless this is a kexec on
> > >> > panic ->shutdown is most definitely called.  Now I am talking about the
> > >> > device layer/pci layer shutdown method I don't know how gpu drivers are
> > >> > wired up.  GPU land was a little strange last I looked.  Hopefully it
> > >> > isn't so strange that there is a method named shutdown that is not wired
> > >> > up.
> > >>
> > >> It doesn't look like the drm infrastructure has a shutdown callback.
> > >> The drm drivers register a drm_driver callback struct that includes an
> > >> unload callback which takes care of all the device teardown (if you
> > >> unload the module for example).  I don't know that it actually gets
> > >> called at kexec time however.  I don't know enough about how kexec
> > >> works.
> > >
> > > BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly
> > > handles the kexec case during init. So maybe an r600_restore_sanity()
> > > function is needed?
> > >
> > > (One of the advantages of using kexec (besides the much quicker boot
> > > time) is that the monitor is not switched off and then on during boot.
> > > I guess that advantage would be lost if the unload callback would be
> > > called.)
> > 
> > r100_restore_sanity() is basically a set of hacks (that gets called at
> > driver startup) to work around the fact that with kexec the drm driver
> > is not torn down correctly.  So we could add a bunch more asic
> > specific tear down sequences to deal with dpm (and all the other
> > engines on the GPU that may potentially cause problems if they are not
> > torn down properly), but that will just turn into a mess.  All of
> > these hacks also add latency to the driver load.  I think the best
> > solution would probably be to figure how to hook up the drm unload
> > callback to the shutdown method that kexec uses.
> 
> FYI the following (ugly) hack works for me. 

No. It still fails, although much more infrequently (~ on every 6-8 boot).

I begin to wonder if:
 [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) 
is an simple initialization bug that doesn't directly depend on kexec at
all.

-- 
Markus