Funky new vblank counter regressions in Linux 4.4-rc1

[Alex Deucher]

On Thu, Nov 19, 2015 at 12:46 PM, Mario Kleiner
<mario.kleiner.de at gmail.com> wrote:
> Hi Alex and Michel and Ville,
>
> it's "fix vblank stuff" time again ;-)

Adding Harry from our display team.  He might be able to fill in the
blanks of on some of this better than I can.  It might also be worth
checking to see how our DAL (our new display code which is being
developed directly by our display team) code handles this.  It could
be that we are just missing register settings:
http://cgit.freedesktop.org/~agd5f/linux/log/?h=DAL-wip
Additionally we've published full registers headers for the display block:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/amd/include/asic_reg/dce
The DCE8 stuff should generally apply back to DCE4.  If you have
questions about registers older asics not covered in the hw docs, let
me know.  Note the new headers are dword aligned rather than byte
aligned.

>
> Ville's changes to the DRM's drm_handle_vblank() / drm_update_vblank_count()
> code in Linux 4.4 not only made that code more elegant, but also removed the
> robustness against the vblank irq quirks in AMD hw and similar hardware. So
> now i get tons of off-by-one errors and
>
> "[   432.345] (WW) RADEON(1): radeon_dri2_flip_event_handler: Pageflip
> completion event has impossible msc 24803 < target_msc 24804" XOrg messages
> from that kernel.
>
> One of the reasons for trouble is that AMD hw quirk where the hw fires an
> extra vblank irq shortly after vblank irq's get enabled, not synchronized to
> vblank, but typically in the middle of active scanout, so we get a redundant
> call to drm_handle_vblank in the middle of scanout.
>
> To fix that i have a minor patch to make drm_update_vblank_count() again
> robust against such redundant calls, which i will send out later to the
> mailing list. Diff attached for reference.
>
> The second quirk of AMD hw is that the vblank interrupt fires a few
> scanlines before start of vblank, so drm_handle_vblank ->
> drm_update_vblank_count() -> dev->driver->get_vblank_counter() gets called
> before the start of the vblank for which the new vblank count should be
> queried.
>
> The third problem is that the DRM vblank handling always had the assumption
> that hardware vblank counters would essentially increment at leading edge of
> vblank - basically in sync with the firing of the vblank irq, so that a hw
> counter readout from within the vblank irq handler would always deliver the
> new incremented value. If this assumption is violated then the counting by
> use of the hw counter gets unreliable, because depending on random small
> delays in irq handling the code may end up sampling the hw counter pre- or
> post-increment, leading to inconsistent updating and funky bugs. It just so
> happens that AMD hardware doesn't increment the hw counter at leading edge
> of vblank, so stuff falls apart.
>
> So to fix those two problems i'm tinkering with cooking the hw vblank
> counter value returned by radeon_get_vblank_counter_kms() to make it appear
> as if the counter incremented at leading edge of vblank in sync with vblank
> irq.
>
> It almost sort of works on the rs600 code path, but i need a bit of info
> from you:
>
> 1. There's this register from the old specs for m76.pdf, which is not part
> of the current register defines for radeon-kms:
>
> "D1CRTC_STATUS_VF_COUNT - RW - 32 bits - [GpuF0MMReg:0x60A8]"
>
> It contains the lower 16 bits of framecounter and the 13 bits of vertical
> scanout position. It seems to give the same readings as the 24 bit
> R_0060A4_D1CRTC_STATUS_FRAME_COUNT we use for the hw counter. This would
> come handy.
>
> Does Evergreen and later have a same/similar register and where is it?

Yes.  CRTC_STATUS_VF_COUNT

CRTC:CRTC_STATUS_VF_COUNT  ·  [R/W]  ·  32 bits  ·  Access: 8/16/32  ·
 [INST0] GpuF0MMReg:0x6e9c, [INST1] GpuF0MMReg:0x7a9c, [INST2]
GpuF0MMReg:0x1069c, [INST3] GpuF0MMReg:0x1129c, [INST4]
GpuF0MMReg:0x11e9c, [INST5] GpuF0MMReg:0x12a9c
DESCRIPTION: Current composite vertical and frame count for CRTC
Field Name   Bits    Default   Description
CRTC_VF_COUNT
(Access: R)    28:0    0x0    Reports current vertical and frame count

>
> 2. The hw framecounter seems to increment when the vertical scanout position
> wraps back from (VTOTAL - 1) to 0, at least on the one DCE-3 gpu i tested so
> far. Is this so on all asics? And is the hw counter increment happening
> exactly at the moment that vertical scanout position jumps back to zero, ie.
> both events are driven by the same signal? Or is the framecounter increment
> just happening somewhere inside either scanline VTOTAL-1 or scanline 0?

Unless Harry knows, we can ask the hw team, but I doubt they would
have changed it.  I think it's tied to the start line which is
controlled by this register:
CRTC:CRTC_START_LINE_CONTROL  ·  [R/W]  ·  32 bits  ·  Access: 8/16/32
 ·  [INST0] GpuF0MMReg:0x6ecc, [INST1] GpuF0MMReg:0x7acc, [INST2]
GpuF0MMReg:0x106cc, [INST3] GpuF0MMReg:0x112cc, [INST4]
GpuF0MMReg:0x11ecc, [INST5] GpuF0MMReg:0x12acc
DESCRIPTION: move start_line signal earlier by 1 line in CRTC
Field Name   Bits          Default         Description
CRTC_PROGRESSIVE_START_LINE_EARLY         0    0x0    move start_line
signal by 1 line eariler in progressive mode

CRTC_INTERLACE_START_LINE_EARLY         8     0x1     move start_line
signal by 1 line earlier in interlaced timing mode

CRTC_ADVANCED_START_LINE_POSITION         19:16     0x4  Advanced
Start Line position respect to not VBLANK. Default of 4 means the
Advanced Start Line is 4 lines before the first non VBLANK line

The following info I dug up internally may be useful:

The position of the CRTC output timing generator when the “start of
frame” indicator occurs depends on several conditions.  These
conditions are whether the timing generator is outputting timing for a
progressive or interlaced display mode, whether the scaler is enabled,
and if so, whether the register to select an earlier than normal
“start of frame” indicator is set.  The “start of frame” indicator
typically occurs 2 lines before the vertical blank end position
(DxCRTC_V_BLANK_END) is reached

When interlaced output display modes are enabled
(DxCRTC_INTERLACE_ENABLE = 1) and the CRTC timing generator is enabled
(DxCRTC_MASTER_EN = 1), the timing generator’s vertical counter counts
by 2 for both the even fields and odd fields of each frame.  For both
the even fields and the odd fields of interlaced output modes, the
“start of frame” indicator occurs 2 lines before the end of the
vertical blank occurs (DxCRTC_V_BLANK_END – 4 or DxCRTC_V_BLANK_END –
3 depending on the field since the vertical counter counts by 2 in
interlaced), since the vertical counter counts by 2 in this mode).
There is one exception to the previous statement.  If scaling is
enabled (DxSCL_SCALE_EN = 1) and the early start line is enabled
(DxCRTC_INTERLACE_START_LINE_EARLY = 1) in interlaced output mode then
the “start of frame” indicator happens 3 lines before the end of the
vertical blank (DxCRTC_V_BLANK_END – 6 or DxCRTC_V_BLANK_END – 5
depending on the field since the vertical counter counts by 2 in
interlaced).

When progressive output display modes are enabled
(DxCRTC_INTERLACE_ENABLE = 0) and the CRTC timing generator is enabled
(DxCRTC_MASTER_EN = 1), the “start of frame” indicator occurs 2 lines
before the end of the vertical blank occurs (DxCRTC_V_BLANK_END - 2).
Similar to interlaced output mode, there is one exception to the
previous statement.  If scaling is enabled (DxSCL_SCALE_EN = 1) and
the early start line is enabled (DxCRTC_PROGRESSIVE_START_LINE_EARLY =
1) in progressive output mode then the “start of frame” indicator
happens 3 lines before the end of the vertical blank
(DxCRTC_V_BLANK_END – 3)

I hope this helps,

Alex

>
>
> If we can fix this and get it into rc2 or rc3 then we could avoid a bad
> regression and with a bit of luck at the same time improve by being able to
> set dev->vblank_disable_immediate = true then and allow vblank irqs to get
> turned off more aggressively for a bit of extra power saving.
>
> thanks,
> -mario