[PATCHv2 0/3] Intel FPGA VIP Frame Buffer II DRM Driver

[Ong, Hean Loong]

On Tue, 2017-05-09 at 09:40 -0700, Eric Anholt wrote:
> "Ong, Hean Loong" <hean.loong.ong at intel.com> writes:
> 
> > 
> > On Mon, 2017-05-08 at 09:03 -0700, Eric Anholt wrote:
> > > 
> > > "Ong, Hean Loong" <hean.loong.ong at intel.com> writes:
> > > 
> > > > 
> > > > 
> > > > On Thu, 2017-05-04 at 10:11 -0700, Eric Anholt wrote:
> > > > > 
> > > > > 
> > > > > "Ong, Hean Loong" <hean.loong.ong at intel.com> writes:
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Wed, 2017-05-03 at 13:28 -0700, Eric Anholt wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > hean.loong.ong at intel.com writes:
> > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > > From: Ong Hean Loong <hean.loong.ong at intel.com>
> > > > > > > > 
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > The new Intel Arria10 SOC FPGA devkit has a Display
> > > > > > > > Port IP
> > > > > > > > component 
> > > > > > > > which requires a new driver. This is a virtual driver
> > > > > > > > in
> > > > > > > > which
> > > > > > > > the
> > > > > > > > FGPA hardware would enable the Display Port based on
> > > > > > > > the
> > > > > > > > information
> > > > > > > > and data provided from the DRM frame buffer from the
> > > > > > > > OS.
> > > > > > > > Basically
> > > > > > > > all
> > > > > > > > all information with reagrds to resolution and bits per
> > > > > > > > pixel
> > > > > > > > are
> > > > > > > > pre-configured on the FPGA design and these information
> > > > > > > > are
> > > > > > > > fed
> > > > > > > > to
> > > > > > > > the driver via the device tree information as part of
> > > > > > > > the
> > > > > > > > hardware 
> > > > > > > > information.
> > > > > > > I started reviewing the code, but I want to make sure I
> > > > > > > understand
> > > > > > > what's going on:
> > > > > > > 
> > > > > > > This IP core isn't displaying contents from system memory
> > > > > > > on
> > > > > > > some
> > > > > > > sort
> > > > > > > of actual physical display, right?  It's a core that
> > > > > > > takes
> > > > > > > some
> > > > > > > input
> > > > > > > video stream (not described in the DT or in this driver)
> > > > > > > and
> > > > > > > stores
> > > > > > > it
> > > > > > > to memory?
> > > > > > If the IP Core you are referring to is some form of GPU
> > > > > > then in
> > > > > > this
> > > > > > case we are using the Intel FPGA Display Port Framebuffer
> > > > > > IP.
> > > > > > It
> > > > > > does
> > > > > > display contents streamed from the ARM/Linux system to the
> > > > > > Display
> > > > > > Port
> > > > > > of the physical Monitor.
> > > > > > 
> > > > > > Below a simple illustration of the system:
> > > > > > 
> > > > > > ARM/Linux --DMA-->Intel FPGA Display Port Framebuffer IP
> > > > > > 				|
> > > > > > 				|
> > > > > > 			Physical Connection
> > > > > > 			Display Port
> > > > > The "DMA" in this diagram is the frame reader IP, right?  The
> > > > > frame
> > > > > reader, as described in the spec, sounds approximately like a
> > > > > DRM
> > > > > plane,
> > > > > so if you have that in your system then that needs to be part
> > > > > of
> > > > > this
> > > > > DRM driver (otherwise you won't be putting the right things
> > > > > on
> > > > > the
> > > > > screen!).
> > > > Would the drm_simple_display_pipe_init be able to handle this ?
> > > > It
> > > > seems to be displaying the proper images on screen, based on my
> > > > current
> > > > changes. There were recommendations to use the
> > > > drm_simple_display_pipe_init instead of creating the CRTC and
> > > > planes
> > > > myself
> > > The docs I've read for the Frame Buffer IP don't fit into any
> > > current
> > > DRM component, since it's what we're currently calling a
> > > "writeback
> > > connector".
> > > 
> > While running my own test (since the intel-gpu-tools doesn't seems
> > applicable.) The drm_simple_display_pipe_init seems to be simple
> > enough
> > for displaying a matchbox console. The use case for this driver is
> > only
> > part of the enablemennt of the reference design board so that users
> > of
> > the Arria10 devkit can use this as base for their own designs.
> > 
> > > 
> > > I don't understand your system enough to answer.  You didn't
> > > answer
> > > if
> > > the "DMA" block you diagrammed was the frame reader IP (or maybe
> > > the
> > > frame reader IP comes after).  I also don't see how the frame
> > > buffer
> > > IP
> > > could possibly be connected directly to a physical display
> > > connector.
> > The FPGA FrameBuffer 2 soft IP reads the information provided to it
> > from the Linux Kernel via the platform device register provided in
> > the
> > DT. The Linux driver here allocates a chunk of coherent memory that
> > directly maps to the SDRAM. The FPGA soft IP would stream the
> > display
> > out to the Display Port based on the information that was written
> > to
> > the SDRAM.
> Your "FPGA soft IP" that's reading pixels from an area of memory (a
> DRM
> plane does this) and outputting that to a display port (a DRM CRTC
> and
> encoder does this) would make a lot of sense as a DRM driver.
> 
> However, this piece of hardware that takes in pixels in a stream from
> elsewhere and stores it to memory doesn't make sense to me as a DRM
> driver.
> 
> I can't really offer more advice until I understand what's generating
> the pixels that the FrameBuffer IP is storing.

My previous explanation might have been confusing here. I apologize for
that. 
To put it in simpler terms the FPGA FrameBuffer Soft IP could be seen
as the GPU and the DRM driver patch here is allocating memory for
information to be streamed from the ARM/Linux to the display port.
Basically the driver just wraps the information such as the pixels to
be drawn by the FPGA FrameBuffer 2.

The piece of hardware in discussion is the SoC FPGA where Linux runs on
the ARM chip and the FGPA is driven by its NIOS soft core with its own
proprietary firmware.

For example the application from the ARM Linux would have to write
information on the /dev/fb0 with the information stored in the SDRAM to
be fetched by the FPGA framebuffer IP and displayed on the Display Port
Monitor.