[Mesa-dev] [PATCH 2/2] gallium: add flag PIPE_TRANSFER_MAP_PERMANENTLY

[Jose Fonseca]

----- Original Message -----
> On 01/10/2012 01:13 PM, Jose Fonseca wrote:
> > ----- Original Message -----
> >> On 10.01.2012 12:29, Jose Fonseca wrote:
> >>> Still catching up on email traffic during holidays...
> >>>
> >>> I agree that user buffer uploads should be moved out of drivers,
> >>> but I don't think this is the way to go.
> >>
> >> I don't. If the state tracker uploads user buffers and presents
> >> them
> >> to
> >> the driver as normal buffers, it will have to change the
> >> pipe_vertex_buffers for each draw, causing annoying re-validation
> >> overhead.
> > 
> > This is not strictly necessary.  We could simply use index_bias
> > provided the vertex strides don't change (the likely scenario for
> > the sort of cases you are talking here). Thomas did that something
> > along those lines for the svga driver.
> > 
> Changing the index bias changes the VertexID, so you can't. But I
> suppose we could add a second index bias that doesn't affect it ...
> 
> > And we could do much better than that: I'm talking about proper
> > reverse-engineering of the application intention, such as using
> > one-shot dynamic vertex buffers for one-shot data, and reusable
> > static vertex buffers for immutable data. Employing both
> > heuristics like scanning the user memory for changes and end of
> > data; and also get information from the OS virtual memory
> > subsystem (e.g., if the user memory pointer is in a read-only
> > segment then it won't likely change, if it's in a stack segment it
> > will likely be one-off, if its in malloc, then we can get an upper
> > bound from the OS).   This analysis / memory tracking is something
> > too complex to do for each driver, but worthwhile if its done in
> > the state tracker for all drivers benefit.
> > 
> > I honestly think that's unnecessary to bother pipe drivers with
> > user pointers and that we should put our bets on solving this
> > problem efficiently once for all -- but I'm fine leaving these
> > user pointers in the interface eternally, for sake of both
> > skeptics and software renderers (who can always benefit from
> > accessing user memory directly).
> > 
> > 
> >> This is very bad if there are a lot of small draw calls.
> >> Unfortunately many applications do this, and we do care about
> >> these
> >> cases.
> >>
> >> Also, another example I'm dealing with atm, it will be difficult
> >> to
> >> extract a small set of wide-spread vertices from a user buffer
> >> into a
> >> oneshot vertex buffer (there are apps where this really helps a
> >> lot)
> >> because you're changing the vertex index / gl_VertexID.
> >> I can do that, because I know the hw has a special vertex-index
> >> buffer
> >> (nvc0) or manual VERTEX_ID attribute (nv50) that can be routed to
> >> the
> >> VERTEXID system value so I don't need to modify the shader.
> > 
> > Backing user buffers on hardware buffers does not necessary imply
> > remapping vertices -- that's merely how it is implemented now. We
> > could either create a one-shot hardware buffer with the same size
> > and fill in just the necessary spots; or better to recognize the
> > underlying user memory buffer start and size and track changes. We
> > could also expose this vertex-index buffer in the interface (other
> > drivers could implement it as vertex texture).
> > 
> 
> Filling in just the necessary spots wastes a lot of space, especially
> if
> these spots are re-used with different data (you need to reallocate
> space), and checking if the relevant data changed is a performance
> loss
> in itself.

Don't forget that the alternative to checking is to copy blindly, which also implies reading the source.  If one uses hashes instead of compares, and one can determine statistically that certain contents is usually immutable, then one can guarantee that checking to be a win, especially considering the bandwidth difference on PCIe vs VRAM.

After a couple of frames it should be clear whether a user memory buffer at a given address is static or dynamic.

> 
> > 
> > Probably the best approach is a mixture: back big dynamic/static
> > user buffers in individual hardware buffers, back small dynamic
> > user buffers in linearly suballocated ranges of coarse sized
> > hardware buffers (i.e., what we do now).
> > 
> 
> In the end, I'll only be convinced removing user buffers is good if
> the
> new solution performs equally or better in all cases (or at least not
> significantly worse in the ugliest of cases) ...

Of course. As I said, I'm fine w/ leaving user buffers interface if just for software renderers sake. And this is all just an idea at this stage.

Jose