LLM GPU Support

[Blake McBride]

I am sorry.  Perhaps I am under the wrong impression.  My understanding is
that there are three pieces needed (in addition to the hardware) as follows:

1. the low-level drivers (nouveau or Nvidia)
2. the GPU library (Nvidia only)
3. the LLM software

#3 requires #2.  #2 requires the Nvidia low-level driver and will not work
with the nouveau driver.

If I am correct, specifying any of what you ask is superfluous.  If I am
incorrect, great!  Glad to hear it!  I need to investigate further.

Please let me know if nouveau supports the Nvidia's GPU library or not (or
perhaps my assumptions are incorrect)?

Thanks!

Blake McBride


On Wed, Jun 5, 2024 at 5:42 AM Philipp Stanner <pstanner at redhat.com> wrote:

> On Tue, 2024-06-04 at 10:27 -0500, Blake McBride wrote:
> > Greetings,
> >
> > I have used the nouveau driver with my Nvidia card on Linux.  Works
> > fine.  However, my problem has to do with running LLM on my GPU with
> > your driver.  My impression is, it doesn't work.  Am I correct?
>
> Yo,
>
> "it doesn't work" is not a very precise error description.
>
> The maintainers will need info such as:
>
> Which kernel version, what payload, how's the payload invoked and what
> is the observed error?
>
>
> P.
>
> >
> > Given today's environment, this may be a good FAQ.
> >
> > Thanks!
> >
> > Blake McBride
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/nouveau/attachments/20240605/fc4d0afd/attachment.htm>