[Spice-devel] Fwd: Streaming video command line

Wed May 11 14:17:29 PDT 2011

On Wed, May 11, 2011 at 07:38:27PM +0200, Andrea Celestino wrote:
> Hi, I have other few simple questions:
> 
> 1)  file: red_worker.c, function: red_worker_main():
> What does evt_listener->action(evt_listener, event->events) do? Does it call
> handle_dev_input() function? Is it called every time an event occurs on
> socket?

This is the callback from the event loop. red_worker's entry function is
red_worker_main, it has an epoll based event loop implementation (i.e. it
doesn't use SpiceCoreInterface, which uses qemu's loop, since this is a
different thread).

> 
> 2)file: red_worker.c
> In red_worker_main() there is a call to red_process_commands().What type of
> command does this function handle?
> 

The driver writes graphics commands to the command ring buffer, referred to in
general as "commands" for short, since those are the main things we are
interested with.

red_process_commands reads from the command ring, and handles each command.

> 3) What is a RedDrawable?
> 

It holds a single command after it has been converted from the device format
to the spice-server representation. This is here for maintaining ABI compatibility,
i.e. we can change the spice-server data structures independently from driver's structs
(QXLDrawable, defined in spice-protocol/spice/qxl_dev.h).

a RedDrawable is basically a copy of QXLDrawble, the only thing not copied is the raw
bitmaps (used in several places, opaque copy data, brushes, qmask - git grep red_get_image)
that is still accessed from the device memory.

> Thanks a lot
> 
> 2011/5/5 Alon Levy <alevy at redhat.com>
> 
> > On Thu, May 05, 2011 at 09:45:11AM +0200, Andrea Celestino wrote:
> > > Ok, in this way the option -streaming-video=[all|off|filter] will be set
> > > before any workers are started and I can't send message to change the
> > > streaming video option.... In which way can I change streaming-video
> > option?
> >
> > The change done from the command line affects all the workers started
> > afterwards.
> > If what you mean is that you want to change during run time, we used to
> > have qemu
> > monitor commands for that, but it seems they are not in upstream. Can be
> > added
> > back easily.
> >
> > >
> > > p.s. Can I send email to your email address directly or you prefer the
> > > mailing list?
> >
> > I prefer the mailing list, but I answer both :) The preferred way is to the
> > mailing list, and cc me.
> >
> > >
> > > 2011/5/4 Alon Levy <alevy at redhat.com>
> > >
> > > > On Wed, May 04, 2011 at 05:00:49PM +0200, Andrea Celestino wrote:
> > > > > Sorry for the forwarded message.
> > > > > About 3, is it correct that it doesn't send
> > > > > RED_WORKER_MESSAGE_SET_STREAMING_VIDEO? The variable "now" is NULL so
> > the
> > > > > program doesn't go inside the while loop and it doesn't send the
> > message
> > > > and
> > > > > data..
> > > >
> > > > yes, it is fine (looked at the source finally) - the dispatchers are
> > the
> > > > list of
> > > > already active workers, if this parameter is set before any workers are
> > > > started,
> > > > then none needs to be updated, they will receive the correct value when
> > > > they are
> > > > created. Follow the initialization of them (better to follow where this
> > > > variable
> > > > is actually used, and you'll see that it is received in the process of
> > > > creating
> > > > a new red worker thread). There is a worker thread for every card, so
> > for a
> > > > single
> > > > qxl pci card you will have one worker, the dispatcher is used to talk
> > to it
> > > > over
> > > > a pipe.
> > > >
> > > > >
> > > > > 2011/5/4 Alon Levy <alevy at redhat.com>
> > > > >
> > > > > > On Wed, May 04, 2011 at 03:27:52PM +0200, Andrea Celestino wrote:
> > > > > > > Hi,
> > > > > > > I have gathered other questions about spice and how it handles
> > > > streaming
> > > > > > > video. I think you can help me:
> > > > > > >
> > > > > > > 1)File red_worker.c.
> > > > > > > I would like to have more information about these members of the
> > > > struct
> > > > > > > RedWorker :
> > > > > > >
> > > > > > >    - RedSurface surfaces[NUM_SURFACES],
> > > > > > one for each surface. The first is the primary surface, which is
> > the
> > > > > > equivalent
> > > > > > of the framebuffer. Each RedSurface contains a canvas (i.e. pixmap
> > > > > > implemented / opengl
> > > > > > implemented) and the operations that are active on that surface
> > (i.e.
> > > > the
> > > > > > state
> > > > > > the client meant to have at any point would be taking the canvas
> > and
> > > > > > applying all the
> > > > > > surface kept operations from oldest to newest).
> > > > > >
> > > > > > >    - Ring current_list,
> > > > > > List of operations I think - drawables that are being used.
> > > > > >
> > > > > > >    - _Drawable drawables[NUM_DRAWABLES];
> > > > > > Preallocated drawables. Just used instead of mallocing everything
> > when
> > > > > > required.
> > > > > >
> > > > > > >    - Stream streams_buf[NUM_STREAMS];
> > > > > > Same for streams.
> > > > > >
> > > > > > >
> > > > > > > 2)Function red_worker_main(), what does
> > *red_handle_streams_timout()*
> > > > do?
> > > > > > Don't recall.
> > > > > >
> > > > > > >
> > > > > > > 3)When you run qemu with option *-streaming_video=off* for
> > example,
> > > > the
> > > > > > > function *spice_server_set_streaming_video() *in reds.c, calls *
> > > > > > > red_dispatcher_on_sv_change(). *
> > > > > > > **In this function there is a while loop on RedDispatcher *now.
> > > > > > Dispatcher
> > > > > > > is created in red_dispatcher_init_function, that it's called
> > later,
> > > > in
> > > > > > this
> > > > > > > way *now *is NULL and it doesn't enter in the while loop. As a
> > > > > > consequence
> > > > > > > of this,  *red_dispatcher_on_sv_change() *doesn't send
> > > > > > > *RED_WORKER_MESSAGE_SET_STREAMING_VIDEO
> > > > > > > *message.
> > > > > > >
> > > > > > > Did I make myself clear?
> > > > > >
> > > > > > Not sure about 3 - what is the question?
> > > > > >
> > > > > > > Thanks very much.
> > > > > > >
> > > > > > >
> > > > > > > 2011/4/21 Alon Levy <alevy at redhat.com>
> > > > > > >
> > > > > > > > On Thu, Apr 21, 2011 at 10:22:15AM +0200, Andrea Celestino
> > wrote:
> > > > > > > > > Thanks.
> > > > > > > > > Now I am trying to study the code I am interested in.
> > > > > > > > > Unfortunately, the file red_worker.c is very huge and there
> > are
> > > > not
> > > > > > any
> > > > > > > > > comments that can help me.
> > > > > > > > >
> > > > > > > > > I would like to know:
> > > > > > > > > 1- which parts/functions of red_worker.c are video related
> > stuff?
> > > > > > > > A lot. Seriously - this requires a little more checking then I
> > can
> > > > give
> > > > > > you
> > > > > > > > off
> > > > > > > > the top of my head. Questions 2 and 3 are more specific, so
> > easier
> > > > to
> > > > > > > > answer.
> > > > > > > > I would seriously suggest trying systemtap or just adding some
> > > > prints
> > > > > > to
> > > > > > > > see
> > > > > > > > what is going on. Start from red_process_drawable.
> > > > > > > >
> > > > > > > > > 2- You said:
> > > > > > > > >  "determine if a given draw operation (has to be an opaque
> > copy
> > > > iirc)
> > > > > > is
> > > > > > > > >  repeated to the same area."
> > > > > > > > > What do you mean with "opaque copy iirc"?
> > > > > > > >
> > > > > > > > iirc - if I recall correctly, meaning I'm not sure.
> > > > > > > > opaque copy - an operation that copies a bitmap to a surface.
> > The
> > > > video
> > > > > > > > streaming
> > > > > > > > is done based on seeing repeated operations that copy a bitmap
> > to
> > > > the
> > > > > > same
> > > > > > > > surface, always surface 0 (that is the primary surface, we
> > ignore
> > > > > > repeated
> > > > > > > > copies
> > > > > > > > to non primary surfaces).
> > > > > > > > opaque - meaning the destination bits are overwritten by the
> > source
> > > > > > bits,
> > > > > > > > no binary
> > > > > > > > operation is done (xor/and/or).
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 3- There are a red worker thread for each QXL device
> > instance.
> > > > How
> > > > > > many
> > > > > > > > QXL
> > > > > > > > > device instance are there?
> > > > > > > >
> > > > > > > > One per device. But you can ignore the support for multiple
> > devices
> > > > for
> > > > > > > > this,
> > > > > > > > so just one if you run a single device ("-vga qxl").
> > > > > > > >
> > > > > > > > > 4- Which function does red_worker.c use to detect video
> > > > streaming?
> > > > > > > > >
> > > > > > > > interesting functions are:
> > > > > > > > __red_is_next_stream_frame
> > > > > > > > anything with ->stream
> > > > > > > > red_attach_stream
> > > > > > > > anything with mjpeg
> > > > > > > > red_current_add, the bottom part (that checks for OPAQUE and
> > calls
> > > > > > > > red_use_stream_trace)
> > > > > > > >
> > > > > > > > > Thanks for any helps you can give me
> > > > > > > > Sure, please take a look at the above points and try what I
> > > > suggested.
> > > > > > > >
> > > > > > > > Alon
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 2011/4/15 Alon Levy <alevy at redhat.com>
> > > > > > > > >
> > > > > > > > > > On Thu, Apr 14, 2011 at 03:20:40PM +0200, Andrea Celestino
> > > > wrote:
> > > > > > > > > > > I would like to improve the streaming video performance
> > in
> > > > Spice,
> > > > > > do
> > > > > > > > you
> > > > > > > > > > > think that is possible to improve it? In what way?
> > > > > > > > > > > Are there documents that describes in detail how spice
> > works
> > > > with
> > > > > > > > video?
> > > > > > > > > >
> > > > > > > > > > The best I can offer in terms of documentation is the pdfs
> > in
> > > > the
> > > > > > > > documents
> > > > > > > > > > section of spice-space.org, I imagine you are looking for
> > > > > > something
> > > > > > > > more,
> > > > > > > > > > sorry to disappoint.
> > > > > > > > > >
> > > > > > > > > > The code you are interested in is server/red_worker.c, but
> > > > that's
> > > > > > not
> > > > > > > > very
> > > > > > > > > > helpful since that file is huge. Would be nice to split the
> > > > video
> > > > > > > > related
> > > > > > > > > > stuff to a separate file.
> > > > > > > > > >
> > > > > > > > > > worker->streaming_video - determines how we handle
> > streaming
> > > > video
> > > > > > > > > >
> > > > > > > > > > git grep stream -- server/red_worker.c
> > > > > > > > > >
> > > > > > > > > > I can give you better pointers if you ask more specific
> > > > questions,
> > > > > > but
> > > > > > > > > > doing
> > > > > > > > > > the documentation myself right now.. too much work.
> > > > > > > > > >
> > > > > > > > > > overview of what spice does:
> > > > > > > > > >  * determine if a given draw operation (has to be an opaque
> > > > copy
> > > > > > iirc)
> > > > > > > > is
> > > > > > > > > >  repeated to the same area.
> > > > > > > > > >  * if so, starts a mjpeg encoder and sends the stream to
> > the
> > > > client
> > > > > > > > > >
> > > > > > > > > > how to improve this (some is on the Features page in
> > > > spice-space):
> > > > > > > > > >  * learn to do various video acceleration apis (windows has
> > it,
> > > > > > linux
> > > > > > > > has
> > > > > > > > > > it)
> > > > > > > > > >  * this is by far the best way to go regarding performance,
> > > > unless
> > > > > > you
> > > > > > > > are
> > > > > > > > > >  willing to sacrifice cpu for bandwidth in which case I
> > guess
> > > > you
> > > > > > would
> > > > > > > > > > reencode.
> > > > > > > > > >  * otoh this is a pretty large change to multiple parts of
> > > > spice:
> > > > > > > > > >   * you need to update the driver for each platform you
> > choose
> > > > to
> > > > > > > > support,
> > > > > > > > > > update the protocol, teach server and client
> > > > > > > > > >  * it's definitely on the todo list, but no date set.
> > > > > > > > > >  * replace mjpeg with a different encoder
> > > > > > > > > >  * if it's just a better performing encoder, clear win.
> > > > > > > > > >  * otherwise you get the usual cpu/bw tradeoff.
> > > > > > > > > >  * or you could try to learn which encoder fits which
> > stream
> > > > (based
> > > > > > on
> > > > > > > > some
> > > > > > > > > >  metric, could be the performance of the encoders
> > themselves)
> > > > > > > > > >   * just throwing an idea.
> > > > > > > > > >
> > > > > > > > > > Alon
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 2011/4/11 Alon Levy <alevy at redhat.com>
> > > > > > > > > > >
> > > > > > > > > > > > On Mon, Apr 11, 2011 at 03:48:08PM +0200, Andrea
> > Celestino
> > > > > > wrote:
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > I have installed Spice and now I'm trying to use it.
> > I'm
> > > > > > > > interested
> > > > > > > > > > in
> > > > > > > > > > > > how
> > > > > > > > > > > > > spice manage streaming video. I read that there is
> > the
> > > > option
> > > > > > > > > > > > >
> > > > > > > > > > > > > -streaming-video=[all|off|filter]
> > > > > > > > > > > > >
> > > > > > > > > > > > > Can you explain me the difference between these three
> > > > option?
> > > > > > > > > > > > > Which source file read the command line?
> > > > > > > > > > > > > Why I have tried to play a video in a player but i
> > have
> > > > not
> > > > > > > > notice
> > > > > > > > > > any
> > > > > > > > > > > > > difference between the two options 'off' and
> > 'filter'?
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Look at
> > > > > > > > > >
> > > > http://cgit.freedesktop.org/spice/spice/tree/server/red_worker.c
> > > > > > > > > > > >
> > > > > > > > > > > > search for STREAM_VIDEO_FILTER
> > > > > > > > > > > >
> > > > > > > > > > > > The switch toggles modes for creating streams:
> > > > > > > > > > > >  off - never create
> > > > > > > > > > > >  all/filter - create either based on smallest size, or
> > just
> > > > > > based
> > > > > > > > on
> > > > > > > > > > > >  number of consecutive opaque copies to same area.
> > > > > > > > > > > >
> > > > > > > > > > > > > Thanks very much for the help.
> > > > > > > > > > > >
> > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > Spice-devel mailing list
> > > > > > > > > > > > > Spice-devel at lists.freedesktop.org
> > > > > > > > > > > > >
> > > > http://lists.freedesktop.org/mailman/listinfo/spice-devel
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > Spice-devel mailing list
> > > > > > > > > > > Spice-devel at lists.freedesktop.org
> > > > > > > > > > >
> > http://lists.freedesktop.org/mailman/listinfo/spice-devel
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >