[Spice-commits] Changes to 'server_multi_client.v5'

[Alon Levy]

New branch 'server_multi_client.v5' available with the following commits:
commit 5813bbe530a88d02c6c773f32a724163d057a843
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Nov 17 22:56:27 2010 +0200

    (temp) client/red_channel: DEBUG: allow no SSL usage (useful for valgrind)

commit 455b7183da5ee8418bf0dad35bbf0f80e95fbb37
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Nov 17 23:07:31 2010 +0200

    (temp) server/reds: DEBUG: add a define for working with no SSL (good for valgrind)

commit e20552fe603a36c1d9fcd529aa6e424af70bf164
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Nov 24 12:47:39 2010 +0200

    (temp) server/reds: use ALLOW_NO_SSL define externally to trigger debug only DEBUG_GENERATE_NULL_KEY

commit c34c202c8a6fe58837445852bace876706e98346
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 13:42:03 2011 +0300

    server/red_worker: DEBUG_CURSORS
    
    Add cursor allocation debugging code that is turned off as long as
    DEBUG_CURSORS is not defined.

commit e7582d3ff92b8ff72011d8f5fec3659632b2177e
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 12:03:28 2011 +0300

    server/red_worker: give each client a different id for caches
    
    We need to have a different id for each client, otherwise we get a reference to
    other client's cache, and of course we don't sync with the new client.
    NOTE: This is quite strange - why are we using a client provided id? only makes sense
    for migration (it already has cache, we get cache by migration). So right now
    we will advance the id if we find it is already there, unless migrate is on, in
    which case we do nothing.
    
    At this point connecting multiple clients with display and cursor channels works.
    previously you could already connect with multiple clients where the second and
    more used only the inputs channel.
    
    You can still crash on lack of Drawables or CursorItems due to speed mismatch between
    the clients (easier to trigger with windows vms).
    This will be fixed in later patches by introducing pipe squashing.
    
    FIXME: Note that the current patchset turns on multiclient support unconditionaly. This must
    be fixed before applying, by adding spice api and using it from qemu (and relevant testers
    will help too).

commit 15b9aceac1fd5c8229f262b0b42d24ee3fb2aaf6
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 12:00:43 2011 +0300

    server/red_worker: on_new_display_channel_client: push ack, cleanup
    
    small cleanup patch, only functional change is sending a set ack message.

commit 55f448e595178e1681ad85407866948358921e39
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 11:31:28 2011 +0300

    server/red_worker: red_current_add_equal: add dcc null checks
    
    check for dcc!=NULL before adding to pipe in several places.

commit 4707283f061284821960e7a217b6361515c942e6
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 09:44:21 2011 +0300

    server/red_worker: add cursor_channel_client_disconnect
    
    makes RED_WORKER_MESSAGE_CURSOR_DISCONNECT_CLIENT disconnect only a
    single client.

commit ed813c37210b16be0d9532c0b49adb3ed6376c9b
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 09:43:35 2011 +0300

    server/red_worker: remove forced disconnect on connect

commit 8bd6aa121675a10aca75c9dc661f264a4942f713
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 13:39:41 2011 +0300

    server/red_worker: split cursor pipe item from cursor item
    
    Required to support multiple clients.
    Also changes somewhat the way we produce PIPE_ITEM_TYPE_LOCAL_CURSOR. Btw,
    I haven't managed to see when we actually produce such an item during my
    tests.
    
    Previously we had a single pipe item per CursorItem, this is impossible
    with two pipes, which happens when we have two clients.

commit ffe2be16a4f52b29cef8953bce273e15b8c0548b
Author: Alon Levy <alevy at redhat.com>
Date:   Fri Apr 22 23:05:08 2011 +0300

    server/red_worker: move free from display_channel_send_item to _release_item

commit afc6bab547889df369ec6479e8f12880d8932c19
Author: Alon Levy <alevy at redhat.com>
Date:   Sat Apr 23 09:30:42 2011 +0300

    server/red_worker: get_streams_timeout: go over all clients

commit 3a60edc059672fcaecf1f5fa335aa26db4bc1783
Author: Alon Levy <alevy at redhat.com>
Date:   Fri Apr 22 23:09:42 2011 +0300

    server/red_worker: copy and free new surfaces after first client

commit c6dc757ba1e4d842288130736b15caa44bb7ba54
Author: Alon Levy <alevy at redhat.com>
Date:   Fri Apr 22 23:04:27 2011 +0300

    server/red_worker: whitespace fixes

commit 6630776ce2ea88d9f8fa9aa7cacfdfd46d027cb9
Author: Alon Levy <alevy at redhat.com>
Date:   Fri Apr 22 14:17:15 2011 +0300

    server/red_worker: start using RENDER_FOREACH
    
    handle_dev_destroy_surface_wait: all clients render state
    handle_dev_destroy_surfaces: clear all surfaces
    handle_dev_destroy_primary_surface: clear all primary copies
    handle_dev_input RED_WORKER_MESSAGE_STOP: all clients
    red_worker_main: call red_handle_streams_timeout for all clients

commit 45a087a50b12e5aae33b19ac6fe49f238ea5c079
Author: Alon Levy <alevy at redhat.com>
Date:   Thu Apr 21 22:21:36 2011 +0300

    server/red_worker: add ref counting to RedDrawable
    
    When we start having multiple clients each
    drawable will be referenced by multiple clients, release happens when all
    clients are done with it.

commit 7b36e80dbfde8c0200c5f31c789c46f39fce38c4
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Apr 13 09:42:35 2011 +0300

    server/red_worker: introduce RedSurfaceReleaseInfo
    
    with multiple clients any surface creation or deletion cleanup needs to
    wait until all the clients have been sent the command, so add reference
    counting to the create and destroy release calls.
    
    Also introduces the SURFACES_FOREACH macro, a bit ugly due to the need
    to update multiple variables and trying to do it without introducing any
    function, so the macro can be used as a for loop head.

commit d40a14d19ecd8c7d41e43120e78540b9ff48bc31
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 10:31:41 2011 +0300

    server/reds: add RedsState.allow_multiple_clients
    
    Currently set by environment variable SPICE_DEBUG_ALLOW_MC (any value means
    to allow multiple connections). Later will be set by spice api from qemu.

commit 8ffb5ccba83b18a12710fcf97527b4dfc52cc3b9
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Apr 13 09:24:31 2011 +0300

    server/red_channel: introduce client ring in RedChannel
    
    main_channel and red_worker had several locations that still accessed rcc
    directly, so they had to be touched too, but the changes are minimal.
    
    Most changes are in red_channel: drop the single client reference in RedChannel
    and add a ring of channels.
    
    We still are missing:
     * surfaces copying / deleting in red_worker when a new client comes/goes
     * remove the disconnect calls on new connections

commit 140207987d3a780ecc91ebe2e451a897eafe6aca
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Apr 13 08:52:44 2011 +0300

    server/red_channel: add pipe_size helpers

commit 153e1de1cf8857deb31a17390b71daafbdc5fda8
Author: Alon Levy <alevy at redhat.com>
Date:   Wed Apr 13 00:35:20 2011 +0300

    server/red_worker: remove more direct access to RedChannelClient.rcc

commit 626ecb4bfd50a51ffe2a0f4a8a485fb1e80e921d
Author: Alon Levy <alevy at redhat.com>
Date:   Tue Apr 12 22:44:41 2011 +0300

    server/red_worker: split display and cursor channels
    
    Introduce DisplayChannelClient, CursorChannelClient, CommonChannelClient.
    don't disconnect channel on client disconnect.
    Move all caches to the ChannelClient's.
    
    Remove reference counting of the channel - need to split this to a separate
    patch. The rationalization for this:
    /* TODO: Why do we reference count the listener?
     * really - what's the lifecycle of a listener? isn't it from peer connection
     * until removal? the drawable's etc. have a valid lifetime outside of this, but
     * isn't the listener's lifetime exactly this?
     * This was meant to allow deletion of the display_channel itself - but now
     * we don't actually ever delete that.
     * */
    
    No new functionality introduced.
    
    TODO: fix crash:
    Still have the possibility of crashing if too many disconnections happen at
    once. Noticed that this happens if we call disconnect with two different channel
    clients pointing to the same channel. should make sure the channel destruction
    happens from a single thread? should only happen from main thread. check
    by printing pthread_self.

commit b7b0b3f9b9f15b92ff005845cf6dc77aa64954ee
Author: Alon Levy <alevy at redhat.com>
Date:   Thu May 5 12:49:35 2011 +0300

    server/red_worker: cleanup

commit 0e044a795ef8501d12b065d8ff7bad2a1ac94524
Author: Alon Levy <alevy at redhat.com>
Date:   Thu May 5 12:46:21 2011 +0300

    server/red_worker: split RedRender from RedWorker
    
    Split the operations list, the surfaces and the streams out from RedWorker
    into a RedRender struct that has a reference to RedWorker.
    This patch is technical, changing signatures to use RedRender *render
    instead of RedWorker *worker, and changing s/worker/worker->surfaces/.
    
    In later patches there will be a multiplicity of RedRender:
    0        | 1
    n >=1    | n
    
    The first instance is embedded in RedWorker. The later are embdedded
    in the DisplayChannelClient which has not been introduced yet.
    
    The surfaces backing store needs to be separate for the main RedRender
    instance and for the rest. For the main it needs to be vram, since
    the guest will read from there after an UPDATE_AREA. For the others
    it will be host memory.
    
    A multiplicity of rendering trees and surfaces makes having a separate
    pipe easy, since we can replace a number of operations in the pipe with
    their equivalent without effecting the other clients. Such a replacement
    may require rendering the operations, so it requires the surfaces to be
    in sync with the tree for that client.
    
    It might be possible to avoid some of this overhead later - in particular,
    possibly having groups of clients associated with the same RedRender, and
    only create a copy on change. The current implementation is the simplest,
    and it doesn't cost anything extra for a single client.
    
    Many function signatures are changed already to use RedRender as the first
    parameter instead of using worker, this is in preparation for the next patches
    where RedRender instances multiply.

commit 78b3578631b7d44509eb39ec3c2d768df576b818
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 23:58:52 2011 +0300

    server/smartcard: support multiple clients
    
    each client supplying a smartcard channel gets it's own smartcard. If
    there are not enough smartcards provided by the server (read: qemu)
    then it will be as though there are none.
    
    currently disabled - later patches that enable smartcard don't make
    this channel available to any but the first client.

commit c7569a9b892131695d3624def8996986891f1cb0
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 23:51:14 2011 +0300

    server/red_tunnel_worker: trivial multi client support
    
    s/TunnelChannel/TunnelChannelClient/
    
    That's about it.  this is probably the wrong way to do it. Not tested
    at all. What do we want, a separate interface per client? same interface
    for all clients? probably the later. This doesn't do that. Not tested,
    so probably doesn't even work.
    
    changes red_channel_pipe_item_is_linked to red_channel_client_pipe_item_is_linked,
    since tunnel channel is the only user, must be done in patch to not break compilation.

commit 27da64ab9ac57f759e6531c125292103d8156720
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 22:51:54 2011 +0300

    server/inputs_channel: support multiple clients
    
    from server events are broadcast - leds change. The rest is client
    to server, so it is just passed on.

commit 27c7530b8ad7a7a006bd03052ebc7f8845e3a7e2
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 22:29:58 2011 +0300

    server/main_channel: support multiple clients
    
    The main channel deals with connecting new clients, announcing mouse mode
    changes, and the agent channel. The implementation is currently done without
    any changes to the protocol, so everything has to be either broadcast or
    to a specific client.
    
     channels list - specific client
     mouse mode - broadcast
     agent - broadcast
     notify - broadcast (should have two modes, and use the appropriate)
    
    Notable TODOs:
     * migration testing
     * agent tokens are wrongly sent (or did I fix that? check)

commit 6d87d84c5992f8feb21badaf2b8f37a0a996d777
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 21:44:16 2011 +0300

    server/red_channel: introduce pipes functions
    
    Introduce functions to add (via producer method) the same item to multiple
    pipes, all for the same channel.
    
    Note: Right now there is only a single channel, but the next patches will do the
    per-channel breakdown to channel and channel_client before actually introducing
    a ring in RedChannel, this makes it easier to make smaller changes - the
    channel->rcc link will exist until removed in the ring introducing patch.

commit a5cd2a22d454d30f17ca6d5ad07e72b29d406618
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 21:42:05 2011 +0300

    server/red_channel: ignore error if already shutdown
    
    on red_channel_peer_on_incoming_error, if we are already shutdown, do not
    call the channel's error handler. Since the channel has been shutdown, we
    assume this is a second or later error, and handling has already occured.

commit dbca82642f0e468b80edd1cb2dd455c57a40758f
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 20:58:01 2011 +0300

    server/main_channel: move connection_id from reds
    
    Expose additional api to find a client given a connection_id. The connection_id
    is first set when the first channel connects, which is the main channel.
    It could also be kept in the RedClient instead, not sure.
    
    TODO:
     multiple todo's added for multiclient handling. I don't remember why
     I wrote them exactly, and besides if I did any migration tests. So: TODO.

commit cf6ca033d068e8217e7e4b8b423a6581b7893895
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 18:54:51 2011 +0300

    server/main_channel: move ping here from reds.
    
    cleanup only. Note that the ping function is half used since the opt parameter
    stopped being called with anything but NULL, should be returned at some point,
    specifically when we drop the 250kbyte ping on start and do a continuous check
    for latency and bandwidth.
    
    See:
     81945d897 - server: add new vd interface QTerm2Interface, Yaniv Kamay
      introducing the usage of ping with a non NULL opt
     3f7ea8e7a - zap qterm interfaces, Gerd Hoffman
      removing it

commit b4d62d2932c4f78737f9e5ffa1681d2598d3ec0a
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 21:11:20 2011 +0300

    server/main_channel: move latency and bitrate to channel client
    
    They were globals before. This introduces api for other channels
    to query the low bandwidth status. The queries themselves are still done
    from the wrong context (channel and not channel client) but that's because
    the decoupling of channel and channel client will be done in the following
    patches.
    
    Note that snd_worker.c got two copied function declarations that belong to
    main_channel.h but can't be easily dragged into snd_worker.c since it still
    uses it's own RedChannel struct.

commit 6befd0832e92f0e1861cbb33e4d842c7a02227c3
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 24 10:32:10 2011 +0300

    server/reds: add concept of secondary channels
    
    Secondary channels are those that don't support multiple clients. The
    support added in this patch just doesn't let the second or more connected
    client receive the unsupported channels in the channels list sent by the
    server to the client. This doesn't handle the situation where:
    
    client A connects (gets all channels)
    client B connects (gets supported multiple client channels)
    client A disconnects (Suboptimal 1: B doesn't get new channels at this point)
    client C connects (Suboptimal 2: C doesn't get the full list of channels, but
    the partial one)
    
    Specifically the channels that only support a single client are:
     sound (both playback and record channels)
     smartcard
     tunnel

commit c13babc7dd5ceb8f0b235161facb2647d6cdcba4
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Apr 11 12:44:00 2011 +0300

    server: Add RedClient
    
    That means RedClient tracks a ring of channels. Right now there will be only
    a single client because of the disconnection mechanism - whenever a new
    client comes we disconnect all existing clients. But this patch adds already
    a ring of clients to reds.c (stored in RedServer).
    
    There is a known problem handling many connections and disconnections at the
    same time, trigerrable easily by the following script:
    
    export NEW_DISPLAY=:3.0
    
    Xephyr $NEW_DISPLAY -noreset &
    for ((i = 0 ; i < 5; ++i)); do
        for ((j = 0 ; j < 10; ++j)); do
            DISPLAY=$NEW_DISPLAY c_win7x86_qxl_tests &
        done
        sleep 2;
    done
    
    I fixed a few of the problems resulting from this in the same patch. This
    required already introducing a few other changes:
     * make sure all removal of channels happens in the main thread, for that
     two additional dispatcher calls are added to remove a specific channel
     client (RED_WORKER_MESSAGE_CURSOR_DISCONNECT_CLIENT and
     RED_WORKER_MESSAGE_DISPLAY_DISCONNECT_CLIENT).
     * change some asserts in input channel.
     * make main channel disconnect not recursive
     * introduce disconnect call back to red_channel_create_parser
    
    The remaining abort is from a double free in the main channel, still can't
    find it (doesn't happen when running under valgrind - probably due to the
    slowness resulting from that), but is easy to see when running under gdb.

commit 0ef1b8c6f0ee412fd0a7fff10dbb8ee249647c48
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 21:51:58 2011 +0300

    server/main_channel+reds: make main_channel_init return MainChannelClient
    
    This makes it easier to introduce RedClient in the next patch.

commit 6b1f61afd3f8e4e627efe4d950547033e7fa1ad1
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 20:17:39 2011 +0300

    server/red_worker: release PIPE_ITEM_TYPE_VERB in display channel
    
    We send a SPICE_MSG_DISPLAY_MARK verb.

commit 7415248e16fa53cb52c2d04b94c655d8ae540d58
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 20:03:11 2011 +0300

    server/red_client: clear pipe on disconnect

commit 0763b9685714c2ce8e387794e3c25770b069f3da
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 20:02:50 2011 +0300

    server/red_channel: workaround for fast client disconnect bug (TODO - real fix)

commit a86021ba86c0d37b29c858bacbacccc14b7162e5
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 18:18:11 2011 +0300

    server/main_channel: use MainChannel in sig
    
    use MainChannel* instead of Channel* for a many functions in main_channel.h
    (affects main_channel.c and reds.c).
    
    some one liner fixes are hidden in here too.

commit 1972d214f7ac4634176f30e1d935e363226b909d
Author: Alon Levy <alevy at redhat.com>
Date:   Sat Apr 2 12:54:52 2011 +0300

    server: move pipe from RedChannel to RedChannelClient
    
    Another cleanup patch, no change to behavior (still one client, and it
    disconnects previous client if any).
    
    The implementation for multiple client is straightforward: the pipe
    remains per (channel,client) pair, so it needs to move from the RedChannel
    that to RedChannelClient. Implementation using a single pipe with multiple
    consumers (to reflect different latencies) doesn't fit well with pipe rewriting
    that is used by the display channel. Additionally this approach is much simpler
    to verify. Lastly it doesn't add considerable overhead (but see the display
    channel changes in a later patch for a real place to rethink).
    
    This patch is just technical, changing signatures to reflect the first
    argument (oop style) so red_channel becomes red_channel_client. Some places
    may seem odd but they should be fixed with later comits where the channels
    grow to support multiple clients.
    
    Sound (playback/record) channels are the only ones not touched - this is
    consistent with previous patches, since they have been left out of the
    RedChannel refactoring.  That is left as future work. (note that they don't use
    a pipe, which was the reason for not refactoring).

commit b72084a38789655802a99da98407a3abf7b12982
Author: Alon Levy <alevy at redhat.com>
Date:   Mon Nov 22 16:56:58 2010 +0200

    server/red_worker: introduce {display,cursor}_is_connected
    
    Instead of checking for worker->{display,cursor}_channel directly.

commit 9d46c7e5f54d3395b00d96db4f799ea6732920a0
Author: Alon Levy <alevy at redhat.com>
Date:   Sat Nov 13 13:23:02 2010 +0200

    server/red_channel (all): introduce RedChannelClient
    
    This commit adds a RedChannelClient that now owns the stream connection,
    but still doesn't own the pipe. There is only a single RCC per RC
    right now (and RC still means RedChannel, RedClient will be introduced
    later). All internal api changes are in server/red_channel.h, hence
    the need to update all channels. red_worker.c is affected the most because
    it makes use of direct access to some of RedChannel still.
    
    API changes:
    
     1. red_channel_client_create added.
      rec_channel_create -> (red_channel_create, red_channel_client_create)
     2. two way connection: rcc->channel, channel->rcc (later channel will
      hold a list, and there will be a RedClient to hold the list of channels
      per client)
     3. seperation of channel disconnect and channel_client_disconnect

commit 7d5e7d258003d0544f04ccc956ce02330bb4e610
Author: Alon Levy <alevy at redhat.com>
Date:   Thu Mar 24 18:17:17 2011 +0200

    server/red_worker: drop red_pipe_add_tail, use red_channel_pipe_add_tail
    
    The only difference between them being that the later also does a push.
    I don't believe that to be a problem, but if it does I can always introduce
    a push'less version.

commit 93c60ed2b8c961c695435a4fe53d998e1103e545
Author: Alon Levy <alevy at redhat.com>
Date:   Thu Mar 24 16:41:02 2011 +0200

    server/red_channel: renames to use _proc postfix consistently
    
    rename types - we use _proc suffix mostly to indicate function pointer types,
    use it for some function pointers that were missing it.
    
    s/channel_handle_migrate_flush_mark/channel_handle_migrate_flush_mark_proc/
    s/channel_handle_migrate_data_get_serial/channel_handle_migrate_data_get_serial_proc/
    s/channel_handle_migrate_data/channel_handle_migrate_data_proc/

commit d529b583b074a4423677a1a869d4e2241f71c9fe
Author: Alon Levy <alevy at redhat.com>
Date:   Sun Apr 10 16:21:15 2011 +0300

    server/main_channel.c: set NET_TEST_STAGE_INVALID=0 explicitly (TODO - the rest)
    
    should change all the enums that don't explicitly set their first value to 0.

commit 1d9a33216d80d990e1bba656da2126a6b7859f6f
Author: Alon Levy <alevy at redhat.com>
Date:   Tue Apr 12 14:27:51 2011 +0300

    server/red_worker: fix typo (lats_send_time)

commit 1527b2cf0b61d7c0bab138445efb6136c4b3be81
Author: Alon Levy <alevy at redhat.com>
Date:   Tue Apr 12 14:24:10 2011 +0300

    server/red_worker: make stat_now static

commit 2d28636ede51f2228b45472383bbc6b3641491de
Author: Alon Levy <alevy at redhat.com>
Date:   Sun May 8 14:32:22 2011 +0300

    server/tests: show port to connect to