[Spice-devel] Tripping assertions in spice-common/common/ring.h

[Nahum Shalman]

I need help tracking this down:
((null):66324): SpiceWorker-ERROR **: 
../spice-common/common/ring.h:56:ring_add: assertion `ring->next != NULL 
&& ring->prev != NULL' failed

I've been seeing this from time to time on my VMs.
When it happens, the UI stops being responsive, but QEMU doesn't crash.

I've avoided reporting it because I was using a non-release version of QEMU.
I've recently had help getting the Solaris(ish) patches applied to QEMU 
1.1.0 and using that, I still tripped the assert.
I launched a VM (Fedora 16), opened a browser and went to a site that 
has some weird overly fancy graphics and poked around on it for a little 
while until everything locked up.
And there in the log was my old friend the assertion failure in ring.h

I'm pretty sure the culprit is somewhere in the Spice code base.

<I slept on the problem, and picked it up this morning with dtrace>

In the end I added a call to spice_backtrace to the spice_assert macro, 
and while the backtrace it provides wasn't all that helpful
("main_channel_handle_parsed: agent start" was all I saw in the logs), 
it gave me a probe point for dtrace.


Here's the dtrace script I used to hunt down the error:
------------------------------------------------------------
#!/usr/sbin/dtrace -s

#pragma D option quiet

pid$target:libspice-server.so.1:ring_add:entry
{
         self->follow = 1;
}

pid$target:libspice-server.so.1:ring_add:return
/self->follow >= 1/
{
         self->follow = 0;
/*      printf("ring_add returned cleanly\n"); */
}

pid$target:libspice-server.so.1:spice_backtrace:entry
/self->follow >= 1/
{
         printf("called spice_backtrace from ring_add\n");
         ustack();
}
------------------------------------------------------------

And this is the output I got when the VM decided to hang:

called spice_backtrace from ring_add

               libspice-server.so.1`spice_backtrace
               libspice-server.so.1`ring_add+0x2e
               libspice-server.so.1`ring_add_after+0x23
               libspice-server.so.1`__current_add_drawable+0x61
               libspice-server.so.1`red_current_add_equal+0xee
               libspice-server.so.1`red_current_add+0x18d
               libspice-server.so.1`red_current_add_qxl+0xa7
               libspice-server.so.1`red_process_drawable+0x223
               libspice-server.so.1`red_process_commands+0x231
               libspice-server.so.1`handle_dev_oom+0x13a
               libspice-server.so.1`dispatcher_handle_single_read+0x14b
               libspice-server.so.1`dispatcher_handle_recv_read+0x19
               libspice-server.so.1`handle_dev_input+0x32
               libspice-server.so.1`red_worker_main+0x287
               libc.so.1`_thrp_setup+0x83
               libc.so.1`_lwp_start

called spice_backtrace from ring_add

               libspice-server.so.1`spice_backtrace
               libspice-server.so.1`spice_logv+0x1e7
               libspice-server.so.1`spice_log+0xbc
               libspice-server.so.1`ring_add+0x59
               libspice-server.so.1`ring_add_after+0x23
               libspice-server.so.1`__current_add_drawable+0x61
               libspice-server.so.1`red_current_add_equal+0xee
               libspice-server.so.1`red_current_add+0x18d
               libspice-server.so.1`red_current_add_qxl+0xa7
               libspice-server.so.1`red_process_drawable+0x223
               libspice-server.so.1`red_process_commands+0x231
               libspice-server.so.1`handle_dev_oom+0x13a
               libspice-server.so.1`dispatcher_handle_single_read+0x14b
               libspice-server.so.1`dispatcher_handle_recv_read+0x19
               libspice-server.so.1`handle_dev_input+0x32
               libspice-server.so.1`red_worker_main+0x287
               libc.so.1`_thrp_setup+0x83
               libc.so.1`_lwp_start

Does this give any insight into what I might be suffering from?

-Nahum

P.S. In addition to those errors, I've also seen some of:
((null):7636): SpiceWorker-ERROR **: 
../spice-common/common/ring.h:83:ring_remove: assertion `item->next != 
NULL && item->prev != NULL' failed
((null):8838): SpiceWorker-ERROR **: 
../spice-common/common/ring.h:84:ring_remove: assertion `item->next != 
item' failed