[Spice-devel] aspeed: video frames pass-through

[Anton D. Kachalov]

Hello,

sorry for the previous HTML-only message.

I'm trying to add support for Aspeed SoC (AST2150/2300/2400) raw video streaming to Spice (this SoC might be found in the most server's BMC). It is a kind of a compressed data with screen change detection algo. Right after Video streaming started, first frame is the full frame, continues frames might be NULL (no video has changed) or very small piece of changed part (without decoding you don't actually now which part has changed). Image in YUV444 or YUV420 format compressed with Aspeed-specific Vector Quantization lossless algo (as an option AST VideoEngine may use JPEG compression). Spice server haven't to transcode the video stream.

I've started with Francois Gouget's https://github.com/fgouget/spice repo with GStreamer support. It were my first steps with Spice. I've added one more encoder: "aspeed" and one more image type: "AST" and tries to pass fake image update command from the server command's queue (https://github.com/ya-mouse/spice/blob/gst/server/tests/ast_base.c ) based on test_display_no_ssl to trigger stream encoding. From the server side I stucked at "pixman" decoding at "draw_copy", so I've just skip copy/blitting on semi-invalid image and read actual data from the temporary file during `aspeed_encoder_encode_frame' call. It's very-very ugly, but I need just a working concept.

It works somehow on x86 host, but hangs on ARM after the first frame received. Here is the log from the server running on ARM:  http://pastebin.com/TJjW8MYC
Sometimes (usually) client gets invalid image warn: http://pastebin.com/eCBKT0cz

After a week spent on Spice integration, I realize that I need a "virtual" surface on the server-side with fake "ops" (especially, image blitting/draw) which support only AST compressed data just to pass it through (image marshalling) to the client. All decoding should be done on the client side (like it done right now with "aspeed decoder"), but need ORing with further frames' updates (another kind of image processing like a DRAW_COPY). All further received frames should be draw on top of first frame but not to redraw entire surface (clipped rect draw?). This is what I would like to do next from scratch without any encoder-like workarounds.

Any suggetsions and comments are welcome!

BTW, I've changed a bit "spice*.proto" to eliminate Alignment trapping on ARM. There were a lot of unaligned data access on received data. IMHO, using 16/32bit fields instead of 8/16 when applicable do not overload network so much.

Links to repo:

  https://github.com/ya-mouse/spice/tree/gst
  https://github.com/ya-mouse/spice-common/tree/gst
  https://github.com/ya-mouse/spice-protocol/tree/gst
  https://github.com/ya-mouse/spice-gtk

Attached file is a dump for testing purpose (just put as /dev/videocap to test server/client):

  http://lists.freedesktop.org/archives/spice-devel/attachments/20151027/75769c73/attachment-0001.bin

--
Anton D. Kachalov