[Spice-devel] aspeed: video frames pass-through

[Christophe Fergeau]

On Tue, Oct 27, 2015 at 08:42:03PM +0300, Anton D. Kachalov wrote:
> Hello,
> 
> sorry for the previous HTML-only message.
> 
> I'm trying to add support for Aspeed SoC (AST2150/2300/2400) raw video
> streaming to Spice (this SoC might be found in the most server's BMC).
> It is a kind of a compressed data with screen change detection algo.
> Right after Video streaming started, first frame is the full frame,
> continues frames might be NULL (no video has changed) or very small
> piece of changed part (without decoding you don't actually now which
> part has changed). Image in YUV444 or YUV420 format compressed with
> Aspeed-specific Vector Quantization lossless algo (as an option AST
> VideoEngine may use JPEG compression). Spice server haven't to
> transcode the video stream.

If I understand correctly, the encoding/decoding is all done in
hardware?

> 
> I've started with Francois Gouget's https://github.com/fgouget/spice
> repo with GStreamer support. It were my first steps with Spice. I've
> added one more encoder: "aspeed" and one more image type: "AST" and
> tries to pass fake image update command from the server command's
> queue
> (https://github.com/ya-mouse/spice/blob/gst/server/tests/ast_base.c )
> based on test_display_no_ssl to trigger stream encoding. From the
> server side I stucked at "pixman" decoding at "draw_copy", so I've
> just skip copy/blitting on semi-invalid image and read actual data
> from the temporary file during `aspeed_encoder_encode_frame' call.
> It's very-very ugly, but I need just a working concept.
> 
> It works somehow on x86 host, but hangs on ARM after the first frame
> received. Here is the log from the server running on ARM: 
> http://pastebin.com/TJjW8MYC
> Sometimes (usually) client gets invalid image warn:
> http://pastebin.com/eCBKT0cz

Hmm, not very familiar with ARM, and not sure this has seen a lot of
testing. Maybe getting a backtrace would shed more light?

> After a week spent on Spice integration, I realize that I need a
> "virtual" surface on the server-side with fake "ops" (especially,
> image blitting/draw) which support only AST compressed data just to
> pass it through (image marshalling) to the client. All decoding should
> be done on the client side (like it done right now with "aspeed
> decoder"), but need ORing with further frames' updates (another kind
> of image processing like a DRAW_COPY). All further received frames
> should be draw on top of first frame but not to redraw entire surface
> (clipped rect draw?). This is what I would like to do next from
> scratch without any encoder-like workarounds.
> 
> Any suggetsions and comments are welcome!

Hmm, I guess hacking the stream detection code so that it thinks it
should always streamed the whole screen rather than a small part of it
counts as a workaround?

> 
> BTW, I've changed a bit "spice*.proto" to eliminate Alignment trapping
> on ARM. There were a lot of unaligned data access on received data.
> IMHO, using 16/32bit fields instead of 8/16 when applicable do not
> overload network so much.

Main problem with that is that this breaks the wire protocol, older
clients would not be able to talk to newer servers and vice-versa.
It would be interesting to have more details about which messages are
causing alignment trapping issues. From a quick look, everything in
spice-common/common/messages.h should be fine / easy to fix as this is
read from the network through the generated C demarshallers.
I'd expect more issues from the link messages, and the vdagent/qxl
messages where the parsing is probably done by hand.

> Links to repo:
> 
>   https://github.com/ya-mouse/spice/tree/gst
>   https://github.com/ya-mouse/spice-common/tree/gst
>   https://github.com/ya-mouse/spice-protocol/tree/gst
>   https://github.com/ya-mouse/spice-gtk

Ah, thanks for the links!

Christophe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20151103/579ee1fe/attachment.sig>