androidmedia encoder problems

Sat Jan 25 06:27:41 PST 2014

Hi,

I finally had time to look again, but I am really not familiar enough with all of this.
At the beginning of the stream it receives INFO_OUTPUT_FORMAT_CHANGED, the format seems to contains csd-0 and csd-1 fields (see log bellow), but when I look to the values it seems to always be empty or unfilled.
I tried to reassemble avcc, and put its value in codec_data of caps as a GStBuffer, but it does not give anything (I think again because csd-0 and csd-1 values seem to be empty).
We can see also that it receives a buffer of size 21 with flag 0x00000002 (BUFFER_FLAG_CODEC_CONFIG). Is is the size of csd-0 + csd-1 buffers ... ? but I can't see any values also there.

Here is a piece of log of the start of the stream to show again what happens ...

01-25 15:11:35.508: D/GStreamer+amcvideoenc(15499): 0:00:14.691152507 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1190:gst_amc_video_enc_set_format:<amcvideoenc-omxsecavcencoder0> Setting new caps video/x-raw, width=(int)480, height=(int)800, framerate=(fraction)30/1, format=(string)NV12
01-25 15:11:35.508: D/GStreamer+amcvideoenc(15499): 0:00:14.691317632 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1234:gst_amc_video_enc_set_format:<amcvideoenc-omxsecavcencoder0> picking an output format ...
01-25 15:11:35.513: D/GStreamer+amcvideoenc(15499): 0:00:14.693541257 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1244:gst_amc_video_enc_set_format:<amcvideoenc-omxsecavcencoder0> chose caps video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ], parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string){ 1, 1b, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4 }
01-25 15:11:35.513: D/GStreamer+amc(15499): 0:00:14.693948423 0x4ddba120 jni/androidmedia/gstamc.c:126:gst_amc_attach_current_thread Attaching thread 0x4ddba120
01-25 15:11:35.513: D/GStreamer+amcvideoenc(15499): 0:00:14.694925965 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:275:create_amc_format:<amcvideoenc-omxsecavcencoder0> Color format info: {color_format=21, width=480, height=800, stride=480, slice-height=800, crop-left=0, crop-top=0, crop-right=0, crop-bottom=0, frame-size=576000}
01-25 15:11:35.513: D/GStreamer+amcvideoenc(15499): 0:00:14.695593715 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1252:gst_amc_video_enc_set_format:<amcvideoenc-omxsecavcencoder0> Configuring codec with format: {frame-rate=30.0, height=800, slice-height=800, color-format=21, width=480, bitrate=2097152, mime=video/avc, stride=480, i-frame-interval=8}
01-25 15:11:35.518: W/ACodec(15499): Use baseline profile instead of 8 for AVC recording
01-25 15:11:35.518: I/ACodec(15499): setupVideoEncoder succeeded
01-25 15:11:35.538: D/GStreamer+amcvideoenc(15499): 0:00:14.718050798 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1350:gst_amc_video_enc_handle_frame:<amcvideoenc-omxsecavcencoder0> Handling frame
01-25 15:11:35.553: D/GStreamer+amcvideoenc(15499): 0:00:14.733347548 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:896:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Waiting for available output buffer
01-25 15:11:35.553: D/GStreamer+amc(15499): 0:00:14.733524173 0x4e739e60 jni/androidmedia/gstamc.c:126:gst_amc_attach_current_thread Attaching thread 0x4e739e60
01-25 15:11:35.608: D/GStreamer+amc(15499): 0:00:14.789445090 0x4ddba120 jni/androidmedia/gstamc.c:2398:gst_amc_color_format_copy Buffer sizes equal, doing fast copy
01-25 15:11:35.613: D/GStreamer+amcvideoenc(15499): 0:00:14.792671882 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1448:gst_amc_video_enc_handle_frame:<amcvideoenc-omxsecavcencoder0> Queueing buffer 0: size 576000 time 11287 flags 0x00000000
01-25 15:11:35.623: D/GStreamer+amcvideoenc(15499): 0:00:14.802797882 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:915:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Output format has changed
01-25 15:11:35.623: D/GStreamer+amcvideoenc(15499): 0:00:14.803511048 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:931:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Got new output format: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=800, mime=video/avc, csd-0=java.nio.ByteArrayBuffer[position=0,limit=13,capacity=13], what=1869968451, width=480}
01-25 15:11:35.623: E/GStreamer+amc(15499): 0:00:14.805614507 0x4e739e60 jni/androidmedia/gstamc.c:983:gst_amc_format_get_float Failed to call Java method
01-25 15:11:35.623: E/GStreamer+amc(15499): 0:00:14.806152548 0x4e739e60 jni/androidmedia/gstamc.c:1047:gst_amc_format_get_int Failed to call Java method
01-25 15:11:35.623: E/GStreamer+amc(15499): 0:00:14.806324257 0x4e739e60 jni/androidmedia/gstamc.c:1047:gst_amc_format_get_int Failed to call Java method
01-25 15:11:35.628: D/GStreamer+amcvideoenc(15499): 0:00:14.809142465 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:896:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Waiting for available output buffer
01-25 15:11:35.628: D/GStreamer+amcvideoenc(15499): 0:00:14.809718590 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:989:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Got output buffer at index 0: size 21 time 0 flags 0x00000002
01-25 15:11:35.628: D/GStreamer+amcvideoenc(15499): 0:00:14.809896632 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:821:gst_amc_video_enc_handle_output_frame:<amcvideoenc-omxsecavcencoder0> got codecconfig in byte-stream format
01-25 15:11:35.628: D/GStreamer+amcvideoenc(15499): 0:00:14.811282132 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:1027:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Finished frame: ok
01-25 15:11:35.628: D/GStreamer+amcvideoenc(15499): 0:00:14.811628257 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:896:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Waiting for available output buffer
01-25 15:11:35.688: D/GStreamer+amcvideoenc(15499): 0:00:14.868603548 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1350:gst_amc_video_enc_handle_frame:<amcvideoenc-omxsecavcencoder0> Handling frame
01-25 15:11:35.688: D/GStreamer+amc(15499): 0:00:14.868869423 0x4ddba120 jni/androidmedia/gstamc.c:2398:gst_amc_color_format_copy Buffer sizes equal, doing fast copy
01-25 15:11:35.688: D/GStreamer+amcvideoenc(15499): 0:00:14.872379340 0x4ddba120 jni/androidmedia/gstamcvideoenc.c:1448:gst_amc_video_enc_handle_frame:<amcvideoenc-omxsecavcencoder0> Queueing buffer 4: size 576000 time 265922 flags 0x00000000
01-25 15:11:35.693: D/GStreamer+amcvideoenc(15499): 0:00:14.876930965 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:989:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Got output buffer at index 1: size 28313 time 11287 flags 0x00000001
01-25 15:11:35.698: D/GStreamer+amcvideoenc(15499): 0:00:14.878457507 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:1027:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Finished frame: ok
01-25 15:11:35.698: D/GStreamer+amcvideoenc(15499): 0:00:14.878801632 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:896:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Waiting for available output buffer
01-25 15:11:35.798: D/GStreamer+amcvideoenc(15499): 0:00:14.979295048 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:970:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Dequeueing output buffer timed out
01-25 15:11:35.798: D/GStreamer+amcvideoenc(15499): 0:00:14.979437007 0x4e739e60 jni/androidmedia/gstamcvideoenc.c:896:gst_amc_video_enc_loop:<amcvideoenc-omxsecavcencoder0> Waiting for available output buffer

----- Mail original -----
De: "cee1" <fykcee1 at gmail.com>
À: "Discussions about GStreamer on Android" <gstreamer-android at lists.freedesktop.org>
Envoyé: Jeudi 23 Janvier 2014 13:01:56
Objet: Re: androidmedia encoder problems

2014/1/23 Sebastian Dröge <sebastian at centricular.com>:
> On Mi, 2014-01-22 at 23:39 +0100, stic at free.fr wrote:
>> i would be happy to do it, but it's not clear to me of how to retrieve csd-0 and csd-1 values.
>> Are they sent in the buffer, and with what kind of structure ??
>
> Here's the inverse of it (somewhat):
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/androidmedia/gstamcvideodec.c#n1466
>
> You'll get such a field in the format returned by
> gst_amc_codec_get_output_format() too. Even two of them for h264, which
> then need to be reassembled like in the other code from the previous
> mail.

The current PPS/PPS handling logic in gstamcvideoenc is taking care of
BUFFER_FLAG_CODEC_CONFIG:
https://github.com/cee1/gst-plugins-bad/blob/master/sys/androidmedia/gstamcvideoenc.c#L803

When receives an output buffer with this flag, the h264 encoder will
1) add it to encoder_headers through gst_video_encoder_set_headers; 2)
send it downstream.

For retrieving csd-0 and csd-1, I couldn't get any returned
MediaFormat in my experiment(i.e. no INFO_OUTPUT_FORMAT_CHANGED), see
https://github.com/cee1/gst-plugins-bad/blob/master/sys/androidmedia/gstamcvideoenc.c#L908.
To stic at free.fr: you may enable related logs, and see whether it is
the case with you, if not, then you can handle the format in L917
(i.e. the gst_amc_codec_get_output_format() returned format)

BTW if no PPS/SPS generated by the h264 encoder, the client should
unable to play, right? I'm wondering what does a gst client say.

-- 
Regards,

- cee1
_______________________________________________
gstreamer-android mailing list
gstreamer-android at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/gstreamer-android