Could not capture whole duration audio with pipeline

[Howling wong]

Have tried to capture audio with video together in a dynamic pipeline but it seems like audio could only capture for fragment of time (not the full length in the file). I have used gst-discoverer to find out this problem



File Tested


gst-discoverer-1.0 xxxx.mp4
Analyzing file:///xxxx.mp4

====== AIUR: 4.6.1 build on May 11 2021 03:19:55. ======
    Core: AVI_PARSER_03.06.08  build on Sep 15 2020 02:45:45
 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1
------------------------
    Track 00 [video_0] Enabled
    Duration: 0:04:41.160000000
    Language: und
    Mime:
    video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)byte-stream, width=(int)720, height=(int)480, framerate=(fraction)25/1
------------------------
====== VPUDEC: 4.6.1 build on May 11 2021 03:19:55. ======
    wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jun  3 2021 04:20:32)
    vpulib: 1.1.1
    firmware: 1.1.1.65535
------------------------
    Track 01 [audio_0] Enabled
    Duration: 0:00:12.750476000
    Language: und
    Mime:
    audio/x-raw, format=(string)S16LE, channels=(int)2, layout=(string)interleaved, rate=(int)44100, bitrate=(int)1411200
------------------------
Done discovering file:///xxxx.mp4

Properties:
  Duration: 0:04:41.160000000
  Seekable: yes
  Live: no
  container: Audio Video Interleave (AVI)
    audio: Raw 16-bit PCM audio
      Stream ID: b6e7e8a9768c340295f0f67833e05ab2e8fe2243b4d7ec4e5d6152cbe76dc8af/1
      Language: <unknown>
      Channels: 2 (front-left, front-right)
      Sample rate: 44100
      Depth: 16
      Bitrate: 0
      Max bitrate: 0
    video: H.264 (Constrained Baseline Profile)
      Stream ID: b6e7e8a9768c340295f0f67833e05ab2e8fe2243b4d7ec4e5d6152cbe76dc8af/0
      Width: 720
      Height: 480
      Depth: 24
      Frame rate: 25/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0





>From the description given by gst-discoverer, it seem that the audio was only recording for 12+ seconds. I then constructed the static pipeline and test it out with gst-launch


Static pipeline


 gst-launch-1.0 -e -v \
    v4l2src \
        ! video/x-raw,width=720,height=480,framerate=30/1,is-live=true \
        ! clockoverlay \
        ! videorate \
        ! video/x-raw,framerate=25/1 \
        ! tee name=tv  \
       tv. \
        ! queue name=q1a \
        ! vpuenc_h264 \
        ! h264parse  \
        ! mux.video_0 \
       tv. \
        ! queue name=q1b \
        ! vpuenc_h264 \
        ! tee name=tv2 \
       tv2. \
        ! queue \
        ! rtph264pay pt=96 \
        ! udpsink host="x.x.x.x" port=3456 \
     pulsesrc volume=8.0 \
        ! audioconvert \
        ! audioresample \
        ! volume volume=1.0 \
        ! audio/x-raw,rate=8000,channels=1,depth=8,format=S16LE \
        ! tee name=ta  \
        ! queue \
        ! alawenc \
        ! tee name=ta2  \
       ta2. \
        ! queue \
        ! rtppcmapay pt=8 \
        ! udpsink host="x.x.x.x" port=3458 \
       ta2. \
        ! queue \
        ! mux.audio_0 \
      avimux name=mux \
        ! queue \
        ! filesink location=file%02d.avi


Analyzing file:///home/root/file%2502d.avi

====== AIUR: 4.6.1 build on May 11 2021 03:19:55. ======
    Core: AVI_PARSER_03.06.08  build on Sep 15 2020 02:45:45
 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1
------------------------
Track 00 [video_0] Enabled
Duration: 0:00:40.520000000
Language: und
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)byte-stream, width=(int)720, height=(int)480, framerate=(fraction)25/1
------------------------

====== VPUDEC: 4.6.1 build on May 11 2021 03:19:55. ======
    wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jun  3 2021 04:20:32)
    vpulib: 1.1.1
    firmware: 1.1.1.65535
    Track 01 [audio]: Disabled
    Codec: 2, SubCodec: 0
------------------------
Done discovering file:///home/root/file%2502d.avi

Properties:
  Duration: 0:00:40.520000000
  Seekable: yes
  Live: no
  container: Audio Video Interleave (AVI)
    video: H.264 (Constrained Baseline Profile)
      Stream ID: 2f44a8a002c570424bca50bdc0bc9c743ea882e7cd3f855918368cd108ff977f/0
      Width: 720
      Height: 480
      Depth: 24
      Frame rate: 25/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0



It seem to me that there are no audio recorded but on opening the file, there seem to be some static noise being heard(i assume that audio is perhaps recorded)



So my question is

  1.  For the static pipeline, is there really audio recorded? gst-discoverer and actual opening of files seem to show different outcomes
  2.  For dynamic pipeline (coded in C), what could the problem be that I could look into. Would be grateful if anyone can, with similar prior experience, to point out area I could look into


PS: I am using gstreamer 1.18


Thanks

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220602/efd6f7c1/attachment-0001.htm>