<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div class="elementToProof" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> Have tried to capture audio with video together in a dynamic pipeline but it seems like audio could only capture for fragment of time (not the full length in the file). I have used gst-discoverer to find out this problem <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> File Tested <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <pre style="color:var(--highlight-color);font-size:var(--fs-body1);font-weight:400;background-color:var(--highlight-bg);margin-top:0px;margin-bottom:calc(var(--s-prose-spacing) + 0.4em);padding:var(--su12);line-height:var(--lh-md);font-family:var(--ff-mono);width:auto;max-height:600px;overflow:auto;border-radius:var(--br-md)"><code style="margin:0px;color:var(--black-800);border-radius:0px">gst-discoverer-1.0 xxxx.mp4 Analyzing file:///xxxx.mp4 ====== AIUR: 4.6.1 build on May 11 2021 03:19:55. ====== Core: AVI_PARSER_03.06.08 build on Sep 15 2020 02:45:45 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1 ------------------------ Track 00 [video_0] Enabled Duration: 0:04:41.160000000 Language: und Mime: video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)byte-stream, width=(int)720, height=(int)480, framerate=(fraction)25/1 ------------------------ ====== VPUDEC: 4.6.1 build on May 11 2021 03:19:55. ====== wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jun 3 2021 04:20:32) vpulib: 1.1.1 firmware: 1.1.1.65535 ------------------------ Track 01 [audio_0] Enabled Duration: 0:00:12.750476000 Language: und Mime: audio/x-raw, format=(string)S16LE, channels=(int)2, layout=(string)interleaved, rate=(int)44100, bitrate=(int)1411200 ------------------------ Done discovering file:///xxxx.mp4 Properties: Duration: 0:04:41.160000000 Seekable: yes Live: no container: Audio Video Interleave (AVI) audio: Raw 16-bit PCM audio Stream ID: b6e7e8a9768c340295f0f67833e05ab2e8fe2243b4d7ec4e5d6152cbe76dc8af/1 Language: <unknown> Channels: 2 (front-left, front-right) Sample rate: 44100 Depth: 16 Bitrate: 0 Max bitrate: 0 video: H.264 (Constrained Baseline Profile) Stream ID: b6e7e8a9768c340295f0f67833e05ab2e8fe2243b4d7ec4e5d6152cbe76dc8af/0 Width: 720 Height: 480 Depth: 24 Frame rate: 25/1 Pixel aspect ratio: 1/1 Interlaced: false Bitrate: 0 Max bitrate: 0</code></pre> <pre style="margin-top:0px;margin-bottom:calc(var(--s-prose-spacing) + 0.4em);padding:var(--su12);line-height:var(--lh-md);font-family:var(--ff-mono);font-size:var(--fs-body1);width:auto;max-height:600px;overflow:auto;background-color:var(--highlight-bg);border-radius:var(--br-md);color:var(--highlight-color);text-align:left"><code style="margin:0px;color:var(--black-800);border-radius:0px"> </code></pre> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> >From the description given by gst-discoverer, it seem that the audio was only recording for 12+ seconds. I then constructed the static pipeline and test it out with gst-launch <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> Static pipeline <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <pre style="margin-top:0px;margin-bottom:calc(var(--s-prose-spacing) + 0.4em);padding:var(--su12);line-height:var(--lh-md);font-family:var(--ff-mono);font-size:var(--fs-body1);width:auto;max-height:600px;overflow:auto;background-color:var(--highlight-bg);border-radius:var(--br-md);color:var(--highlight-color);text-align:left"><code style="margin:0px;color:var(--black-800);border-radius:0px"> gst-launch-1.0 -e -v \ v4l2src \ ! video/x-raw,width=720,height=480,framerate=30/1,is-live=true \ ! clockoverlay \ ! videorate \ ! video/x-raw,framerate=25/1 \ ! tee name=tv \ tv. \ ! queue name=q1a \ ! vpuenc_h264 \ ! h264parse \ ! mux.video_0 \ tv. \ ! queue name=q1b \ ! vpuenc_h264 \ ! tee name=tv2 \ tv2. \ ! queue \ ! rtph264pay pt=96 \ ! udpsink host="x.x.x.x" port=3456 \ pulsesrc volume=8.0 \ ! audioconvert \ ! audioresample \ ! volume volume=1.0 \ ! audio/x-raw,rate=8000,channels=1,depth=8,format=S16LE \ ! tee name=ta \ ! queue \ ! alawenc \ ! tee name=ta2 \ ta2. \ ! queue \ ! rtppcmapay pt=8 \ ! udpsink host="x.x.x.x" port=3458 \ ta2. \ ! queue \ ! mux.audio_0 \ avimux name=mux \ ! queue \ ! filesink location=file%02d.avi </code></pre> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <pre style="margin-top:0px;margin-bottom:calc(var(--s-prose-spacing) + 0.4em);padding:var(--su12);line-height:var(--lh-md);font-family:var(--ff-mono);font-size:var(--fs-body1);width:auto;max-height:600px;overflow:auto;background-color:var(--highlight-bg);border-radius:var(--br-md);color:var(--highlight-color);text-align:left"><code style="margin:0px;color:var(--black-800);border-radius:0px">Analyzing file:///home/root/file%2502d.avi ====== AIUR: 4.6.1 build on May 11 2021 03:19:55. ====== Core: AVI_PARSER_03.06.08 build on Sep 15 2020 02:45:45 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1 ------------------------ Track 00 [video_0] Enabled Duration: 0:00:40.520000000 Language: und Mime: video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)byte-stream, width=(int)720, height=(int)480, framerate=(fraction)25/1 ------------------------ ====== VPUDEC: 4.6.1 build on May 11 2021 03:19:55. ====== wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jun 3 2021 04:20:32) vpulib: 1.1.1 firmware: 1.1.1.65535 Track 01 [audio]: Disabled Codec: 2, SubCodec: 0 ------------------------ Done discovering file:///home/root/file%2502d.avi Properties: Duration: 0:00:40.520000000 Seekable: yes Live: no container: Audio Video Interleave (AVI) video: H.264 (Constrained Baseline Profile) Stream ID: 2f44a8a002c570424bca50bdc0bc9c743ea882e7cd3f855918368cd108ff977f/0 Width: 720 Height: 480 Depth: 24 Frame rate: 25/1 Pixel aspect ratio: 1/1 Interlaced: false Bitrate: 0 Max bitrate: 0 </code></pre> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> It seem to me that there are no audio recorded but on opening the file, there seem to be some static noise being heard(i assume that audio is perhaps recorded) <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin-top:0px;margin-bottom:var(--s-prose-spacing);font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> So my question is <ol style="margin-top:0px;margin-bottom:var(--s-prose-spacing);margin-left:30px;font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;list-style:decimal;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <li style="margin-top:0px;margin-bottom:var(--s-prose-spacing-condensed);overflow-wrap:break-word"> For the static pipeline, is there really audio recorded? gst-discoverer and actual opening of files seem to show different outcomes</li><li style="margin:0px;overflow-wrap:break-word">For dynamic pipeline (coded in C), what could the problem be that I could look into. Would be grateful if anyone can, with similar prior experience, to point out area I could look into</li></ol> <p style="margin:0px;font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin:0px;font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> PS: I am using gstreamer 1.18 <p style="margin:0px;font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> <p style="margin:0px;font-family:-apple-system, BlinkMacSystemFont, "Segoe UI Adjusted", "Segoe UI", "Liberation Sans", sans-serif;font-size:15px;clear:both;color:rgb(35, 38, 41);text-align:left;background-color:rgb(255, 255, 255)"> Thanks </div> </body> </html>