How to operate on GstMapInfo

[Nicolas Dufresne]

Le lundi 13 mars 2017 à 20:17 -0700, dingoegret a écrit :
> I'm sending an rtp stream of audio to gstreamer and I have a pipeline
> with
> appsink. I've got a g-signal_connect callback function registered on
> 'new-sample' and this works fine. In that callback function I'm
> retrieving
> the audio data via the function:
> 
> static void new_sample(GstElement *sink) {
>         GstSample *sample;
>         g_signal_emit_by_name(sink, "pull-sample", &sample, NULL);
> 
>         if(sample) {
>                 GstBuffer* buffer = gst_sample_get_buffer(sample);
>                 GstMapInfo info;
>                 gst_buffer_map(buffer, &info, GST_MAP_READ);
> 
>                 //fwrite(info.data, info.size, 1, myfile);
>                 std::cout << info.data[info.size + 10] << std::endl;
> 
>                 gst_buffer_unmap (buffer, &info);
>                 gst_sample_unref(sample);
>         }
> }
> 
> 
> Now the fwrite line seems to be working properly. The audio data gets
> written to a file and that file plays the audio fine. Here is what I
> don't
> get though, and keep in mind I am c/c++/memory management programming
> noob.
> What format is the info.data (*GstMapInfo.data) in? Is it an array? A
> pointer to something? An array of characters or zeroes and ones or
> what
> exactly. The specific problem I face is that I am trying to use a
> google
> speech api

It's an array of bytes, guint8 *, see:

https://developer.gnome.org/gstreamer/stable/gstreamer-GstMemory.html#G
stMapInfo

> 
> #include "google/cloud/speech/v1beta1/cloud_speech.grpc.pb.h"
> 
> and one particular function, which is 
> 
> StreamingRecognizeRequest::set_audio_content(const void* value,
> size_t size)
> 
> accepts a void*? What does that mean? iS this compatible with
> GstMapInfo.data? Furthermore, after some tests it appears that this
> function

Yes, all pointers are compatible with void *. In general you don't even
have to cast, but in C++ it's more restrictive.

> only works properly (google sends back transcribed results) if you
> send it
> at minimum 16 kilobytes of data increments and no less. So even if I
> figure
> out how to work with the GstMapInfo.data to use it with
> set_audio_content, I
> will have to also figure out how to chunk the data into 16kb each.

GStreamer provides GstAdapter, which is a utility to accumulate buffers
and chunk them. See:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-lib
s/html/GstAdapter.html

> 
> Here is an example function which uses google speech api for
> reference
> 
> static void MicrophoneThreadMain(
>         grpc::ClientReaderWriterInterface<StreamingRecognizeRequest,
>                 StreamingRecognizeResponse>* streamer,
>         char* file_path) {
>     StreamingRecognizeRequest request;
>     std::ifstream file_stream(file_path);
>     const size_t chunk_size = 64 * 1024;
>     std::vector<char> chunk(chunk_size);
>     while (true) {
>         // Read another chunk from the file.
>         std::streamsize bytes_read =
>                 file_stream.rdbuf()->sgetn(&chunk[0], chunk.size());
>         // And write the chunk to the stream.
>         request.set_audio_content(&chunk[0], bytes_read);
>         std::cout << "Sending " << bytes_read / 1024 << "k bytes." <<
> std::endl;
>         streamer->Write(request);
>         if (bytes_read < chunk.size()) {
>             // Done reading everything from the file, so done writing
> to the
> stream.
>             streamer->WritesDone();
>             break;
>         } else {
>             // Wait a second before writing the next chunk.
>             std::this_thread::sleep_for(std::chrono::seconds(1));
>         }
>     }
> }
> 
> Any insight into this would be greatly appreciated
> 
> 
> 
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble
> .com/How-to-operate-on-GstMapInfo-tp4682202.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170314/e9e83fe1/attachment.sig>