How to operate on GstMapInfo

[Andrew Grace]

> On Mar 13, 2017, at 11:17 PM, dingoegret <sephvelut at gmail.com> wrote:
> 
> I'm sending an rtp stream of audio to gstreamer and I have a pipeline with
> appsink. I've got a g-signal_connect callback function registered on
> 'new-sample' and this works fine. In that callback function I'm retrieving
> the audio data via the function:
> 
> static void new_sample(GstElement *sink) {
>        GstSample *sample;
>        g_signal_emit_by_name(sink, "pull-sample", &sample, NULL);
> 
>        if(sample) {
>                GstBuffer* buffer = gst_sample_get_buffer(sample);
>                GstMapInfo info;
>                gst_buffer_map(buffer, &info, GST_MAP_READ);
> 
>                //fwrite(info.data, info.size, 1, myfile);
>                std::cout << info.data[info.size + 10] << std::endl;
> 
>                gst_buffer_unmap (buffer, &info);
>                gst_sample_unref(sample);
>        }
> }
> 
> 
> Now the fwrite line seems to be working properly. The audio data gets
> written to a file and that file plays the audio fine. Here is what I don't
> get though, and keep in mind I am c/c++/memory management programming noob.

Gstreamer is written in C, not C++.  C is a simpler language, so I'd avoid C++ if possible.  The GLib library fills in much of what the standard C library lacks.  For trying stuff out, I'd also recommend using the Python bindings pygi (python gobject inspection).


> What format is the info.data (*GstMapInfo.data) in? Is it an array? A
> pointer to something? An array of characters or zeroes and ones or what
> exactly. The specific problem I face is that I am trying to use a google
> speech api
> 
> #include "google/cloud/speech/v1beta1/cloud_speech.grpc.pb.h"
> 
> and one particular function, which is 
> 
> StreamingRecognizeRequest::set_audio_content(const void* value, size_t size)
> 
> accepts a void*? What does that mean?

A void* (void pointer) is awkward syntax (I think C overuses the star - makes things confusing in the beginning).  Void ptr is a data type that lets you accept a pointer of any type without the C compiler complaining of a type error.  You can cast any pointer to void* and vice versa.  It's up to you to know that what you are doing makes sense.  I've seen it used a lot, especially for libraries that create opaque objects (a struct allocated at the other end of that void pointer handle).

If C is totally new for you, maybe check out the classic K&R C book.  But there are also many tutorials online.  I even liked the coverage of C in the Big Nerd Ranch Objective-C book (the first couple chapters are just plain C).  They write in a very clear and concise way in that book.


> iS this compatible with
> GstMapInfo.data? Furthermore, after some tests it appears that this function
> only works properly (google sends back transcribed results) if you send it
> at minimum 16 kilobytes of data increments and no less. So even if I figure
> out how to work with the GstMapInfo.data to use it with set_audio_content, I
> will have to also figure out how to chunk the data into 16kb each.
> 
> Here is an example function which uses google speech api for reference
> 
> static void MicrophoneThreadMain(
>        grpc::ClientReaderWriterInterface<StreamingRecognizeRequest,
>                StreamingRecognizeResponse>* streamer,
>        char* file_path) {
>    StreamingRecognizeRequest request;
>    std::ifstream file_stream(file_path);
>    const size_t chunk_size = 64 * 1024;
>    std::vector<char> chunk(chunk_size);
>    while (true) {
>        // Read another chunk from the file.
>        std::streamsize bytes_read =
>                file_stream.rdbuf()->sgetn(&chunk[0], chunk.size());
>        // And write the chunk to the stream.
>        request.set_audio_content(&chunk[0], bytes_read);
>        std::cout << "Sending " << bytes_read / 1024 << "k bytes." <<
> std::endl;
>        streamer->Write(request);
>        if (bytes_read < chunk.size()) {
>            // Done reading everything from the file, so done writing to the
> stream.
>            streamer->WritesDone();
>            break;
>        } else {
>            // Wait a second before writing the next chunk.
>            std::this_thread::sleep_for(std::chrono::seconds(1));
>        }
>    }
> }
> 
> Any insight into this would be greatly appreciated
> 
> 
> 
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-operate-on-GstMapInfo-tp4682202.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel