How to operate on GstMapInfo

[dingoegret]

I'm sending an rtp stream of audio to gstreamer and I have a pipeline with
appsink. I've got a g-signal_connect callback function registered on
'new-sample' and this works fine. In that callback function I'm retrieving
the audio data via the function:

static void new_sample(GstElement *sink) {
        GstSample *sample;
        g_signal_emit_by_name(sink, "pull-sample", &sample, NULL);

        if(sample) {
                GstBuffer* buffer = gst_sample_get_buffer(sample);
                GstMapInfo info;
                gst_buffer_map(buffer, &info, GST_MAP_READ);

                //fwrite(info.data, info.size, 1, myfile);
                std::cout << info.data[info.size + 10] << std::endl;

                gst_buffer_unmap (buffer, &info);
                gst_sample_unref(sample);
        }
}


Now the fwrite line seems to be working properly. The audio data gets
written to a file and that file plays the audio fine. Here is what I don't
get though, and keep in mind I am c/c++/memory management programming noob.
What format is the info.data (*GstMapInfo.data) in? Is it an array? A
pointer to something? An array of characters or zeroes and ones or what
exactly. The specific problem I face is that I am trying to use a google
speech api

#include "google/cloud/speech/v1beta1/cloud_speech.grpc.pb.h"

and one particular function, which is 

StreamingRecognizeRequest::set_audio_content(const void* value, size_t size)

accepts a void*? What does that mean? iS this compatible with
GstMapInfo.data? Furthermore, after some tests it appears that this function
only works properly (google sends back transcribed results) if you send it
at minimum 16 kilobytes of data increments and no less. So even if I figure
out how to work with the GstMapInfo.data to use it with set_audio_content, I
will have to also figure out how to chunk the data into 16kb each.

Here is an example function which uses google speech api for reference

static void MicrophoneThreadMain(
        grpc::ClientReaderWriterInterface<StreamingRecognizeRequest,
                StreamingRecognizeResponse>* streamer,
        char* file_path) {
    StreamingRecognizeRequest request;
    std::ifstream file_stream(file_path);
    const size_t chunk_size = 64 * 1024;
    std::vector<char> chunk(chunk_size);
    while (true) {
        // Read another chunk from the file.
        std::streamsize bytes_read =
                file_stream.rdbuf()->sgetn(&chunk[0], chunk.size());
        // And write the chunk to the stream.
        request.set_audio_content(&chunk[0], bytes_read);
        std::cout << "Sending " << bytes_read / 1024 << "k bytes." <<
std::endl;
        streamer->Write(request);
        if (bytes_read < chunk.size()) {
            // Done reading everything from the file, so done writing to the
stream.
            streamer->WritesDone();
            break;
        } else {
            // Wait a second before writing the next chunk.
            std::this_thread::sleep_for(std::chrono::seconds(1));
        }
    }
}

Any insight into this would be greatly appreciated



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-operate-on-GstMapInfo-tp4682202.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.