Feeding raw audio data into a GES timeline

[Thibault Saunier]

Hi,

I checked a bit your code and there are a few issues to get that working:
* the `GESAudioSource::create_source` vmethod is not exposed through GI,
and for some reason even properly annotating it makes it not working.
* We need
https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/merge_requests/231
to fix some annotations so we can subclass GESSource properly
* There is a floating reference mishandling in PyGObject leading to a
crash, we need https://gitlab.gnome.org/GNOME/pygobject/-/merge_requests/129
to get it working

I then updated the code here:
https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/snippets/1633
- your source element doesn't seem to work properly yet though.

Also not that I have some WIP that would allow doing something like:

```
class YourSource(GstApp.AppSrc):
   ...
Gst.Element.register("myaudiosrc", 0, YourSource)

clip = GES.Clip.new(video_source_bin_desc=None,
audio_source_bin_desc="myaudiosource")
```

removing the need for any subclass.

- Thibault

On Sat, Feb 20, 2021 at 9:15 AM Daniel Bailey <danieljabailey at gmail.com>
wrote:

> Hi, thanks for the quick reply.
>
> I have stripped down my project code to just the bare minimum to show the
> approach I am using and the problem I'm having, see the code here:
> https://gist.github.com/danieljabailey/594e77728e55b88ff2b81cbf0cbde651
>
> The output from this program is pasted below.
> Thanks.
> Dan.
>
> Adding asset to layer <__gi__.GESTestClipAsset object at 0x7f16541e4540
> (GESTestClipAsset at 0x56462a434b00)>
> Adding asset to layer <GES.SourceClipAsset object at 0x7f16541e4680
> (GESSourceClipAsset at 0x56462a84b050)>
> Creating audio synth element
> Created source
> <__main__.SynthAudioSourceElement object at 0x7f16541e4780
> (__main__+SynthAudioSourceElement at 0x56462a4a08f0)>
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_priority:
> assertion 'object->priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_duration:
> assertion 'priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
> ges_track_element_is_active: assertion 'object->priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
> ges_track_element_set_track: assertion 'object->priv->nleobject' failed
> Creating audio synth element
> Created source
> <__main__.SynthAudioSourceElement object at 0x7f16541e4680
> (__main__+SynthAudioSourceElement at 0x56462a4a08f0)>
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_priority:
> assertion 'object->priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_duration:
> assertion 'priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
> ges_track_element_is_active: assertion 'object->priv->nleobject' failed
>
> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
> ges_track_element_set_track: assertion 'object->priv->nleobject' failed
>
>
> On Fri, Feb 19, 2021 at 10:52 PM Thibault Saunier <tsaunier at gnome.org>
> wrote:
>
>> Hi,
>>
>> Your approach sounds about right, although. I guess the issues are
>> related to assets, could you paste the assertions you get/a piece of code
>> to reproduce?
>>
>> - Thibault
>>
>>
>> Le ven. 19 févr. 2021 à 15:32, Daniel Bailey <danieljabailey at gmail.com>
>> a écrit :
>>
>>> Hi,
>>> I am working on an audio synthesis tool, which allows the user to
>>> generate audio to play over the top of a sequence of videos.
>>> I'm writing this in python.
>>> I am using GES to create a simple video sequence editor.
>>> The application needs to be able to play the synthesised audio in sync
>>> with the video.
>>>
>>> I have written a simple proof of concept subclass of GstApp.AppSrc which
>>> can do some simple audio synthesis, and feed it into a GStreamer pipeline.
>>> This works fine for me if I just connect it up to an autoaudiosink without
>>> using any GES stuff.
>>>
>>> Is there a way to get this gstreamer element to be part of the GES
>>> timeline?
>>>
>>> I tried to do this using the following method:
>>>
>>> Create a Subclass of GES.SourceClip
>>> Implement do_create_track_element on this subclass
>>> do_create_track_element instantiates a custom subclass of GES.AudioSource
>>> the subclass of AudioSource overrides do_create_source and returns a
>>> pipeline that wraps my custom AppSrc.
>>> call layer.add_clip on a layer in a GES timeline to add one of these
>>> custom clips
>>>
>>> This method does not work, I see various assertion errors when I try
>>> this.
>>>
>>> Is this the right approach? Am I doing roughly the right thing? Or is
>>> there a totally different way to achieve what I want?
>>> If this is roughly the right idea then where am I going wrong?
>>>
>>> Thanks.
>>> Dan.
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210225/3da02d43/attachment.htm>