Feeding raw audio data into a GES timeline

Thu Feb 25 10:53:20 UTC 2021

Hi,
Thanks for the detailed reply.
I'll have a play with your updated code soon :)

Is there anywhere your WIP is being tracked so that I follow and know when
it is ready? I would love to switch to that approach once it's ready.

Thanks,
Dan.

On Thu, Feb 25, 2021 at 3:01 AM Thibault Saunier <tsaunier at gnome.org> wrote:

> Hi,
>
> I checked a bit your code and there are a few issues to get that working:
> * the `GESAudioSource::create_source` vmethod is not exposed through GI,
> and for some reason even properly annotating it makes it not working.
> * We need
> https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/merge_requests/231
> to fix some annotations so we can subclass GESSource properly
> * There is a floating reference mishandling in PyGObject leading to a
> crash, we need
> https://gitlab.gnome.org/GNOME/pygobject/-/merge_requests/129 to get it
> working
>
> I then updated the code here:
> https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/snippets/1633
> - your source element doesn't seem to work properly yet though.
>
> Also not that I have some WIP that would allow doing something like:
>
> ```
> class YourSource(GstApp.AppSrc):
>    ...
> Gst.Element.register("myaudiosrc", 0, YourSource)
>
> clip = GES.Clip.new(video_source_bin_desc=None,
> audio_source_bin_desc="myaudiosource")
> ```
>
> removing the need for any subclass.
>
> - Thibault
>
> On Sat, Feb 20, 2021 at 9:15 AM Daniel Bailey <danieljabailey at gmail.com>
> wrote:
>
>> Hi, thanks for the quick reply.
>>
>> I have stripped down my project code to just the bare minimum to show the
>> approach I am using and the problem I'm having, see the code here:
>> https://gist.github.com/danieljabailey/594e77728e55b88ff2b81cbf0cbde651
>>
>> The output from this program is pasted below.
>> Thanks.
>> Dan.
>>
>> Adding asset to layer <__gi__.GESTestClipAsset object at 0x7f16541e4540
>> (GESTestClipAsset at 0x56462a434b00)>
>> Adding asset to layer <GES.SourceClipAsset object at 0x7f16541e4680
>> (GESSourceClipAsset at 0x56462a84b050)>
>> Creating audio synth element
>> Created source
>> <__main__.SynthAudioSourceElement object at 0x7f16541e4780
>> (__main__+SynthAudioSourceElement at 0x56462a4a08f0)>
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> _set_priority: assertion 'object->priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> _set_duration: assertion 'priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> ges_track_element_is_active: assertion 'object->priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> ges_track_element_set_track: assertion 'object->priv->nleobject' failed
>> Creating audio synth element
>> Created source
>> <__main__.SynthAudioSourceElement object at 0x7f16541e4680
>> (__main__+SynthAudioSourceElement at 0x56462a4a08f0)>
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> _set_priority: assertion 'object->priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> _set_duration: assertion 'priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> ges_track_element_is_active: assertion 'object->priv->nleobject' failed
>>
>> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336:
>> ges_track_element_set_track: assertion 'object->priv->nleobject' failed
>>
>>
>> On Fri, Feb 19, 2021 at 10:52 PM Thibault Saunier <tsaunier at gnome.org>
>> wrote:
>>
>>> Hi,
>>>
>>> Your approach sounds about right, although. I guess the issues are
>>> related to assets, could you paste the assertions you get/a piece of code
>>> to reproduce?
>>>
>>> - Thibault
>>>
>>>
>>> Le ven. 19 févr. 2021 à 15:32, Daniel Bailey <danieljabailey at gmail.com>
>>> a écrit :
>>>
>>>> Hi,
>>>> I am working on an audio synthesis tool, which allows the user to
>>>> generate audio to play over the top of a sequence of videos.
>>>> I'm writing this in python.
>>>> I am using GES to create a simple video sequence editor.
>>>> The application needs to be able to play the synthesised audio in sync
>>>> with the video.
>>>>
>>>> I have written a simple proof of concept subclass of GstApp.AppSrc
>>>> which can do some simple audio synthesis, and feed it into a GStreamer
>>>> pipeline. This works fine for me if I just connect it up to an
>>>> autoaudiosink without using any GES stuff.
>>>>
>>>> Is there a way to get this gstreamer element to be part of the GES
>>>> timeline?
>>>>
>>>> I tried to do this using the following method:
>>>>
>>>> Create a Subclass of GES.SourceClip
>>>> Implement do_create_track_element on this subclass
>>>> do_create_track_element instantiates a custom subclass of
>>>> GES.AudioSource
>>>> the subclass of AudioSource overrides do_create_source and returns a
>>>> pipeline that wraps my custom AppSrc.
>>>> call layer.add_clip on a layer in a GES timeline to add one of these
>>>> custom clips
>>>>
>>>> This method does not work, I see various assertion errors when I try
>>>> this.
>>>>
>>>> Is this the right approach? Am I doing roughly the right thing? Or is
>>>> there a totally different way to achieve what I want?
>>>> If this is roughly the right idea then where am I going wrong?
>>>>
>>>> Thanks.
>>>> Dan.
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210225/53fe5270/attachment-0001.htm>