Using GStreamer for a game emulation pipeline (and many other questions)

[athairus]

Hi everyone! I'm part of an open-source project called Phoenix (phoenix.vg).
I've talked to people on IRC (#gstreamer) around two months ago about this
idea of mine but I thought I'd post here to get a more formal conversation
going.

Phoenix is a multi-system emulator for classic game consoles. It's powered
by Qt 5 and Libretro (an API for emulators, in Libretro-speak they're
called cores) and written in C++, QML and a bit of Objective-C and Python.

No need to get into too much detail here, but I'm planning on a massive
variety of features in the future which behooves me to make it as easy as
possible to play around with new ideas. One way to make this possible is to
have a flexible pipeline system. During my research this project came on my
radar early on along with Microsoft's Media Foundation, though it was
obvious which was the more open-source friendly choice!

As I got further into developing my own custom pipeline system I started to
get lost in its complexity. I was starting to do metaprogramming in C++,
mostly to get elements to register themselves at compile time without
having to explicitly register themselves in the pipeline manager. At this
point I decided that I should stop and check out existing solutions to see
if I could accomplish what I wanted much faster. Right now, the only
actively maintained choice is GStreamer so here I am!

The way I see it, running a game emulator isn't too far removed from
playing video. You have a stream of audio/video data that's been "decoded"
and presented to the user. In the case of games you'll also have input to
deal with.

The plan is to develop a series of new elements and include them with each
release. They're so domain-specific that I don't think it's worth getting
them included into any of the gst-plugins-* repos. I imagine I'll either
place the shared libraries near the executable or statically link them. Is
the latter possible? What considerations should I have for the former?

I've noticed that there's a project called gstreamer-qt that's no longer
maintained. What are my options for getting video output into the QML scene?

My project uses CMake and GStreamer uses automake. Is there any work being
done on compiling GStreamer with CMake? Official FindGStreamer.cmake
modules?

Lastly, I have the issue of input. I'm going to use SDL2 to manage
connected controllers. I need to get controller input polled each frame
before the Libretro element can emulate a frame. What approaches do I have?
Another element before the Libretro element? I was told a while ago on IRC
that I could maybe use something meant for DVD menu navigation? What is
that?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160903/caabde74/attachment.html>