<html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> </head> <body text="#000000" bgcolor="#FFFFFF"> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> I'm working on a dataflow interface for live media generation similar to tools like PureData, vvvv and Isadora (veejaying, sound production, interactivity, etc). Currently our dataflow is synchronized but we want to make it unsynchronized and multi-threaded. Here is an example screenshot: <a href="http://ur1.ca/cl992">http://ur1.ca/cl992</a> We would probably choose between either a push or a pull model (not both) although we're not sure which one yet. Also, our datas are "hard typed" and we support just a few types to make things easy for the user (eg. one type of audio, one type of video). Finally, we provide a high-level C++ interface for the effects (elements) that tries to make it easy for unexperimented developers to contribute. My question is: Could I use GStreamer as the backend (ie. managing the dataflow)? Here are the characteristics/constraints of the software: - QT-based GUI - allow for many data types: — audio (we'll support just one type through the flow, like audio/raw-float 44.1k) — video (again, we'll support just one type here as well, RGBA 32bit) — floats and integers — strings — triggers / "bangs" — datastructures such as vectors and maps — vectorial (SVG) — 3d scenes and textures — lambda (ie. elements/plugins can send a copy of themselves in a buffer) (see this example: <a href="http://www.cs.bgu.ac.il/%7Eshemeas/juks/" title="http://www.cs.bgu.ac.il/~shemeas/juks/">http://www.cs.bgu.ac.il/~shemeas/juks/</a>) — any other user-defined datatype should be easily implementable through a high-level interface - composite elements (abstractions) - possibility to change the graph in real-time: — unplug, plug pads — remove elements from the pipeline - multi-threaded — we're not sure how it will be implemented, either each element will be on its own thread, or parts of the graph — freely moving elements from one thread to the other - low-latency, especially for audio eg. possibility of priorizing audio threads over video threads - eventually portable to mobile platforms Example of application: A live performance with a dancer and a video projection. The movements of the dancers are analysed with OpenCV which influences both audio and video in real time. A hardware interface on the dancer sends accelerometer data by wifi through OSC messages, also influencing the audio and video. The video is generated by a mix of video effects applied on a camera image. The video effects influence the image that is projected as a texture on a 3D object, which is then rendered to the projection screen. Finally, sentences submitted through a webpage are also sent to the application: the text is rendered as vectorial objects which are then modified using vectorial effects and projected on the floor. The application is controlled live: the performer controlling it manipulates the graph live, rewiring connections between effects, playing with sliders and generating sound using a microphone. Thanks, Tats </body> </html>