[gst-devel] Proposition for a media test suite

[Stefan Kost]

Hi Edward,

Edward Hervey wrote:
> Morning hackers,
> 
>   Over the past day, and after being fed-up of fixing issues in some
> plugins to make a file play/seek correctly and see that it broke some
> other file instead, I came up with the plan for a simple media test
> suite for GStreamer.
> 
>   I put most of the main lines on gnome live here :
> http://live.gnome.org/GStreamerMediaTest
> 
>   This could be done quite quickly once we agree on what we want in
> that media test suite. The good points about such a test suite are :
>   _ See quickly what we support at any given point
>   _ Spot out (even more) problems with plugins
>   _ Have a system that unifies data to be put in bug reporting from users
>   _ Allow people to figure out where work is needed
> 
>   Propositions and comments are more than welcome. IMHO this is a must
> have for a rock solid GStreamer 0.10
> 
I agree totally! Some comments:

* shoudn't it be done in check?
** to avoid stumbling over stuff in the python layer
** check can do XML reporting (if you want it)
* we probably need to merge the result as not every build can handle all formats
* of course crashes on unhandled media formats are bugs
* alsa has a dummy soundcard and then there is Xvfb, so we can run those test on 
build-slaves too (maybe only on specific ones)

Stefan