[gst-devel] Proposition for a media test suite

[Stefan Kost]

Hi Edward,
> Hi,
> 
> On 11/14/05, Stefan Kost <ensonic at hora-obscura.de> wrote:
> 
>>Hi Edward,
>>
>>Edward Hervey wrote:
>>
>>>Morning hackers,
>>>
>>>  Over the past day, and after being fed-up of fixing issues in some
>>>plugins to make a file play/seek correctly and see that it broke some
>>>other file instead, I came up with the plan for a simple media test
>>>suite for GStreamer.
>>>
>>>  I put most of the main lines on gnome live here :
>>>http://live.gnome.org/GStreamerMediaTest
>>>
>>>  This could be done quite quickly once we agree on what we want in
>>>that media test suite. The good points about such a test suite are :
>>>  _ See quickly what we support at any given point
>>>  _ Spot out (even more) problems with plugins
>>>  _ Have a system that unifies data to be put in bug reporting from users
>>>  _ Allow people to figure out where work is needed
>>>
>>>  Propositions and comments are more than welcome. IMHO this is a must
>>>have for a rock solid GStreamer 0.10
>>>
>>
>>I agree totally! Some comments:
>>
>>* shoudn't it be done in check?
> 
> 
>   I wouldn't be inclined to do that :
>    * This testsuite might take a LOOOONG time
>    * it's not only related to one package, but the whole of GStreamer
> (core, plugins, python, ....)
> 
> 
>>** to avoid stumbling over stuff in the python layer
>>** check can do XML reporting (if you want it)
> 
> 
>   XML reporting in python is way easier than in C
erm, check his this build in already ;) Nothing to do, just one lie to activate.
> 
> 
>>* we probably need to merge the result as not every build can handle all formats
> 
> 
>   Another reason not to do it in check. We could have 'slaves' on
> different machines.
for the buildbot it won't make any diffenrence who produced the xml. @thomas: 
have you found a way to copy files from buildslave to the buildmaster?

> 
> 
>>* of course crashes on unhandled media formats are bugs
> 
> 
>   Indeed, it should handle them gracefully.
> 
> 
>>* alsa has a dummy soundcard and then there is Xvfb, so we can run those test on
>>build-slaves too (maybe only on specific ones)
> 
> 
>   Good idea, but I think we can already get rid of a LOT of bugs just
> by using fakesinks.
> 
oh right, they will eat anything. maybe using read sinks is better to test if 
the media decoding elements set the right caps.
> 
>>Stefan
>>
> 
> 
>   Keep the ideas coming in, I'll hack up something once there's enough
> ideas. Sometime around friday.
> 
>    Edward
> 
> --
> Edward Hervey
> Junior developer / Fluendo S.L.
> http://www.pitivi.org/

Ciao
   Stefan