[gst-devel] Proposition for a media test suite

[Edward Hervey]

Hi,

On 11/14/05, Stefan Kost <ensonic at hora-obscura.de> wrote:
> Hi Edward,
>
> Edward Hervey wrote:
> > Morning hackers,
> >
> >   Over the past day, and after being fed-up of fixing issues in some
> > plugins to make a file play/seek correctly and see that it broke some
> > other file instead, I came up with the plan for a simple media test
> > suite for GStreamer.
> >
> >   I put most of the main lines on gnome live here :
> > http://live.gnome.org/GStreamerMediaTest
> >
> >   This could be done quite quickly once we agree on what we want in
> > that media test suite. The good points about such a test suite are :
> >   _ See quickly what we support at any given point
> >   _ Spot out (even more) problems with plugins
> >   _ Have a system that unifies data to be put in bug reporting from users
> >   _ Allow people to figure out where work is needed
> >
> >   Propositions and comments are more than welcome. IMHO this is a must
> > have for a rock solid GStreamer 0.10
> >
> I agree totally! Some comments:
>
> * shoudn't it be done in check?

  I wouldn't be inclined to do that :
   * This testsuite might take a LOOOONG time
   * it's not only related to one package, but the whole of GStreamer
(core, plugins, python, ....)

> ** to avoid stumbling over stuff in the python layer
> ** check can do XML reporting (if you want it)

  XML reporting in python is way easier than in C

> * we probably need to merge the result as not every build can handle all formats

  Another reason not to do it in check. We could have 'slaves' on
different machines.

> * of course crashes on unhandled media formats are bugs

  Indeed, it should handle them gracefully.

> * alsa has a dummy soundcard and then there is Xvfb, so we can run those test on
> build-slaves too (maybe only on specific ones)

  Good idea, but I think we can already get rid of a LOT of bugs just
by using fakesinks.

>
> Stefan
>

  Keep the ideas coming in, I'll hack up something once there's enough
ideas. Sometime around friday.

   Edward

--
Edward Hervey
Junior developer / Fluendo S.L.
http://www.pitivi.org/