[gst-devel] Proposition for a media test suite

[Edward Hervey]

Hi,

On 11/14/05, Ronald S. Bultje <rbultje at ronald.bitfreak.net> wrote:
> On Mon, 2005-11-14 at 10:56 +0100, Edward Hervey wrote:
> >   I put most of the main lines on gnome live here :
> > http://live.gnome.org/GStreamerMediaTest
>
> - not everybody speaks or wants to speak python. Everybody in our team
> speaks C (or is supposed to).

  One thing is for sure, everybody speaks GStreamer. So understanding
parse-launch syntax and GStreamer API used in python shouldn't be a
problem. Especially since those doing that testsuite master python.
  The whole point of this thread is to gather ideas to make a
testsuite that suit the requirements of most hackers, and increase the
productivity of the GStreamer team.

> - it may be nice to be able to randomly add files that we fix in
> bugzilla. Make sure adding files is easy for maintainers.

  Of course. As Tim proposed, people could post a URI and and MD5 and
the script could download those auto-magically.

> - possibly extra tests for metadata, video snapshotting, stream
> properties etc. (see the nautilus property pages, thumbnailer etc. in
> totem). Also makes for nice extra visual effects in the status page for
> the testsuite (people love this).

  Metadata is already done in gstfile.py so it's not a problem, and so
are stream properties. For thumbnailing, that will come in a second
step I guess.

> - it has never been clear to me how such things are to be handled in a
> non-playback fashion. Is a seek event in GST_STATE_NULL allowed? I
> assume that we want to be able to set a pipeline to PAUSED but not get
> any frames until after we've done a seek (for editing purposes).
> Problem: how do we handle decodebin-pipelines then (where pipeline setup
> is equal to preroll because it's the same statechange)? Once we start
> doing this kind of stuff in testsuites, we want such things to be
> thought out first.

  Seek in STATE_NULL : NO !
  Seek in STATE_PAUSED : the pipeline will preroll, so a buffer will
arrive in all sinks
  Seek in STATE_READY : the seek event is taken into account for when
switching to PAUSED
  When going to PAUSED, decodebin prerolls (as every element), you
dynamically add the elements and set them to PAUSED. Once all elements
are in PAUSED, the pipeline will complete the next step
(PAUSED->PLAYING).

  Apparently you haven't had a look at gstfile.py since it does
EVERYTHING you have mentionned above...

> - what is the difference between SEEK_FLAG_KEYFRAME and no seek flags
> for the visual user perception anyway? How do we test and judge
> compliance? Is 0.1s off on the next buffer for _EXACT wrong? For 1fps
> avi? For 96kHz mp3? And so on.

  What API are you talking about ?
  No flags set on seeks : Try to seek to the given positions
  GST_SEEK_FLAG_KEY_UNIT : Seek to the previous keyframe. Implemented
in oggdemux and avidemux.
  GST_SEEK_FLAG_ACCURATE : Seek to the exact position, if the element
cannot (avi without index for example), return FALSE.

> - Should plugins who don't implement any special seeking mode ignore
> seeking flags? Audio plugins can probably ignore _KEYFRAME, but should
> they error out if they only match the nearest frame while _EXACT is
> given?

  Yes, they should error out. A segment helper file is coming to core
shortly to implement that efficiently in all plugins.
  Plugins should implement *completely* and *properly* the whole API.
We don't want to do the same thing as the previous versions of
GStreamer. This is also a point where the testsuite is meant to speed
up finding out those problems.

> - how do we test output? Not all formats are bitexact. Not all formats
> give the same output depending on library version, arch/os/compiler (and
> this is not a bug!), optimizations used (this is not a bug either!),
> etc.; so we cannot use md5s to test, or we'd need whole series of md5s.

  For a first step, we don't test output. We'll already be able to
crop out a hellovalotov'bugs just getting full API compliance.

> - Also test ffmpegcolorspace behaviour on all supported colourspaces
> (although this should probably be done in make check).

  This should be done separately.

>
> And after reading the l.g.o page:
> - what is the relation between filler events and
> GST_BUFFER_FLAG_DISCONT? How's the filler story going anyway?

  dunno, will inquire.

>
> Ronald

   Edward

>
>


--
Edward Hervey
Junior developer / Fluendo S.L.
http://www.pitivi.org/