[Bug 642671] New: fieldanalysis: New element for analysing video input to produce progressive output

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=642671
  GStreamer | gst-plugins-bad | git

           Summary: fieldanalysis: New element for analysing video input
                    to produce progressive output
    Classification: Desktop
           Product: GStreamer
           Version: git
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: gst-plugins-bad
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: robert.swain at gmail.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
      GNOME target: ---
     GNOME version: ---


I have been working on an element to analyse any incoming video buffers to
identify whether they are progressive, interlaced or telecined and which fields
to use from them and how. The decisions are based on various metrics with
corresponding thresholds. The default metrics seemed to be the best choice
across the limited telecine samples I have.

The analysis consists of five comparisons:

1. top vs bottom fields of the current buffer
2. top of current vs top of previous
3. bottom of current vs bottom of previous
4. top of current vs bottom of previous
5. bottom of current vs top of previous

This creates two classes of required metrics: same parity (2,3) and opposing
parity (1,4,5). It also introduces a one-frame delay to the video branch of the
pipeline.

Based on the semantics of scores from the above comparisons and using the
assumption that same parity comparisons 2 and 3 are likely more reliable
(they're often mathematically identical or just have quantisation
noise/compression artifacts) I formulated some logic to identify the various
different states and apply flags such that downstream deinterlace can act upon
the identified states to produce progressive output. It seems to work quite
well


Known flaws:

The thresholds will need manually adjusting for different sources. They are not
dynamic to the source. This is something that I would like to have in the
element but it will take some more research and design and more time. I would
rather have this base work completed for now.

I actually found that in practice, often only minor tweaks are required. I will
try to put together some guidelines for the tweaking as a stopgap until some
more automagicness can be done to make the thresholds adjust according to the
current state of the incoming buffers. (Note to self: gradients and perhaps
variance of fields or moving averages thereof.)

The windowed comb method is supposed to be the most reliable and flexible but
in practice I found it very difficult to tweak the thresholds to achieve the
desired result and as such not user-friendly enough.

Also the usefulness of the windowed comb metric is limited because we cannot
communicate to downstream the level of combing and so the level of
deinterlacing to be applied in the interlaced case. This could be added later
as buffer metadata perhaps.

Seeking? I need to look into that a bit more but wanted to get it more visibly
into the wild.

Perhaps some other things I am forgetting right now, but it's been looking
pretty good for a little while now.


Basic usage:

some video input... ! fieldanalysis field-threshold=something
frame-threshold=something ! deinterlace locking=auto ! ...

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.