[gstreamer-bugs] [Bug 163577] New: [RFC] Expression of interlaced/progressive media in GStreamer.
bugzilla-daemon at bugzilla.gnome.org
bugzilla-daemon at bugzilla.gnome.org
Mon Jan 10 07:15:47 PST 2005
Please DO NOT reply to this by email. All additional comments should be made in
the comments box of this bug report.
http://bugzilla.gnome.org/show_bug.cgi?id=163577
GStreamer | gst-universe | Ver: HEAD CVS
Summary: [RFC] Expression of interlaced/progressive media in
GStreamer.
Product: GStreamer
Version: HEAD CVS
Platform: Other
OS/Version: Linux
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: gst-universe
AssignedTo: gstreamer-bugs at lists.sourceforge.net
ReportedBy: meiker at upb.de
QAContact: gstreamer-bugs at lists.sourceforge.net
CC: all-bugs at bugzilla.gnome.org
[RFC] Expression of interlaced/progressive media in GStreamer.
================================================================
This is a write down on how interlaced video can be handled in GStreamer.
Without the intend to be complete or final, I'd like this document to be
a starting point of a discussion and refinement process with the ultimate
goal to introduce a sufficient notation/handling of interlaced video in
GStreamer. Where unsure about naming/something else parts in {} braces.
Naming:
-------
Frame {Picture}: Top Field and Bottom Field (interleaved or se-
quential) where of this fields is meant to be dis-
played first. A Frame may also be one progressive
image representing image data from only one point
in time.
Field: Top Field or Bottom Field
Top Field: Sequence of interlaced scanlines containing the
first scanline. (e.g. 0, 2, 4, 6,...)
Bottom Field: Sequence of interlaced scanlines containing the
last scanline. (e.g. 1, 3, 5, 7,...)
Sequential Layout: Layout of two fields in a frame such that all scan-
lines of the first field in time appeare before all
the scanlines of the second field. Just by knowing
a frame contains a sequential field order we still
miss the information whether the bottom or the top
field comes first.
(e.g. 1, 3, 5, ... 0, 2, 4,...
or 0, 2, 4, ... 1, 3, 5,...)
Interleaved Layout: Two fields interleaved (e.g. 1, 2, 3, 4, ...). In
this case it remains to be determined which field
comes first in time.
Top Field First(TFF): W.r.t. time in the case of interleaved order (the
Top Field has a timestamp lower than the Bottom
Field).
W.r.t. spacial order in the case of sequential
order of fields. (Read: Top Field is the first in
that frame)
Bottom Field First(BFF): W.r.t. time in the case of interleaved order (the
Bottom Field has a timestamp lower than the Top
Field).
W.r.t. spacial order in the case of sequential
order of fields. (Read: Bottom Field is the first
in that frame)
Use cases:
----------
Deinterlacing: The intend of displaying interlaced video on a progressive
output device requires the video to be deinterlaced. There
are various method for deinterlacing a stream of fields. The
simplest are weave and bob. The first combines two fields in
one frame and thus halves the framerate. In addition, this
method generates visible comb artefacts. The latter method
interpolates missing scanlines. In GStreamer all video is
treated as progressive. Either interlaced video has been
weaved before/during encoding {or is weaved by source plugins
v4lsrc?}.
Mixing: Mixing of interlaced video with different properties (TFF vs.
BFF) is currently done on the base of weaved fields. Each
frame represents two different timestamps. So information of
four different points in time may end up in one result of the
mixing process. Mixing should just combine two time bases.
Implementation in GStreamer:
----------------------------
Assumtions:
* Currently every video element can handle interlaced video, if it comes in
frames and fields are interleaved.
* Sequential fields are not supported by now.
Requirements:
* An element which can process progressive video, should not bail out on
interlaced video served as interleaved frames. (e.g. ffmpegcolorspace is
fine accept anything w.r.t. interlaced video)
* If an element emits frames with sequential order of fields any plugin
connected to its source pads must signal somehow that it can process
sequential field order. Otherwise the source must abort negotiation and
return GST_PAD_LINK_FAILED.
* If an element emits single fields any plugin connected to its source
pads must signal that it can process single fields. Otherwise the source
must abort negotiation and return GST_PAD_LINK_FAILED.
* Elements arbitrarily emitting frames or fields are evil.
{
They should either require the
same capabilities as field only emitting elements. Or drop incidental
occoring single fields (mpegdemux: NTSC - 3:2 pulldown media might con-
tain incidental single fields in case of an encoding error)
* Elements accepting single fields must also accept frames.
}
* Field order, TFF/BFF are not expected to change during a stream. (If it
should change, caps must be renegotiated e.g. mpeg allows this to change)
* Supporting single fields per buffer requires the buffer to hold informa-
tion if the contained field is a Top or Bottom field. {do we want to do
that}
New properties for mime type video/*:
- (bool) interlaced: TRUE, FALSE
- (bool) top field first: TRUE, FALSE
- (string) field order: SEQUENTIAL, INTERLEAVED
{- (bool) fields only: TRUE, FALSE} // whether one field one buffer
is supported
With "interlaced= FALSE" the two latter properties have no meaning { are not
defined }. Where nothing else is stated "interlaced= FALSE" is be assumed.
Progressive sinks:
Progressive sinks will have "interlaced= [TRUE, FALSE]" in their caps. A
upstream deinterlacer will happily do it's job and provide "interlaced= FALSE".
Interlaced sinks:
Interlaced sinks will ask for "interlaced= TRUE" in their caps. Any upstream
deinterlacer can just "weave" or pass through interlaced media.
{ (Interlaced sinks could require one field per buffer) }
Deinterlacer:
The deinterlace plugin accepts interlaced and progressive video. In the case
of progressive video deinterlace will pass through all buffers. In the case
of interlaced video the deinterlacer might double the framerate according
to the used deinterlace method and settings. Output of the deinterlacer will
be "interlaced= FALSE" in the case of any method but "weave".
Open questions:
---------------
* Do we want to support fields on a per buffer base?
* Anyone seen any interlaced videosinks?
* Specific caps nego examples are missing.
e.g. for "mpeg2dec ! ffmpegcolorspace ! deinterlace2 ! xvimagesink"
, v4l2src ! deinterlace ! v4l2sink
multifilesrc ! video/x-raw-yuv, format:fourcc=YUY2,
interlaced:true, ... ! pngenc
Thanks for reading all this!
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are the QA contact for the bug.
More information about the Gstreamer-bugs
mailing list