[gst-devel] Summer Coding: Screen Recording on the Free Desktop

[cellenro cellenro]

Posted to kde-devel + gstreamer-devel due to interest
of having both perspectives.

There is a clear need for an integrated screen
recording program that covers recording, editing, and
publishing with a focus on usability.  Current tools
make it difficult compared to other platforms to
record and share knowledge effectively on GNU/Linux.
For example the process the Ubuntu screen cast team
uses
(https://wiki.ubuntu.com/ScreencastTeam/RecordingScreencasts)
is more complex than the situation on proprietary
operating systems with already established screen
recording software.

Ogg Theora is not the best codec for screen recording
because even at higher resolutions it is hard to see
text clearly. Part of the project will be to evaluate
the feasibility of using Driac as the codec for
recording and editing. Below is an outline of the
proposal. I look forward to constructive feedback.


Technology:
	Qt Jambi 4.4 -
(http://trolltech.com/company/newsroom/announcements/press.2008-03-11.2168449860)

	JRuby - jruby.codehaus.org
	GStreamer gstreamer.freedesktop.org
	Schrödinger implementation of Driac (diracvideo.org)

License:
	GPL v3

The language used will be Ruby, with the graphics
framework Qt, and making use of  Schrödinger and
GStreamer components.  The program will be released
using the GPL v3 license, although the current Qt
Jambi preview license may prevent this until Qt 4.4 is
officially released under GPL v3 also.


Format
	support Driac/Schrodinger via GStreamer plugin
	GStreamer for rendering / non-linear video editing --
using Ruby bindings
		- preview -- playbin (like in totem?)
		- video editing -- GNonLin
	http://www.xfree86.org/current/xwininfo.1.html --
looks useful

Based on my research the above technology will be
useful to fully implement the specification of the
screen recording program (found below).  I hope to
gain a better understanding of the optimal way to
create the application with fully open source software
and no patented encumbered codecs.  So if you know who
I should be talking to (or links to additional
resources), please let me know! This will be my first
significant independent open source project and I look
forward to learning from other free software
developers. Based on the technology choices it seems
that the KDE and Gstreamer communities are a good
starting point.


Platforms:
	The application will be developed against Ubuntu 7.10
and created from the start with the intent to be
cross-platform. The screen recording software will
first fully support Linux and then Windows and OS X
(Intel). Windows and the Mac OS already have
professional grade screen capture utilities, although
all platforms have a lack of open source options. This
project aims to change that.


Work flow:
	1.) Select -> 2) Record -> 3) Preview -> 4) Edit ->
5) Render

	Screencasting occurs in five phases. 
	First the area to be recorded is selected, then that
area is recorded, after recording the user decides to
keep or retake the video, and then the video is
edited, finally the video is published for consumption
in the desired format. 

Development direction:
	The software development will initially focus on the
core of the program, which is selecting and recording
the screen, before implementing other functionality.

	Priority 1
		Selection & screen recording (Driac codec)
	Priority 2
		Audio recording 
	Priority 3
		Video / Audio editing


1) Select Area to Record -- illustrative gui img of
record
(http://img338.imageshack.us/img338/2264/34503177vy4.png)


Audio and additional video sources (ex webcam), if
any, chosen by the user. 

Two modes of selecting	
	0) draw rectangle on screen (8 resize handles, able
to move by dragging in center) . This is similar to
xvidcap except the selection is movable.
	
	1) select window (interactive selection by moving
over window, frame highlights, then click to select).
Once selection of area is made it is refined in either
of two ways.  
A: interacting with the drawn selection grid -
dragging on resize handles, dragging in center to move
(inspired by GIMP rectangle select tool)
B: interacting with the gui widget - typing in new
height/width values, selecting saved height/widths
from past sessions
Note: if window was selected, resizing selection (via
selection grid or gui widget) will cause the window to
resize. The selection grid becomes the resize handler
of the window. 
	2) select components of window, for example just want
the canvas of the gimp program -- this enables
automatic detection of area size without having to
precisely draw out a rectangle

2) Record
After selection is made and record button is pressed
then the application goes into recording mode.
Selection grid changes color to denote the new mode.
In recording mode the user is able to pause/resume,
finish,  or cancel the recording
	
3) Preview -- illustrative GUI img of preview 
(http://img259.imageshack.us/img259/4373/65638300ql4.png)
After recording is finished the preview mode is
entered. In this mode the user is able to quickly
review the recording just created and decides to save
or delete it.  After choosing, presented with the
option to go into edit mode or back into record mode.

4) Edit
	Edit mode, functionality related to what pitivi
(pitivi.org) attempts to do.
	Must support deleting selected parts of recording,
splicing recording into two recordings at specific
point
	Time line should support reordering of the recordings
	With audio component, time line should have audio
editing functionality integrated.  audio functions
largely similar to jokosher (jokosher.org)

5) Render
	render to desired output format for publishing, ex:
FLV for web video. 

Potential future features:
	1. custom xml based format for saving extra data
generated at record time to be useful in editing
automation. (in addition to saving the video)
	2. inserting geometry/text into the video, drawing
attention
	3. webcam / external video integration -- 'picture in
picture'
	4. capturing of opengl ex physics simulations (Phun) 
	5. playback faster/slower -- variable speed
	6. capturing vector screenshots of applications (http://lists.freedesktop.org/archives/cairo/2007-April/010256.html)


      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs