[Swfdec] as branch progress report

[Benjamin Otte]

Since there seem to be quite some people hacking on the as branch, I'd
like to give some report about its progress. So if you encounter
regressions after a git pull, you know where to look (or who to ask).

For everyone not aware of this, the as branch will become 0.5.0 at
some point. I should probably merge it into HEAD soon and start
releasing it, but I'm still unsure about this, because I'm doing
surgery all the time.


So the first thing that has happened (yesterday), is that I have
enabled the garbage collector. Previously, garbage was only collected
at the end of a SwfdecPlayer's lifetime, which could lead to lots of
wasted memory during a running file. But probably noone noticed, since
noone kept the app running long enough. ;)

Now for a little background on the GC (you might also wanna read [1]):
The garbage collector manages strings and objects. Managed objects
must be subclasses of SwfdecAsObject (note that SwfdecMovie is a
SwfdecAsObject). Strings _must_ be allocated using
swfdec_as_context_get_string() or swfdec_as_context_give_string(). If
you pass in strings that are not allocated using these functions,
you'll both cause incorrect behaviour (see below) and probably
segfaults, since the bytes in front of the string are used for
bookkeeping. An important feature about GC'd strings is that they are
unique, which means only one instance is allocated for every string.
This means you can compare for equality using == instead of strcmp()
or g_str_equal(). And that should speed up variable lookup, which was
a bottleneck in the Spidermonkey engine. There is also the possibility
of pre-allocating static strings inside Swfdec by adding them to the
file swfdec_as_strings.c and then using the SWFDEC_AS_STR_foo macros.

Back to the GC. Garbage collection is done using mark-and-sweep, which
means it collects in two stages: 1) mark all used objects, 2) free all
unused objects. At the end of the context's lifetime, only step 2) is
done, so the everything gets freed. This has been done forever. The
new thing is that since commit
40a2b740ee548eeed3bcbe19f80e3d4590cb4451 a full GC is done after every
frame when more than 8MB have been allocated since the last GC.
Marking objects is done from as_context_class->mark. If an object gets
marked, as_object_class->mark is called on that object. The object
will then call swfdec_as_(value|object|string)_mark() on all objects
it still references. If you implement a custom object, you might need
to override this. (for example the NetStream object will need to
reference its NetConnection).


The second thing that is causing quite some regressions and isn't
finished yet, is the rework of the file parsing. Previously Swfdec
used to parse the whole file into private structures that described
what happens and during execution process this information. This lead
to some obscure bugs, because it seems that the official Flash player
only parses on-demand and interprets the parsed information
immediately. Adapting Swfdec to do this is an ongoing task, and it
happens step by step. Every step can lead to unexpected regressions.
If you encounter them, please tell me. Ideally with a test. ;)

So the first step in this process was getting rid of SwfdecRootMovie.
This was pretty uneventful. It had the side effect of allowing
implementation of the Flash functions createEmptyMovieClip() (in since
a5f1d561b1a24c77ba879af7d004729f6b0e7224) and loadMovie() (not
implemented yet).

The seond step is a rework of placing "static" objects, or the
PlaceObject and RemoveObject tags. This is still ongoing (I intend to
have it done today or tomorrow). It also caused some interesting
regressions already. I'll chase those next. So if some objects in the
Flash files suddenly go missing, have a different color, are at the
wrong place or if some objects show up that shouldn't be there, it's a
fault of this. Poke me about it.
Now, there's of course a positive effect about this. Previously some
objects were replaced (or removed) after a gotoAndPlay/Stop () even
though they were supposed to still exist. This was most visible with
code like this:
some_movie.var = 42;
gotoAndPlay (some_frame);
trace (some_movie.var);
where previously you got undefined instead of 42 as the trace output.
This should not happen anymore after the rework. Again, if it does,
poke me.


So, after all of this, I have some questions to all of you:
1) Should I start doing releases of 0.5, even though it's definitely
less stable, probably has regressions and gets lots of active surgery?
Doing releases is work, so I'd like to have people requesting them.
2) If I start doing releases: At what point should 0.5 be advertised
to distros as preferred to 0.4? Can you think of any guidelines or
should we just throw 0.5 at them and deal with the bugs?
3) This is for all the people hacking on the as branch: Have the
regressions introduced by my surgery caused you problems and would you
prefer me doing them in branches? Should I do more testing before
pushing it? I made sure the testsuite passes, so it wasn't untested
code. However, the testcode seems to be not good enough yet for these
kinds of problems...
4) Is anybody reading these mails? Should I do this more often?


Cheers,
Benjamin


[1] http://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29