[Swfdec] Get URL handling, A/V decoding

[Benjamin Otte]

On Sat, Sep 6, 2008 at 7:14 PM, Nick Zitzmann <seiryu at comcast.net> wrote:
> Swfdec for Mac OS X is coming along well. Watching everything come
> together is actually quite magical.
>
That sounds like my APIs are nice to use. That's cool. :)

> 1. How do I intercept the ActionScript "get url" command? I know the
> player has to handle this by instructing the browser to navigate to
> the URL, but searching through the code, I can't find any obvious
> signal that gets sent out or anything similar...
>
The GetURL command is translated into 3 different possible things:
1. The least important one is the SwfdecPlayer::fscommand signal,
which is used for fscommands, like quit, fullscreen, scaling etc. A
browser plugin will likely not use any of these, the testsuite makes
heavy use of it.
2. Next (and likely the one you asked for) is the SwfdecPlayer::launch
signal, which is used to indicate that an external URL should be
opened.
3. The last thing is GetURL with "_level0" etc, which gets translates
into loadMovie commands and therefore gets handled by SwfdecLoader and
the SwfdecPlayer::loader-type property

> 2. I looked around for documentation, but couldn't find anything about
> the output expected by a decoder. I'm interested in writing a decoder
> that uses Mac OS X's Component Manager instead of GStreamer so I have
> one less thing to install on users' computers, but I'm not sure what
> to ask the component for output. For audio, do I need to use PCM?
> AIFF? What format do I need to use for video?
>
Yeah, that part isn't documented, because I didn't want to export API
for writing decoders yet as there wasn't a user for it.
You obviously found the SwfdecAudioDecoder and SwfdecVideoDecoder
objects already, so here's a small introduction that describes the
vfuncs their classes use. Of course, it doesn't hurt to look at the
existing decoders. The builtin ones are pretty simple.

A SwfdecAudioDecoder works by getting input in the codec/format
combination it was created with - SwfdecAudioFormat is probably pretty
self-explanatory, the codecs can be any number, but you'll likely only
handle some/all of the ones listed at the top of the header (like
SWFDEC_AUDIO_CODEC_MP3). Decoders must provide output as 44.1kHz
stereo 16bit raw in full samples. It's the decoder's job to convert
the data to this format. If you encounter an error during decoding,
just call swfdec_audio_decoder_error() and return from the current
function. The decoder will not be called again after this.
prepare: Gives you a codec and format and asks you to figure out if
you can handle that combination. If yes, return TRUE. If not, return
FALSE. The missing variable is an out variable that can optionally be
set if a decoder class does not handle a format. It's supposed to be
in the GStreamer missing plugins format, so you likely don't wanna use
it. It gets used for emitting a SwfdecPlayer::missing-plugins signal.
create: create a decoder for the given codec/format. If you cannot
decode the given format, return NULL. Note that the codec can be any
value, not just the ones listed.
push, pull: Those are already documented in the .c file. Please keep
in mind that push() gets data directly out of the Flash file, so no
verification on it has been done and it can be completely bogus. It's
your job to verify it.

A SwfdecVideoDecoder is similar to an audio decoder, yet slightly
different. It, too, gets created with a codec, and then the decode
function is called repeatedly to decode the next data and update the
current decoder's image. The decoder is supposed to keep track of the
current image and update it when decode() is called.
prepare, create: The same as for the audio decoder
decode: Decode the given buffer to the expected format and fill in the
values marked protected in the SwfdecVideoDecoder struct.
the expected format is returned by swfdec_video_codec_get_format() and
is based on the formats GStreamer decoders provide.
If there's an error during decoding, call swfdec_video_decoder() and return.
The expected values for the decoding process are:
- width, height: size of the just-decoded image
- plane, rowstride: If RGB, just plane[0] and rowstride[0] are used to
provide a pointer to the image data. If It's I420, the plane variables
describe the Y, U and V planes of the image.
- mask, mask_rowstride: If the video codec can provide translucent
images, set these to an A8 masking plane and rowstride. Only VP6 alpha
does that, and Swfdec has its own decoder for this that uses the VP6
decoder, so you likely don't need to touch it.
An important point here is that your decoder is responsible for the
memory management of all the pointers it sets.

Oh, and be sure to swfdec_audio/video_decoder_register() the decoders
in swfdec_init.c.

If there's any problems with this API and implementing it with Apples
multimedia APIs, tell me and we'll fix it. So far it's been modeled to
work best with GStreamer (and to an extent FFmpeg).

> 3. Speaking of components, does anyone know of a good Nellymoser
> codec? I searched around on Google and found one, and it was taken
> down by a DMCA complaint. :(
>
I guess there is none that you will find that won't get you DMCA'd. ;)
But as Ricardo said, ffmpeg people might know where to find one if you care.

Cheers,
Benjamin