[gst-devel] volume plugin with dparams

[Karsten N . Strand]

On 2001.11.15 04:16 Erik Walthinsen wrote:
> On Wed, 14 Nov 2001, Karsten N . Strand wrote:
> 
> > I'm currently looking into gstreamer as the platform for my new digital
> > audio mixing system project, and have a few questions. The plans for
> the
> > mixer is to build a external control board with sliders, aux buttons,
> and
> > two "turntables" (for scratching, searching and looking cool), using a
> > board with a bunch of adc's and a philips P89C51RC+ controller chip.
> The
> > control board will have a serial interface. This step is pretty
> > straightforward, with some kindly given soldering help from my father.
> 
> Very cool.  I've been thinking about the hardware design of something
> like
> that for quite a while.  I want to make sure it's a fully automated
> board,
> though.  Easiest would be the buttons, which would be keyboard-style
> momentary tactile buttons, with an LED for status (preferably the button
> itself would be translucent...).

I like it! Though I have to admit I'm more on a "hacking something
together" stage, and haven't thought so much about details like that.
Translucent, blinking buttons. Fancy, I like it :)

> 
> The sliders would be motorized somehow, I'm thinking of either using a
> regular fader with a motor and a belt, or using a rotary encoder and
> motor, just using a slider body and not a variable resistor.  Using a
> variable resistor fader would be cheaper, but potentially less accurate.
> An optical rotary encoder of that size and resolution could be expensive.

As of now, I have two Alps RSA0K faders, which I will use for testing.
Having the sliders motorized, would oh course be much better, but I think
it will cost a lot of money I don't currently have (I'm a leftover from dot
com these days :P ). But eventually, on the final design, I'm all for it.
We might want to investigate how professional studio mixers do this.

> 
> As for knobs, I've thought of putting a ring of LEDs around the thing
> like
> all the manufacturers do, but realized that it sucks.  Someone pointed
> out
> to me that you don't need fine-grained display of the knob position,
> because it just doesn't really matter.  You just need a high-resolution
> sensor.  Well, that's fine, but LEDs around a knob body must be viewed
> from the *top* down, which doesn't cut it.  So I'm thinking of some
> mechanism using a rotary encoder and a small stepper motor.  Question is
> whether the stepper resolution can be enough, though it's back to the
> argument that where the knob visually is isn't as critical.  That means a
> low-res stepper would work just fine, as long as the system resets the
> precise origin of the knob relative to the rotary encoder each time.

Agreed.

> 
> As for the electronics, I'd use Microchip PIC's for each control group on
> a I2C bus, with some kind of main controller.  I'm liking the SitePlayer
> (www.siteplayer.com), because it's $30 and has ethernet.  Problem is that
> it only has a single SPI port, and isn't all that fast.  But heck, it's
> ethernet <g>

I have a few PIC's laying around, so that would be possible too. Right now,
I use a leftover board (originally made by my father for controlling focus
and zoom motors on a camera) with a Philips P89C51RC+ chip (intel 8051
clone). The adc is a MAXIM MAX1202, which is a 8 channel 12 bit 133khz adc
(with interpolation, I think the precision will do fine). There are three
possible solutions for this:

	choose 8051
	choose PIC
	make the code portable

 To make the code portable, we could use sdcc (sdcc.sourceforge.net). I
have only tried sdcc a little, with a gameboy coding project in the past,
so I don't know if it will be good enough, but we're used to experimental
software, right? :) This would also make it possible having linux as the
main development platform (which is important to me). But I guess there's
others alternatives too, using asm.

On the flip side, multiple designs and portable code, do create some
duplicate effort. But it would also make it easier for more people to
actually build it. The most important thing as I see it, is that the
protocol coming out of the device will be standard.

I don't really think ethernet is the way to go, as it's packet based, which
possibly could introduce some latency. But if I understand right, it needs
a rs 232 serial interface anyway, so using this part would be optional. 

> 
> > The next step is to hook all these controls up to actual mixing
> software,
> > and this is were I thought maybe gstreamer could be The choice. I have
> been
> > following the project for a while, and I really think it's the future
> for
> > streaming media in linux (the gtk/gnome world anyway, it seems like the
> > qt/kde people have somewhat settled for arts). Well, anyway, that's my
> > story.. Here's the questions:
> >
> > Is anyone working on adding dparams to volume?
> I think the current volenv element is what you want, since that was one
> of the test elements (iirc) for dparams.  I'm still a little fuzzy on
> dparams myself, esp in the context of a live application.  steveb, you
> have any comments?

Then I'll take look at volenv :)
> 
> > Are the latency numbers good?
> As long as the input and output devices are good, yes.  The plugins can
> introduce latency, but I don't think any of the raw audio ones do.  The
> cothread switch time is constant and quite small (<1000 cycles per
> switch,
> probably a lot less).  Any threaded pipeline with queues and such will of
> course add relatively unpredictable latency, so you just don't do that
> <g>
> 
> > Any work on some kind of sequencer to record stream changes and dparam
> > values?
> Can you elaborate a little more on what you mean by this?
Ok.. example: In a mixing session, I want to change samples/songs on the
fly, and maybe add a filter midway trough, remove it, and scratch a bit. I
dont want to record the resulting audio data, I just want to record where
in the timeline the different elements are triggered, and when the volume
goes up and down, and the motion data from the turntable. Then I want to
open this sequence in some kind of editor, add filters between the elements
in certain parts of the timeline, and maybe go over the material again,
correcting mistakes (here the motorized faders would be perfect.

> 
> > Any plans for a terminatorX-like* element for gstreamer? terminatorX (
> > http://terminatorx.cx ) is a kickass virtual turntable app for linux,
> but
> > you probably knew that anyway :)
> Hmm, I have no idea what it actually does.  I guess the question is what
> precisely would the element do?  I would guess that it takes some kind of
> data pointer and uses it to output samples from an, um, sample, doing
> various interpolation and stuff.  That would mean it would have to load
> in
> the whole sample up front, which would be interesting.  It could handle
> demand-loading by seeking to the appropriate places in the file. 
> However,
> if you want to display a peakgraph, you'll have to load them all anyway,
> probably either up-front or via a smart tee.  A peakgraph element needs
> to
> be written, along with a really good waveform display widget for gtk....

The element would take dynamic parameters, direction and a speed factor.
Preferably it wouldn't have to preload the whole song, but then it would
require some prebuffering and postbuffering of the data, as the source data
might be compressed, and in the situation of fast searches (backward or
forward), we wouldn't want to risk that there's too little cpu to decode
this data. The peakgraph is not really important for me.

> 
> > I would very much like to help with any of these issues, as they're all
> > vital to my application, but I'm no wizard, so it might take some time
> to
> > me to understand gstreamer's architecture and api completely. I've
> never
> > had any experience with GObjects (just basic gtk+) before, and I think
> the
> > official docs is pretty sparse at this moment, so I would be grateful
> for
> > pointers to good alternative documentation.
> Unfortunately, the docs on gtk.org are about it, besides the source,
> which
> is of course always canonical.  The API freeze is in place, so the docs
> should soon match the source, but API docs aren't quite enough usually
> ;-(
> It took me several tries to start effectively using GtkObject, so....

I have been reading a lot the last days, so the it's starting to look a bit
clearer now.

> 
> > If anything useful come out of the mixer project, I will (oh'course)
> > release the code and schematics under lgpl, gpl. And then, maybe
> someone
> > else will want to be a part of the project too, and we will take over
> the
> > evil music industry together, and ride happy into the sunset :)
> Sounds good!  If you want to collaborate on hardware design, let me know
> and I'll start preparing to build stuff (i.e. clean my room <g>, acquire
> some hardware and devel kits, etc.).  If we can do an open-source control
> surface, that'd be very cool <g>

Ok! We'll do it! Then all we need is a project name, some prelim
documentation, and a place to put it on the net (sourceforge, right?).

Karsten N. Strand (BeeWarloc on irc)

Member of LUG: NLC (Norwegian Linux Community)