General Simple Question

Sat Sep 24 07:32:13 PDT 2011

Hi,

On Fri, Sep 23, 2011 at 9:29 PM, Rafael Guerra <rafael at lanline.com> wrote:
>
> Sean,
>
> Thank you for your input below. I am sorry that I was so general.
>
> Here is more detail.
>
> I will be using Linux or FreeBSD. Would one be preferred for gstreamer ?

I won't start an OS flame war by taking sides. I'll say this:
Gstreamer *runs* on both of these platforms. I'll leave it at that.

> I would basically like to set up a program to attach a stand alone video
> camera that provides the video
> using network packets over IP. I would like to stick to well known
> protocols.

It's good that *you'd* like to stick to well-known protocols, but if
your IP camera's firmware decides it wants to use a custom protocol,
there's not much you can do, right?

So you have to gather that information before you can really start
your project: *what* protocol does your particular IP camera use?
Without any hardware specifics, I can't tell you the answer.

You'll also need to know if your camera comes with built-in
functionalities of a NVR (Network Video Recorder) or if you'll need to
do that on your end.

And is the protocol "push" or "pull"? That is, does the camera try to
establish a remote connection to push the data to your computer, or
does it listen as a server and allow you to make the connection
whenever you want (presumably to fetch real-time video stream)?

There are a lot of different types of IP camera out there, and the way
that it operates at the protocol level will make all the difference in
what your solution will look like.

>
> So basically I would configure the camera's ip address and protocol and once
> the camera is turned on the video
> would be collected and saved to a file format to be read later and displayed
> on a web browser.
>
> Do you have any suggestions on how to get started in terms of sample
> programs? I would think that the simple case of
> one camera being recorded for a set amount of time has been done in the
> past. Once I get that step I will build
> up the logic around it.

There are some samples in the gstreamer sources, of course, and if you
just google around you can find a lot of production code using
gstreamer.

Once you understand the core framework of gstreamer, you will probably
be able to get started on constructing your gstreamer pipeline even
without knowing where the data will be coming from! Since gstreamer is
so modular, you can use a "mock" audio/video source (such as filesrc,
the element that reads data from a regular file) and run it through a
pipeline to do the processing you want.

You'll have to use some facility to do the timing of the state control
of the pipeline, which is how you change the pipeline from the
"playing" state (where data is flowing) to the "paused" state (where
data movement is blocked). Since gstreamer requires GLib, you may as
well use a periodic timer function as part of GLib's mainloop library.

So very roughly, your prototype code will look something like:

1. Initialize gstreamer (which IIRC automatically initiates gobject
type system as well), i.e. gst_init()
2. Set up the GLib mainloop
3. Set your glib mainloop timer(s) for slicing the video into chunks,
and define the callback function for when the timer expires. The
callback will need to call the string manipulation helper and control
the state of the pipeline.. I'm sure you can figure out the logic.
4. Write some string manipulation helper function to determine how to
name the files, based on user input or time or sequential or whatever
you want
5. Make a new gstreamer pipeline, either using repeated
GstElementFactory.make() or GstParse, linking the elements together
with e.g. gst_element_link_many() if you *don't* use GstParse
5a. The elements to include in the pipeline will minimally consist of
a source (where the data comes from) and a sink (where the data is
sent to). The source you use depends entirely upon the IP camera
hardware, and you might have to write a new sink class if your
hardware uses a custom protocol, as I've said. The sink, in your case,
seems like it'd be filesink. Between the source and sink, if you want
to transcode the raw data sent to you by the camera, you can put a
decodebin2 followed by some element to re-encode the data in the
format you desire, or you can use encodebin. You might want to throw
in a queue element to allow data to pile up in RAM, in the event that
the sink end of the pipeline can't process fast enough to keep up with
the source. Otherwise you'll get an overrun.
6. Wire up some miscellaneous callbacks (maybe listen on the GstBus so
you can get event notifications if the IP camera gets disconnected
from the network or something, for robust error handling). Start the
pipeline playing for the first time.
7. Run the mainloop, then sit back and enjoy the show!

Instead of a glib timer, you could instead use GstMultiFileSink, but
it's not as flexible with file names, and you can only convince it to
"split" the files if one of two conditions is met: upon a
"discontinuity" (e.g. if the source stops sending data), or upon the
next buffer being sent. If you set the buffer size large enough for
the periodicity you're looking for, you might be able to get the
next-buffer functionality to work, but if you want the time (or the
filenames) to be variable / dynamic at all, you'll have to code it up
yourself.

Since it seems from your email address that you're working for a
consultancy company, I would hope you will be able to figure out most
of the actual implementation by reading documentation (APIs, gstreamer
devel docs on http://gstreamer.freedesktop.org, Google) or by trial
and error -- since, you know, they're paying you to figure it out ;-).
But some parts of the gstreamer API are understandably confusing, and
sometimes the semantics of specific APIs is unclear.

If you do encounter a situation where you are trying to use an API but
it doesn't do what you think, and the documentation doesn't provide
any explicit advice, you might want to just ask on IRC. You can get a
much faster answer, and, most specific questions like that can be
answered in just a few words. You were wise, however, to ask such
general questions here on the mailing list instead.

I intentionally did not provide many function names for the prototype
I sketched out above, and it's all from my memory besides. Gstreamer
has fairly good development documentation; you should be able to match
the keywords in my prototype to class names and/or function names.

LIMITATIONS of the prototype I sketched out includes, for instance,
that time-based slicing doesn't account for protocol lag. If you don't
have a zero-hop or one-hop LAN connection between your IP camera and
your computer, the chances of a pipeline stall due to source
starvation (can't get data from the camera at 1.0x real time)
increase. Especially if you've got the public Internet sitting between
your computer and your IP camera. One of the big advantages of
GstMultiSink is that its timing is integrated into the pipeline, so it
will reliably slice the files by exactly the parameters you specify.
Of course, buffer size (in bytes) is not exactly the most
user-friendly tunable I could conceive of, when what you *really* want
to control is time, rather than file size. Also, I didn't include any
logic for handling multiple cameras. One way to approach that (just
brainstorming) would be to concurrently run a mainloop/pipeline pair
in a sequence of threads, or even separate processes, for each IP
camera.

Finally, I can't guarantee that anything I've written here is accurate
or represents the current state of gstreamer :) I'm just basing this
on past experience, as I don't actively work with gstreamer on a day
to day basis anymore (though I used to). I know that you are a
consultant, but if you feel extremely overwhelmed by all of this, you
might need to pay a "sub-consultant" ;-) to further help you develop
your design and implementation. There are at least a couple companies
out there that house and pay for some of the maintainers of gstreamer,
who obviously know this stuff better than anyone else, because they
wrote it!

The last suggestion I have for you is to just try stuff. Experiment.
Start out by using a programming language that isn't so godawful
verbose, so you don't get bogged down by mechanics such as memory
allocation or syntax. Python and Vala each have quite good bindings
for Gstreamer, and provide a level of succinctness that C/C++ can't.
Their support for subclassing existing GObject classes (such as the
gstreamer base source class, which you might be getting *very*
familiar with) is an order of magnitude simpler than doing it in C. At
least, that is the case for Vala; I'm not sure about Python. Once
you're more familiar with the API, you should be able to easily
transfer your work to a C or C++ program if that is your desire.

HTH,

Sean

>
> I am an experienced programmer in C and C++ but do not want to do the work
> of writing and testing code that is already available.
>
> Rafael
>
> -----Original Message-----
> From: gstreamer-devel-bounces+rafael=lanline.com at lists.freedesktop.org
> [mailto:gstreamer-devel-bounces+rafael=lanline.com at lists.freedesktop.org] On
> Behalf Of Sean McNamara
> Sent: Friday, September 23, 2011 7:26 PM
> To: Discussion of the development of and with GStreamer
> Subject: Re: General Simple Question
>
> Hi,
>
> On Fri, Sep 23, 2011 at 6:58 PM, Rafael Guerra <rafael at lanline.com> wrote:
>> I am looking to write a process under Unix that will record 5 minutes of
>> video input from various ip cameras, of different types,  and save them
> into
>> files, to be
>>
>> played back later.
>
> What's "Unix"? AT&T UNIX from the 70s? Solaris? BSD? Linux? The term
> "Unix" doesn't really refer to anything anymore unless you're actually
> running that proprietary operating system from the 70s.
>
> The good news however, is that almost any platform "with Unix-like
> interfaces" does support Gstreamer. :) So almost regardless of whether
> you meant Solaris, BSD, Linux, HP-UX or any of the other platforms
> supported by Gstreamer, the core libraries at least will work, and
> most of the plugins should also be available.
>
>>
>>
>>
>> I am new to gstreamer but from a brief look at the documentation it looks
>> like it might be a good fit for my requirement.
>
> Yes, Gstreamer is a good library for writing programs with complex
> business logic, but with the desire to not have to deal with the
> implementations of codecs and other such low level stuff. As long as
> gstreamer elements exist for the file formats, codecs, and devices
> you'll be working with, the API itself is extremely flexible. And
> there's already a lot of existing code out there for a variety of
> codecs and devices.
>
>>
>>
>>
>> I would appreciate any feedback and guidance. I am also interested in how
>> growable and versatile this solution would be.
>
> For a general question, you'll get a general answer ;) Gstreamer is
> about as flexible as they come. It's as flexible as you want to make
> it, mainly because it's open source, so if it doesn't do what you
> want, you can make it do so with some code. The extent that your
> program will be "growable" will depend on the flexibility of your
> software architecture, choice of programming language / environment,
> etc.
>
> My biggest immediate concern with your description is "ip cameras".
> What do you mean? I assume you mean a standalone device that
> communicates using network packets over IP. Well that's great, but the
> range of possible ways that they could interface with other computers
> over IP is too broad for me to know if gstreamer would support your
> particular devices.
>
> One possible "IP camera" implementation that would be supported almost
> out of the box in gstreamer would be if it broadcasts streaming video
> over a well-known protocol, such as RTP or even just plain HTTP.
> Provided that gstreamer has a codec that can decode the encoded file
> received from the camera, the protocol layers I mentioned are already
> well-supported in gstreamer.
>
> But of course, it's entirely possible (and probable, I guess) that any
> random IP camera would use some proprietary network protocol that's
> only designed to work with some proprietary software distributed only
> to run on Windows. That wouldn't surprise me one bit. The bad news
> then, is that gstreamer probably would not support that out of the
> box, and you'd have to actually write a source element implementation
> for the camera's custom wire protocol (which might involve reverse
> engineering it from the network packets, etc). If you can do *that*,
> then the modular nature of gstreamer's elements will seamlessly take
> care of transferring your data stream to decoders, encoders, and any
> output element (a file, in your case) that you want.
>
> Hard to be any more specific than that with the very little info
> you've provided, but hope it helps.
>
> Sean
>
>>
>>
>>
>> Thank you in advance for your help.
>>
>>
>>
>> Rafael Guerra
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>