Capturing jpegs from an h264 stream

[Alex Hewson]

Hi Tim,

Thanks for the advice - answers inline.



On 05/06/2013 19:33, Tim-Philipp Müller wrote:
> a) why not reduce the framerate before feeding it into the encoder? Do
> you want the full feed as well, in case something interesting happened?
Precisely that that - once I get the jpeg output working for motion 
detection I'll add a tee which saves the full, unprocessed h264 stream 
to a file (actually a pipe) so I can splice together a video with 30s of 
extra footage around the detected event.  What I'm building is 
essentially a motion sensitive security camera based on a raspberry pi 
and the camera module.

> b) you could try using gdppay ! fdsink/tcpserversink  and then
> fdsrc/tcpclientsrc ! gdpdepay - that will pass through the original
> timestamps from the sending pipeline. Make sure to use something like
>
>    dec ! videorate skip-to-first=true ! video/x-raw,framerate=1/1 ! enc

I've given this a go but to no avail.  Here's what I tried:

raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - |  \
   gst-launch-1.0 fdsrc fd=0 ! decodebin ! gdppay ! fdsink fd=1 | \
   gst-launch-1.0 fdsrc fd=0 ! gdpdepay  \
     ! videorate skip-to-first=true ! video/x-raw,framerate=1/1 ! 
jpegenc quality=75 \
     ! multifilesink location='img_%04d.jpg'

The pipeline sticks in 'prerolling' and I get a lot of warnings like:

(gst-launch-1.0:4360): GStreamer-WARNING **: 
gstpad.c:3908:gst_pad_push_data:<gdppay0:src> Got data flow before 
stream-start event

(gst-launch-1.0:4360): GStreamer-WARNING **: 
gstpad.c:3677:gst_pad_chain_data_unchecked:<fdsink0:sink> Got data flow 
before stream-start event

Tried moving the 'videorate skip-to-first=true' into the first gstreamer 
instance.  I get the 'Got data flow before stream-start event' error 
three times but still gstreamer won't leave 'prerolling'.


> c) if the sender produces data in real-time (live), then you could
> probably just tell the source to timestamp according to the clock with
> do-timestamp=true. That will also put timestamps on the buffers.
Yes it does - raspivid simply produces a live stream of what the rpi's 
camera sees.  Presently the back of my head :)

For this I've tried:

raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - |  \
   gst-launch-1.0 fdsrc fd=0 do-timestamp=true ! decodebin \
     ! videorate ! video/x-raw,framerate=1/1 ! jpegenc quality=75 /
     ! multifilesink location='img_%04d.jpg'

root at raspberrypi:~/streamtest# ./l2.sh
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

(gst-launch-1.0:4247): GStreamer-WARNING **: 
gstpad.c:4503:store_sticky_event:<typefind:src> Sticky event 
misordering, got 'caps' before 'stream-start'


...but it sticks in the prerolling state.  Tried adding the 
do-timestamp=true to decodebin but nothing changes.


> d) even if the h264 data going into h264parse is not timestamped, I
> would expect h264parse to timestamp if you put a capsfilter with a
> framerate in front of it (or after it?). (If not, that should be fixed
> imho).

So I have tried:

raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - |  \
   gst-launch-1.0 fdsrc fd=0 ! decodebin \
     ! capssetter caps='video/x-raw,framerate=25' replace=true \
     ! videorate skip-to-first=true ! video/x-raw,framerate=1/1 ! \
     jpegenc quality=75 ! multifilesink location='img_%04d.jpg'


 From this I get:

root at raspberrypi:~/streamtest# ./l2.sh
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

(gst-launch-1.0:4108): GStreamer-WARNING **: 
gstpad.c:4503:store_sticky_event:<typefind:src> Sticky event 
misordering, got 'caps' before 'stream-start'
ERROR: from element 
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0: 
Internal data stream error.
Additional debug info:
gstomxvideodec.c(1670): gst_omx_video_dec_loop (): 
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-negotiated
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

It may well be (is likely!) that my pipeline elements are in the wrong 
order as I'm a novice to gstreamer.  I've tried moving the capssetter 
before the decodebin but then gstreamer never leaves the 'prerolling' state.



-- 
Alex Hewson
m: +44 7895 265219 | e: mock at mocko.org.uk | Skype: alex.hewson