[uvch264src in 1.1.1.1] "Got data flow before segment event" warnings until crash

Sat Aug 10 18:20:24 PDT 2013

Hi Peter,

I found the problem.  In v4l2src, four buffers are acquired from the UVC
driver, and each is mapped to user space and each wrapped in a GstMemory
object which is then appended to a GstBuffer.  However, in v4l2src
gst_buffer_resize() can be called on a GstBuffer and this function
allocates a new GstMemory object with an appropriately sized data area and
copies the contents of the current GstMemory object's data area in to it.
 The old GstMemory object with a data area comprised of one of the four
buffers previously acquired from the UVC driver is then replaced by the new
GstMemory object whose data area was allocated in user space of which the
UVC has no knowledge.  The replaced GstMemory object is unreferenced
(deallocated) and the UVC driver buffer is orphaned and it's lights out for
the pipeline.  I was able to hack a fix by removing the to instances of
gst_buffer_resize() and placing the real size from vbuffer.bytesused in the
GstBuffer metadata in v4l2src and then retrieving it from the GstBuffer
metadata in uvch264src.  I cannot think of a proper way to fix this right
now.

Best Regards,

Rob

On Fri, Aug 9, 2013 at 3:39 PM, Robert Krakora <
rob.krakora at messagenetsystems.com> wrote:

> Hi Peter,
>
> Yes, I made an array of garbage so that GstBuffer would not
> complain...v4l2src places a v4l2_buffer in the GstBuffer meta data as well
> as a pointer to the real memory that contains the JPEG data.  When the
> GstBuffer gets to the uvch264src plugin, I modified the source to retrieve
> a pointer to the real memory containing the JPEG data from the GstBuffer
> meta data.
>
> The difference between what is done in regards to v4l2src versus other
> plugins that I know of is that buffers are allocated via the UVC video
> driver from kernel space, mapped to user space and wrapped in a GstBuffer.
>
> Best Regards,
>
> Rob
>
>
>
> On Fri, Aug 9, 2013 at 3:34 PM, Peter Rennert <p.rennert at cs.ucl.ac.uk>wrote:
>
>>  Hi Rob,
>>
>> I think GstBuffers and GstMemory goes far beyond my knowledge about
>> gstreamer. But if it would be a general error related to GstBuffer and
>> GstMemory, would that not lead to other applications fail as well after
>> some time?
>>
>> But what you were saying is that you can insert as often as you want
>> "fake" (but still valid) JPEG data into v4l2src and it will not crash? I
>> did not even try yet to capture from the cam with v4l2. Did you try that?
>>
>> I was also thinking to try to stream a video with v4l2sink and capture it
>> using uvch264src (not sure if they are compatitble). Do you think that
>> should crash as well? I am sorry its getting late here. I will try and do
>> some experiments tomorrow on that.
>>
>>
>>
>>
>> On 08/09/2013 08:03 PM, Robert Krakora wrote:
>>
>> Hi Peter,
>>
>>  I further modified v4l2src to pass a static dummy array (instead of the
>> real memory) wrapped in the GstBuffer with a pointer to the real memory (as
>> usual) in the meta data of the GstBuffer.  When the GstBuffer arrived at
>> uvch264src I pluck the real pointer out of the meta data in the chain
>> function and the JPEG parsing is not failing.  I really, really think that
>> there is an issue with GstBuffer and/or GstMemory corrupting memory.
>>
>>  Best Regards,
>>
>>  Rob
>>
>>
>>
>> On Fri, Aug 9, 2013 at 9:15 AM, Robert Krakora <
>> rob.krakora at messagenetsystems.com> wrote:
>>
>>> Hi Peter,
>>>
>>>  I placed the JPEG parse code in the UVC driver, v4l2src and uvch264src
>>> plugins.  The JPEG parse code did not fail in the UVC driver but it did in
>>> the v4l2src and uvch264src plugins.  In the v4l2src plugin I noticed that a
>>> pointer to the memory mapped form kernel space was not only wrapped in a
>>> GstBuffer but also placed in it's meta data.  I then used the pointer that
>>> was placed in the meta data as the pointer acted on by the JPEG parse code
>>> and then the JPEG parse code in the v4l2src plugin no longer failed (i.e. I
>>> removed the gst_buffer_map() and gst_buffer_unmap() calls from around the
>>> JPEG parse code in v4l2src and just used the pointer from the meta data of
>>> the GstBuffer).  I am thinking that there is a bug or bugs in the GstMemory
>>> class where memory is being merged, moved and replaced that results in
>>> memory corruption.  I compared UVC driver logs from the JPEG parse code and
>>> uvch264src plugin logs from the JPEG parse code and everything jives
>>> segment size wise in regard to the first segment encountered in the buffer
>>> until the failure where the sizes of the first segments in both logs do not
>>> match each other.
>>>
>>>  Best Regards,
>>>
>>>  Rob
>>>
>>>
>>>
>>> On Fri, Aug 9, 2013 at 7:45 AM, Peter Rennert <p.rennert at cs.ucl.ac.uk>wrote:
>>>
>>>> Hi Rob,
>>>>
>>>> Is there a way for you to print both buffers (uvch264src and UVC
>>>> Driver) out and compare them?
>>>>
>>>> Maybe the v4l2src inserts some kind of metadata in the buffer at some
>>>> time stamps. Or maybe just modifies somehow the two bytes after the first
>>>> 0xff 0xe4.
>>>>
>>>> If that is the case then this line (gst_uvc_h264_mjpg_demux_chain: 498):
>>>>
>>>>       segment_size = GUINT16_FROM_BE (*((guint16 *) (info.data + i +
>>>> 2)));
>>>>
>>>>
>>>> will set the segment_size wrong.
>>>>
>>>> However, I compared your first segment size with the one I've got
>>>> earlier (email from the 25/07). They are different. So whatever those bytes
>>>> are they are not static. It just pops into my mind that perhaps the v4l2
>>>> swallows first bytes of the stream. In this case the 0xff 0xe4 that the
>>>> uvch264src detects are actually image data rather than part of the metadata.
>>>>
>>>> What puzzles me as well is that the first segment size is parsed wrong
>>>> and still it fails at the last segment. Which is weird. I would expect that
>>>> if it gets a too large first segment it will parse/copy too much of the
>>>> buffer as the first segment. The result should be that it would not be able
>>>> to find (i.e. skip) the second APP4 marker, including its segment size tag
>>>> and continue with the third one. This is because i in the loop is
>>>> incremented with the segment_size to avoid looping through the entire
>>>> buffer in order to find the APP4 markers (line 648).
>>>>
>>>> Peter
>>>>
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>
>>>
>>>
>>>
>>>   --
>>>
>>> Rob Krakora,
>>> Senior Software Engineer
>>>
>>> MessageNet Systems
>>> 101 E Carmel Dr, Suite 105
>>> Carmel, IN 46032
>>>
>>> MessageNetSystems.com<http://www.messagenetcommunicationsystems.com/?utm_source=email+signature&utm_medium=email&utm_campaign=email+signature+to+homepage>
>>> Rob.Krakora at MessageNetSystems.com <rob.krakora at messagenetsystems.com>
>>> P: 317.566.1677, 212
>>> F: 317.663.0808
>>>
>>> For the latest news, information, and blogs, please be sure to visit,
>>> follow, and like us...
>>>
>>>
>>> <http://www.messagenetcommunicationsystems.com/get-the-message-out-blog/?utm_source=email+signature&utm_medium=email&utm_campaign=gmail+signature+to+blog>
>>>    <http://www.youtube.com/user/MessageNetConnection/feed>  <http://www.linkedin.com/company/messagenet-systems>
>>>    <http://twitter.com/MessageNet>  <http://www.facebook.com/MessageNetsystems>
>>>
>>
>>
>>
>>  --
>>
>> Rob Krakora,
>> Senior Software Engineer
>>
>> MessageNet Systems
>> 101 E Carmel Dr, Suite 105
>> Carmel, IN 46032
>>
>> MessageNetSystems.com<http://www.messagenetcommunicationsystems.com/?utm_source=email+signature&utm_medium=email&utm_campaign=email+signature+to+homepage>
>> Rob.Krakora at MessageNetSystems.com <rob.krakora at messagenetsystems.com>
>> P: 317.566.1677, 212
>> F: 317.663.0808
>>
>> For the latest news, information, and blogs, please be sure to visit,
>> follow, and like us...
>>
>>
>> <http://www.messagenetcommunicationsystems.com/get-the-message-out-blog/?utm_source=email+signature&utm_medium=email&utm_campaign=gmail+signature+to+blog>
>>    <http://www.youtube.com/user/MessageNetConnection/feed>  <http://www.linkedin.com/company/messagenet-systems>
>>    <http://twitter.com/MessageNet>  <http://www.facebook.com/MessageNetsystems>
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing listgstreamer-devel at lists.freedesktop.orghttp://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>
>
> --
>
> Rob Krakora,
> Senior Software Engineer
>
> MessageNet Systems
> 101 E Carmel Dr, Suite 105
> Carmel, IN 46032
>
> MessageNetSystems.com<http://www.messagenetcommunicationsystems.com/?utm_source=email+signature&utm_medium=email&utm_campaign=email+signature+to+homepage>
> Rob.Krakora at MessageNetSystems.com <rob.krakora at messagenetsystems.com>
> P: 317.566.1677, 212
> F: 317.663.0808
>
> For the latest news, information, and blogs, please be sure to visit,
> follow, and like us...
>
>
> <http://www.messagenetcommunicationsystems.com/get-the-message-out-blog/?utm_source=email+signature&utm_medium=email&utm_campaign=gmail+signature+to+blog>
>    <http://www.youtube.com/user/MessageNetConnection/feed>  <http://www.linkedin.com/company/messagenet-systems>
>    <http://twitter.com/MessageNet>  <http://www.facebook.com/MessageNetsystems>
>

-- 

Rob Krakora,
Senior Software Engineer

MessageNet Systems
101 E Carmel Dr, Suite 105
Carmel, IN 46032

MessageNetSystems.com<http://www.messagenetcommunicationsystems.com/?utm_source=email+signature&utm_medium=email&utm_campaign=email+signature+to+homepage>
Rob.Krakora at MessageNetSystems.com <rob.krakora at messagenetsystems.com>
P: 317.566.1677, 212
F: 317.663.0808

For the latest news, information, and blogs, please be sure to visit,
follow, and like us...

<http://www.messagenetcommunicationsystems.com/get-the-message-out-blog/?utm_source=email+signature&utm_medium=email&utm_campaign=gmail+signature+to+blog>
   <http://www.youtube.com/user/MessageNetConnection/feed>
<http://www.linkedin.com/company/messagenet-systems>
   <http://twitter.com/MessageNet>  <http://www.facebook.com/MessageNetsystems>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130810/31684fec/attachment-0001.html>