[Bug 664778] [souphttpsrc] chunk allocator: memory waste

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=664778
  GStreamer | gst-plugins-good | unspecified

--- Comment #8 from René Stadler <mail at renestadler.de> 2011-11-30 09:49:23 UTC ---
(In reply to comment #7)
> I’d like to know more detail regarding 0 bytes receive in got_chunk_cb.
> 
> Could you give me briefly answer about below questions?
> 
> Q1. Why 0 bytes flow into souphttpsrc? As far as I know, libsoup read socket if
> the socket goes to readable state using some polling method, select, poll,
> epoll and so on.

Yes, but you don't know how many bytes you can read when the socket is
signalled as readable. In order to have zero-copy between libsoup and
souphttpsrc, this is what is done:

 - libsoup asks the source to allocate a buffer to write into
 - libsoup asks the os to fill the buffer with data from the socket
 - the buffer is filled partially or fully, in non-blocking fashion
 - if the buffer is filled fully (easily doable for 4KB), libsoup asks for
another buffer (since there *could* be more data to read right now)
 - libsoup asks the os again to fill the next buffer, but now it is very
possible that 0 bytes are read
 - libsoup tells us to deallocate the buffer without being used

That patch I had solved this by recycling partially filled (including empty)
buffers. I need to look around a bit in old backups, maybe it's sitting
somewhere. Otherwise I can rewrite it I guess (after the next releases though,
since a freeze is coming).

> Q2. Could you explain more detail regarding reallocation scenario including
> related codes? And what’s the exactly overhead?

malloc() followed by free(), not using the buffer in between, is useless
overhead. Producing smaller GStreamer buffers than asked/needed is overhead
because you need more calls to process the same data. 

> Q3. Could you guess any side-effect if I rollback the chunk_allocator patch?

Not sure what you mean actually.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.