[packagekit] Yum and locking

[Robin Norwood]

Richard Hughes <hughsient at gmail.com> writes:

> On Wed, 2007-10-17 at 13:37 -0400, Robin Norwood wrote:

>> The yum cli does locking right after configuration is setup (right about
>> where we set the throttle above), and unlocks right after commands
>> complete.  To duplicate these, we'd need to lock at about the time we
>> set config.throttle, and unlock when the transaction is done - we'd
>> probably also need some signal handling for when PK cancels an operation
>> in progress.
>
> We could send yum something over stdin before we kill it, although
> that's non-ideal.

I know pretty much nothing about python signal handling.  It might be
nice to have an "I'm about to kill you" signal from PK, but I think
ideally we should be able to catch SIGTERM and unlock before exiting.
This is probably safest.

>> I talked to Seth about this a bit today, and he agrees that we must do
>> locking.  Even if an operation like a search might be considered 'read
>> only', yum may update the cache if it is out of date for a repository -
>> two actions running at the same time could do very bad things.  When an
>> action is run on a repository, it will automatically update the cache if
>> it is out of date.
>
> Is cache invalidation the only thing that needs to be protected? If so
> this seems like using a sledgehammer to crack a nut.

Well, I'm not 100% sure, but I think 'corruption' is probably a more
accurate term than 'invalidation'.  My understanding is you'd end up
with multiple yum processes trying to write to the same cache file at
the same time.  I don't think yum detects and corrects for that other
than by dying if the cache doesn't parse.  We can probably catch
whatever exception is thrown by a corrupt cache and regenerate the cache
instead of just dying, however.

>> One option we could try is to tell yum to only look in the cache when
>> doing synchronous actions:
>> 
>> yumbase.conf.cache = 1
>> 
>> We already do this for searching.  However, the cache could be
>> invalidated in the middle of a transaction by another yum process
>> running without this flag set (like from the commandline).
>
> Is this a problem? Can't yum just detect the cache being invalidated and
> recreate it?

I'll have to check to be sure.  I think my paragraph above about
corruption is correct, though.

>> The final option I know of is to add finer grained read/write locking to
>> yum, but this would obviously be non-trivial to implement.
>
> This is what has to be done - we can't have people waiting a hour for a
> download to complete before they can search for and queue up more
> software.

Maybe some kindly yum ninja on the list can give us a better idea about
the feasability of this.

>
>> So my take is that the yum backend needs to be patched to do proper
>> locking for all actions, unless and until someone patches yum to allow
>> for finer-grained locking.  Which means the yum backend won't be able to
>> support synchronous actions at all.
>
> If it has to be done, it has to be done. This will really suck from a
> user interface point of view, and most of the time we will be waiting
> for something to complete.

So shall we go ahead and implement the nasty locking for now?

-RN

-- 
Robin Norwood
Red Hat, Inc.

"The Sage does nothing, yet nothing remains undone."
-Lao Tzu, Te Tao Ching