Problems with release-wranglers mailman interface...

[Carl Worth]

On Tue, 30 Nov 2004 21:29:49 -0800, Keith Packard wrote:
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\xa7' in
> > position 104: ordinal not in range(128)
> 
> Argh.  I've seen this somewhere, but perhaps it was in another bit of
> python code.  It appears that python is using ASCII as the default encoding
> and then whining when you write a non-ASCII value.

This is quite easy to replicate:

	$ python
	Python 2.3.4 (#2, Sep 24 2004, 08:39:09) 
	[GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2
	Type "help", "copyright", "credits" or "license" for more information.
	>>> s = u'\xa7'
	>>> s
	u'\xa7'
	>>> print s
	Traceback (most recent call last):
	  File "<stdin>", line 1, in ?
	UnicodeEncodeError: 'ascii' codec can't encode character u'\xa7'
	in position 0: ordinal not in range(128)

So the problem here is actually with the stdout stream which defaults to
trying to do an ascii encode. Note that we get the same error with:

	>>> s.encode('ascii')
	Traceback (most recent call last):
	  File "<stdin>", line 1, in ?
	UnicodeEncodeError: 'ascii' codec can't encode character u'\xa7'
	in position 0: ordinal not in range(128)

However, utf-8 encoding would work just fine of course:

	>>> s.encode('utf-8')
	'\xc2\xa7'

So, if the problem is with stdout, the trick seems to be to train it to
use utf-8 instead. A little google work[*] reveals the following recipe:

	>>> import codecs, sys
	>>> sys.stdout = codecs.getwriter('utf-8')(sys.stdout)

After which we get the pleasing result:

	>>> print s
	§

-Carl

[*] http://www.gusmueller.com/blog/archives/2004/11/8.html
    Note the date on that blog entry. Looks like we're not the only ones
    still struggling with this aspect of python.