unicode problem, if cssclass is given as unicode...

[guest]

I get an UnicodeDecodeError? if cssclass if unicode. The problem is in /pygments/formatters/html.py in the linenos wrapper. It used StringIO. If cssclass is unicode, the first line is unicode but all other lines are strings.

example:

# -*- coding: utf-8 -*-

from pygments import lexers
from pygments.formatters import HtmlFormatter
from pygments import highlight

CSSCLASS = u"pygments" # <- With string instead of unicode it works!

sourcecode = u'/* test äöüß */'

lexer = lexers.get_lexer_by_name("css")

formatter = HtmlFormatter(
    linenos=True,
    encoding="utf-8",
    cssclass = CSSCLASS,
)

html = highlight(sourcecode, lexer, formatter)

output:

...
  File "/usr/lib/python2.5/site-packages/Pygments-0.10-py2.5.egg/pygments/formatters/html.py", line 477, in _wrap_tablelinenos
    yield 0, dummyoutfile.getvalue()
  File "/usr/lib/python2.5/StringIO.py", line 270, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 24: ordinal not in range(128)

Mfg.

Jens

[gbrandl]

Fixed in [3e42a617512b].