Question

我在使用python的库灯泡从neo4j插入和查找数据时遇到了一些麻烦。问题与字符编码有关。我明白了：

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 22: ordinal not in range(128)

尝试查找索引中的节点时。我已经搜索了在neo4j灯泡中更改字符编码的方法，但似乎无法找到方法。

修改这是重现错误的代码：

from bulbs.model import Node
from bulbs.neo4jserver import Graph
from bulbs.property import String
import MySQLdb
import sys


class Topic(Node):
    element_type = 'node'
    name = String(nullable=False)


g = Graph()
g.add_proxy('topics', Topic)

con = MySQLdb.connect(host='127.0.0.1', user='root', db='wiki_new', charset='utf8')
cur = con.cursor()
cur.execute('SELECT page_title FROM page')
while True:
    row = cur.fetchone()
    if not row:
        break

    sys.stdout.write(row[0] + '\n')
    nds = g.topics.index.lookup(name=row[0])
    if not nds:
        g.topics.create(name=row[0])

导致错误的字符串是：！Xóõ。

更新

我现在使用python的sax解析器从XML文件（维基百科页面转储）获取数据。代码基本相同，我得到错误：

  File "graph.py", line 197, in <module>
    build_wikipedia_graph(WIKI_DUMP_PATH)
  File "graph.py", line 195, in build_wikipedia_graph
    filter_handler.parse(open(wiki_dump_path))
  File "/usr/lib/python2.7/xml/sax/saxutils.py", line 255, in parse
    self._parent.parse(source)
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.7/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 304, in end_element
    self._cont_handler.endElement(name)
  File "/home/pedro/wiki/1.0/page_parser.py", line 55, in method
    getattr(self._downstream, method_name)(*a, **k)
  File "/home/pedro/wiki/1.0/page_parser.py", line 87, in endElement
    self.pageCallBack(self.currentPage, self.callbackArgs)
  File "graph.py", line 181, in _callback
    kgraph.set_links_to(page.title, target)
  File "graph.py", line 59, in set_links_to
    topic_dst = self._g.topics.get_or_create('name', topic_dst, name=topic_dst)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/element.py", line 607, in get_or_create
    vertex = self.index.get_unique(key, value)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/neo4jserver/index.py", line 335, in get_unique
    resp = lookup(self.index_name,key,value)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/neo4jserver/client.py", line 878, in lookup_vertex
    path = build_path(index_path, vertex_path, index_name, key, value)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/utils.py", line 126, in build_path
    segments = [quote(str(segment), safe='') for segment in args if segment is not None]
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 22: ordinal not in range(128)

当我尝试创建名称为atp-toernooivanmontréal/ toronto的节点时发生错误。

另一次更新 使用更新的灯泡库，我得到一个不同的错误：

  File "/usr/local/lib/python2.7/dist-packages/bulbs/utils.py", line 129, in build_path
    segments = [quote(unicode(segment), safe='') for segment in args if segment is not None]
  File "/usr/lib/python2.7/urllib.py", line 1238, in quote
    return ''.join(map(quoter, s))
KeyError: u'\xe9'

任何帮助？

谢谢！

Answer 1

Bulb在Neo4j Server中将字符串存储为unicode - 注意属性类型String将值转换为unicode（unicode字符串是Python 3中的默认值）：

https://github.com/espeed/bulbs/blob/master/bulbs/property.py#L171

请参阅Python Unicode HOWTO：

http://docs.python.org/2/howto/unicode.html#python-2-x-s-unicode-support

首先，验证您的MySQL服务器是否支持UTF-8：

mysql> show character set like 'utf%';

另外，请注意我的更改和评论......

from bulbs.model import Node
from bulbs.neo4jserver import Graph
from bulbs.property import String
import MySQLdb
import sys


class Topic(Node):
    element_type = 'node'           # by convention name this 'topic'
    name = String(nullable=False)


g = Graph()
g.add_proxy('topics', Topic)

# Make sure use_unicode to set True
con = MySQLdb.connect(host='127.0.0.1', user='root', db='wiki_new', use_unicode=True, charset='utf8')
cur = con.cursor()
cur.execute('SELECT page_title FROM page')
while True:
    row = cur.fetchone()  
    if not row:
        break

    sys.stdout.write(row[0] + '\n')

    # Use Bulbs' get_or_create method to simplify your code
    nds = g.topics.get_or_create(name, row[0], name=row[0])

neo4j，灯泡和utf8

1 个答案: