在Django admin中上传文件时出现UnicodeEncodeError

时间：2012-02-09 10:04:49

标签： django file-upload unicode nginx gunicorn

我的问题类似于报告的问题here，但建议的解决方案对我不起作用。我正在尝试通过Django管理应用程序上传名为“Testaråäö.txt”的文件。

我在Debian 6服务器上使用Gunicorn 0.13.4和Nginx 0.7.6.7运行Django 1.3.1。数据库是PostgreSQL 8.4.9。其他Unicode数据保存到数据库没有问题，所以我猜问题必须以某种方式与文件系统有关。

我已经设置了

http {
    charset utf-8;
}

在我的nginx.conf中。 LC_ALL和LANG设置为'sv_SE.UTF-8'。运行'locale'验证这一点。我甚至尝试在我的nginx init脚本中设置LC_ALL和LANG，以确保正确设置了语言环境。

这是追溯：

Traceback (most recent call last):

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 307, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view
response = view_func(request, *args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/views/decorators/cache.py", line 79, in _wrapped_view_func
response = view_func(request, *args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/sites.py", line 197, in inner
return view(request, *args, **kwargs)

File "/srv/django/letebo/app/cms/admin.py", line 81, in change_view
return super(PageAdmin, self).change_view(request, obj_id)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 28, in _wrapper
return bound_func(*args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view
response = view_func(request, *args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 24, in bound_func
return func(self, *args2, **kwargs2)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/transaction.py", line 217, in inner
res = func(*args, **kwargs)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 985, in change_view
self.save_formset(request, form, formset, change=True)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 677, in save_formset
formset.save()

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 482, in save
return self.save_existing_objects(commit) + self.save_new_objects(commit)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 613, in save_new_objects
self.new_objects.append(self.save_new(form, commit=commit))

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 717, in save_new
obj.save()

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 504, in save_base
self.save_base(cls=parent, origin=org, using=using)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 543, in save_base
for f in meta.local_fields if not isinstance(f, AutoField)]

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 255, in pre_save
file.save(file.name, file, save=False)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 92, in save
self.name = self.storage.save(name, content)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 48, in save
name = self.get_available_name(name)

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 74, in get_available_name
while self.exists(name):

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 218, in exists
return os.path.exists(self.path(name))

File "/srv/.virtualenvs/letebo/lib/python2.6/genericpath.py", line 18, in exists
st = os.stat(path)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 52-54: ordinal not in range(128)

更新：我尝试在启用调试的情况下运行Gunicorn，文件上传没有任何问题。我想这肯定意味着问题出在Nginx上。尽管如此，仍然可以打败我。以下是Gunicorn和Nginx的原始响应标题，如果它有任何意义：

Gunicorn：

HTTP/1.1 302 FOUND
Server: gunicorn/0.13.4
Date: Thu, 09 Feb 2012 14:50:27 GMT
Connection: close
Transfer-Encoding: chunked
Expires: Thu, 09 Feb 2012 14:50:27 GMT
Vary: Cookie
Last-Modified: Thu, 09 Feb 2012 14:50:27 GMT
Location: http://my-server.se:8000/admin/cms/page/15/
Cache-Control: max-age=0
Content-Type: text/html; charset=utf-8
Set-Cookie: messages="yada yada yada"; Path=/

Nginx的：

HTTP/1.1 500 INTERNAL SERVER ERROR
Server: nginx/0.7.67
Date: Thu, 09 Feb 2012 14:50:57 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Cookie

500

2 个答案:

答案 0 :(得分：9)

运行django服务的服务似乎是一个常见的问题，无论是apache，gunicorn + supervisor，daemontools等......并不总是使用正确的环境变量。

在这种情况下，gunicorn由主管启动，因此您必须明确告诉主管使用环境选项使用utf-8：

environment=LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8

答案 1 :(得分：3)

我在genericpath.py遇到同样的问题，在尝试上传带有非ASCII字符的文件名时会出现UnicodeEncodeError。

我正在使用nginx，uwsgi和django和python 2.7。

一切都在本地工作，但不在服务器上

以下是我采取的步骤：

  1. 添加到/etc/nginx/nginx.conf（没解决问题）

    http {
     charset utf-8;
}

  2. 我将此行添加到etc / default / locale（没有解决问题）

    LANGUAGE="en_US.UTF-8"

  3. 我按照标题＆＃39;成功＆＃39;中列出的说明进行操作。 https://code.djangoproject.com/wiki/ExpectedTestFailures（没有解决问题）

    aptitude install language-pack-en-base

  4. 找到这张票 https://code.djangoproject.com/ticket/17816 建议在服务器上测试视图信息

    5. 发生的事情

    Django视图

    import locale

def locales(request):
    """Display the locales"""
    locales = "Current locale: %s %s -- Default locale: %s %s" % (
        locale.getlocale() + locale.getdefaultlocale())
    default_encoding =  sys.getdefaultencoding()
    file_system_encoding = sys.getfilesystemencoding()

    context = {
        'locales': locales,
        'default_encoding': default_encoding,
        'file_system_encoding': file_system_encoding,  # affects file uploads
    }
    return render(request, 'testing/locales.html', context)

    Django模板

    <h2>Locales</h2>
<p>{{ locales }}</p>

<h2>Default Encoding</h2>
<p>{{ default_encoding }}</p>

<h2>File System Encoding</h2>
<p>{{ file_system_encoding }}</p>

    对我来说，问题是我的Ubuntu服务器上没有语言环境和默认语言环境（虽然我确实在我的本地OSX开发机器上有它们）然后使用python无法正确上传非ASCII文件名/路径的文件提出UnicodeEncodeError，但仅在生产服务器上。

    <强>解决方案

    我将此添加到我的网站和我的网站管理员uwsgi配置文件中 例如/ etc / uwsgi-emperor / vassals / my-site-config-ini file 

    env = LANG=en_US.utf8

    <强>更新

    转移到docker后，我又开始遇到同样的错误。 在进一步研究后，我意识到在上面的uwsgi django实例import sys; sys.getfilesystemencoding()或{{ file_system_encoding }}返回ANSI_X3.4-1968，但是如果我启动了自己的python实例并运行import sys; sys.getfilesystemencoding()我会得到UTF-8。 ANSI_X3.4-1968格式是抛出UnicodeEncodeError。

    所以除了上面这个答案中列出的uwsgi解决方案之外，我还必须将它添加到我的Django dockerfile 

    ENV LANG en_US.UTF-8
RUN apt-get update && install -y locales && \
    sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
    locale-gen --purge &&\
    update-locale LANG=$LANG

    或者如果你没有使用docker，你可以在shell中运行它

    $ export LANG=en_US.UTF-8

    然后运行上面的RUN命令（但没有运行）。

    参考文献：

    http://stackoverflow.com/a/37246853/3003438
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/uwsgi/
相关问题
最新问题