PyPy明显慢于CPython

时间:2011-08-15 09:38:34

标签: python optimization jit pypy cprofile

我一直在测试我制作的缓存系统。其目的是加速Django Web应用程序。它将所有内容存储在内存中。根据cPro​​file,在我的测试中大部分时间都花在QuerySet._clone()中,结果证明效率非常低(实际上并没有那么奇怪)。

我非常希望使用PyPy来加快速度。我有一台64位机器。然而,在安装了所有必需的库之后,事实证明PyPy编译的代码运行速度比常规Python代码慢2.5倍,而且我不知道该怎么做。代码是CPU绑定的(绝对没有数据库查询,因此IO边界不是一个选项)。单个测试运行大约10秒,所以我想JIT应该足够了。我正在使用PyPy 1.5。一个注意事项 - 我自己没有编译源代码,只是下载了一个64位的linux版本。

我想知道CPU密集型代码在PyPy下实际运行速度有多快。希望我能做些什么会阻止PyPy运行到最佳状态。

修改

确切的cPython输出:

PyPy 1.5:

    3439146 function calls (3218654 primitive calls) in 19.094 seconds

    Ordered by: cumulative time

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       2/1    0.000    0.000   18.956   18.956 <string>:1(<module>)
       2/1    0.000    0.000   18.956   18.956 /path/to/my/project/common/integrity/models/transactions.py:200(newfn)
       2/1    0.000    0.000   18.956   18.956 /path/to/my/project/common/integrity/models/transactions.py:134(recur)
       2/1    0.000    0.000   18.956   18.956 /usr/local/pypy/site-packages/django/db/transaction.py:210(inner)
       2/1    0.172    0.086   18.899   18.899 /path/to/my/project/common/integrity/tests/optimization.py:369(func_cached)
      9990    0.122    0.000   18.632    0.002 /usr/local/pypy/site-packages/django/db/models/manager.py:131(get)
      9990    0.127    0.000   16.638    0.002 /path/to/my/project/common/integrity/models/cache.py:1068(get)
      9990    0.073    0.000   12.478    0.001 /usr/local/pypy/site-packages/django/db/models/query.py:547(filter)
      9990    0.263    0.000   12.405    0.001 /path/to/my/project/common/integrity/models/cache.py:1047(_filter_or_exclude)
      9990    0.226    0.000   12.096    0.001 /usr/local/pypy/site-packages/django/db/models/query.py:561(_filter_or_exclude)
      9990    0.187    0.000    8.383    0.001 /path/to/my/project/common/integrity/models/cache.py:765(_clone)
      9990    0.212    0.000    7.662    0.001 /usr/local/pypy/site-packages/django/db/models/query.py:772(_clone)
      9990    1.025    0.000    7.125    0.001 /usr/local/pypy/site-packages/django/db/models/sql/query.py:226(clone)
129942/49972  1.674    0.000    6.021    0.000 /usr/local/pypy/lib-python/2.7/copy.py:145(deepcopy)
140575/110605 0.120    0.000    4.066    0.000 {len}
      9990    0.182    0.000    3.972    0.000 /usr/local/pypy/site-packages/django/db/models/query.py:74(__len__)
     19980    0.260    0.000    3.777    0.000 /path/to/my/project/common/integrity/models/cache.py:1062(iterator)
      9990    0.255    0.000    3.154    0.000 /usr/local/pypy/site-packages/django/db/models/sql/query.py:1149(add_q)
      9990    0.210    0.000    3.073    0.000 /path/to/my/project/common/integrity/models/cache.py:973(_query)
      9990    0.371    0.000    2.316    0.000 /usr/local/pypy/site-packages/django/db/models/sql/query.py:997(add_filter)
      9990    0.364    0.000    2.168    0.000 /path/to/my/project/common/integrity/models/cache.py:892(_deduct)
29974/9994    0.448    0.000    2.078    0.000 /usr/local/pypy/lib-python/2.7/copy.py:234(_deepcopy_tuple)
     19990    0.362    0.000    2.065    0.000 /path/to/my/project/common/integrity/models/cache.py:566(__init__)
     10000    0.086    0.000    1.874    0.000 /path/to/my/project/common/integrity/models/cache.py:1090(get_query_set)
     19990    0.269    0.000    1.703    0.000 /usr/local/pypy/site-packages/django/db/models/query.py:31(__init__)
      9990    0.122    0.000    1.643    0.000 /path/to/my/project/common/integrity/models/cache.py:836(_deduct_recur)
     19980    0.274    0.000    1.636    0.000 /usr/local/pypy/site-packages/django/utils/tree.py:55(__deepcopy__)
      9990    0.607    0.000    1.458    0.000 /path/to/my/project/common/integrity/models/cache.py:789(_deduct_local)
     10020    0.633    0.000    1.437    0.000 /usr/local/pypy/site-packages/django/db/models/sql/query.py:99(__init__)
    129942    0.841    0.000    1.191    0.000 /usr/local/pypy/lib-python/2.7/copy.py:267(_keep_alive)
 9994/9992    0.201    0.000    1.019    0.000 /usr/local/pypy/lib-python/2.7/copy.py:306(_reconstruct)

Python 2.7:

   3326403 function calls (3206359 primitive calls) in 12.430 CPU seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   12.457   12.457 <string>:1(<module>)
        1    0.000    0.000   12.457   12.457 /path/to/my/project/common/integrity/models/transactions.py:200(newfn)
        1    0.000    0.000   12.457   12.457 /path/to/my/project/common/integrity/models/transactions.py:134(recur)
        1    0.000    0.000   12.457   12.457 /usr/local/lib/python2.7/dist-packages/django/db/transaction.py:210(inner)
        1    0.000    0.000   12.457   12.457 /path/to/my/project/common/integrity/models/transactions.py:165(recur2)
        1    0.089    0.089   12.450   12.450 /path/to/my/project/common/integrity/tests/optimization.py:369(func_cached)
     9990    0.198    0.000   12.269    0.001 /usr/local/lib/python2.7/dist-packages/django/db/models/manager.py:131(get)
     9990    0.087    0.000   11.281    0.001 /path/to/my/project/common/integrity/models/cache.py:1068(get)
     9990    0.040    0.000    8.161    0.001 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:547(filter)
     9990    0.110    0.000    8.121    0.001 /path/to/my/project/common/integrity/models/cache.py:1047(_filter_or_exclude)
     9990    0.127    0.000    7.983    0.001 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:561(_filter_or_exclude)
     9990    0.100    0.000    5.593    0.001 /path/to/my/project/common/integrity/models/cache.py:765(_clone)
     9990    0.122    0.000    5.125    0.001 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:772(_clone)
     9990    0.405    0.000    4.899    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:226(clone)
129942/49972 1.456    0.000    4.505    0.000 /usr/lib/python2.7/copy.py:145(deepcopy)
129899/99929 0.191    0.000    3.117    0.000 {len}
     9990    0.111    0.000    2.968    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:74(__len__)
    19980    0.070    0.000    2.843    0.000 /path/to/my/project/common/integrity/models/cache.py:1062(iterator)
     9990    0.208    0.000    2.190    0.000 /path/to/my/project/common/integrity/models/cache.py:973(_query)
     9990    0.182    0.000    2.114    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:1149(add_q)
19984/9994   0.291    0.000    1.644    0.000 /usr/lib/python2.7/copy.py:234(_deepcopy_tuple)
     9990    0.288    0.000    1.599    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:997(add_filter)
     9990    0.171    0.000    1.454    0.000 /path/to/my/project/common/integrity/models/cache.py:892(_deduct)
    19980    0.177    0.000    1.208    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/tree.py:55(__deepcopy__)
     9990    0.099    0.000    1.199    0.000 /path/to/my/project/common/integrity/models/cache.py:836(_deduct_recur)
     9990    0.349    0.000    1.040    0.000 /path/to/my/project/common/integrity/models/cache.py:789(_deduct_local)

1 个答案:

答案 0 :(得分:17)

撇开PyPy可能真的对你的情况本质上较慢的事实,有一些因素可能会使它变得不必要地变慢:

  • 分析已经比CPython减慢了PyPy的速度。
  • 某些调试/日志记录代码可以禁用优化(例如,通过强制帧)。
  • 您正在使用的服务器可能是性能的主导因素(考虑一下JIT经典CGI会有多糟糕:它永远不会热身)。它也可以简单地影响结果(不同的WSGI服务器已经显示出各种加速)。
  • 旧式课程比新式课程慢。
  • 即使一切都在记忆中,你也可以PyPy的SQLite中的慢速路径。

您还可以查看JIT Friendliness wiki页面,了解更多关于可以使PyPy变慢的提示。 nightly build也可能更快,因为相对于1.5有许多改进。

您的堆栈(服务器,操作系统,数据库)和设置的更详细描述(您是如何基准测试?有多少查询?)将使我们能够提供更好的答案。