为什么导入时会创建.pyc文件?

时间:2015-08-07 16:54:52

标签: python bytecode python-internals pyc

我已经看到了几个描述.pyc文件是什么以及何时创建文件的资源。但现在我想知道为什么在导入.py文件时创建它们?

另外,为什么不为执行导入的主Python文件创建一个.pyc文件?

我猜测它与性能优化和学习有关,这鼓励我打破我的文件,因为内置的编译似乎很好用。但是我不确定是不是这种情况,而且我也很好奇是否有人有关于运行带有和不带.pyc文件的程序之间差异的统计数据,如果确实是为了速度。< / p>

我自己运行它们,但我没有一个好的,大型的Python代码库来测试它。 :(

1 个答案:

答案 0 :(得分:23)

Python源代码被编译为字节码,它是运行的字节码。 .pyc文件包含该字节码的副本,并且通过缓存Python不必在每次需要加载模块时重新编译Python代码。

通过计算compile()函数:

,您可以了解节省了多少时间
>>> import urllib2
>>> import timeit
>>> urllib2_source = open(urllib2.__file__.rstrip('c')).read()
>>> timeit.timeit("compile(source, '', 'exec')", 'from __main__ import urllib2_source as source', number=1000)
6.977046966552734
>>> _ / 1000.0
0.006977046966552734

因此编译urllib2.py源代码需要7毫秒。这听起来并不多,但随着Python在其生命周期中加载 lot 模块,这会迅速增加。只需使用-v command-line switch运行一个平均脚本;在这里,我运行pydoc tool的帮助输出:

$ bin/python -v -m pydoc -h
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.py
import site # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.py
import os # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.py
import posixpath # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.py
import stat # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.py
import genericpath # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.py
import warnings # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.py
import linecache # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.py
import types # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.py
import UserDict # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.py
import _abcoll # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.py
import abc # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.py
import _weakrefset # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.pyc
import _weakref # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.py
import copy_reg # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.pyc
import encodings # directory /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.py
import encodings # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.py
import codecs # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.pyc
import _codecs # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.py
import encodings.utf_8 # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.pyc
Python 2.7.8 (default, Sep  9 2014, 11:33:29) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.py
import runpy # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.pyc
import imp # builtin
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.py
import pkgutil # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.py
import re # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.py
import sre_compile # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.pyc
import _sre # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.py
import sre_parse # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.py
import sre_constants # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_locale.so", 2);
import _locale # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_locale.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.py
import inspect # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.py
import string # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/strop.so", 2);
import strop # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/strop.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.py
import dis # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.py
import opcode # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.py
import tokenize # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/itertools.so", 2);
import itertools # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/itertools.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.py
import token # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/operator.so", 2);
import operator # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/operator.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.py
import collections # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_collections.so", 2);
import _collections # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_collections.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.py
import keyword # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.py
import heapq # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_heapq.so", 2);
import _heapq # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_heapq.so
import thread # builtin
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.py
import repr # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.py
import traceback # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.py
import locale # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.py
import functools # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_functools.so", 2);
import _functools # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_functools.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.py
import getopt # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.pyc
pydoc - the Python documentation tool

pydoc.py <name> ...
    Show text documentation on something.  <name> may be the name of a
    Python keyword, topic, function, module, or package, or a dotted
    reference to a class or function within a module or module in a
    package.  If <name> contains a '/', it is used as the path to a
    Python source file to document. If name is 'keywords', 'topics',
    or 'modules', a listing of these things is displayed.

pydoc.py -k <keyword>
    Search for a keyword in the synopsis lines of all available modules.

pydoc.py -p <port>
    Start an HTTP server on the given port on the local machine.

pydoc.py -g
    Pop up a graphical interface for finding and serving documentation.

pydoc.py -w <name> ...
    Write out the HTML documentation for a module to a file in the current
    directory.  If <name> contains a '/', it is treated as a filename; if
    it names a directory, documentation is written for all the contents.

# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] _collections
# cleanup[1] locale
# cleanup[1] functools
# cleanup[1] encodings
# cleanup[1] site
# cleanup[1] runpy
# cleanup[1] operator
# cleanup[1] supervisor
# cleanup[1] _heapq
# cleanup[1] abc
# cleanup[1] _weakrefset
# cleanup[1] sre_constants
# cleanup[1] collections
# cleanup[1] _codecs
# cleanup[1] opcode
# cleanup[1] _warnings
# cleanup[1] mpl_toolkits
# cleanup[1] inspect
# cleanup[1] encodings.utf_8
# cleanup[1] repr
# cleanup[1] codecs
# cleanup[1] getopt
# cleanup[1] pkgutil
# cleanup[1] _functools
# cleanup[1] thread
# cleanup[1] keyword
# cleanup[1] strop
# cleanup[1] signal
# cleanup[1] traceback
# cleanup[1] itertools
# cleanup[1] posix
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _weakref
# cleanup[1] token
# cleanup[1] dis
# cleanup[1] tokenize
# cleanup[1] heapq
# cleanup[1] string
# cleanup[1] imp
# cleanup[1] zipimport
# cleanup[1] re
# cleanup[1] _locale
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] sre_parse
# cleanup[2] copy_reg
# cleanup[2] posixpath
# cleanup[2] errno
# cleanup[2] _abcoll
# cleanup[2] types
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] os.path
# cleanup[2] linecache
# cleanup[2] os
# cleanup sys
# cleanup __builtin__
# cleanup ints: 21 unfreed ints
# cleanup floats

这是53次进口:

$ bin/python -v -m pydoc -h 2>&1 | egrep ^import | wc -l
      53

而不是每次加载(更大)源文件并编译它,可以立即读取和使用较小的字节码文件。只需打印一些命令行工具的帮助信息,就可以轻松地加上1/3或1/2秒。

Python不会为主脚本创建缓存文件;那是因为这会使你的脚本目录混乱,文件不会像模块加载那样频繁加载。

如果您经常运行 脚本,那么该文件的编译时间会影响您,您可以随时将大部分代码移动到模块中(并避免编译大型脚本) )或者您可以使用compileall tool为脚本创建.pyc缓存文件,然后直接运行.pyc文件 。请注意,如果您更改了脚本,Python将不会重新编译该文件!