Question

我在Python中有一个游戏状态，大约有1000个对象（行星系统+星星+行星），我需要复制它并在请求时应用一堆变换。但是，在大约1个请求/秒时，这是taking up 24.63% of my runtime。我怎样才能让它变得快速？请注意，复制较少不是一个选项，因为变换几乎触及所有内容。

编辑：通过明智地执行__deepcopy__来降低8％。不过，还不够好。（足够好是1％或更少，我打算在这上面扔更多的东西。）timeit说每deepcopy() 41.8ms。

Answer 1

实际上，深度扫描非常慢。但我们可以使用json，ujson或cPickle。我们可以使用json / cPickle来转储对象，并在以后加载它。这是我的考验：

Total time: 3.46068 s
File: test_deepcopy.py
Function: test at line 15
Line #   Hits          Time Per Hit   % Time  Line Contents
==============================================================
15                                             @profile
16                                             def test():
17       100       957585   9575.9     27.7        b = deepcopy(a)
18       100          862      8.6      0.0        c = copy(a)
19       100        42295    422.9      1.2        d = ujson.loads(ujson.dumps(a))
20       100        85040    850.4      2.5        e = json.loads(json.dumps(a))
21       100      2323465  23234.7     67.1        f = pickle.loads(pickle.dumps(a, -1))
22       100        51434    514.3      1.5        g = cPickle.loads(cPickle.dumps(a, -1))

正如我们所看到的，json / ujson / cPickle比deepcopy更快，但是pickle ......

Answer 2

如果您创建自己的类来保存这些对象，则可以创建自己的方法来处理复制和深层复制。 ~~http://www.rafekettler.com/magicmethods.html#copying~~ （Broken Link）

github存储库的新链接https://github.com/RafeKettler/magicmethods

class MyClass():
    def __copy__(self):
        copy_object = MyClass()
        return copy_object

    def __deepcopy__(self, memodict={}):
        copy_object = MyClass()
        copy_object.value = self.value
        return copy_object

if __name__ == "__main__":
    my_inst = MyClass()
    print(copy.deepcopy(my_inst))

以下是与之前的断开链接相似的描述。

<强>复制

有时，特别是在处理可变对象时，您希望能够复制对象并进行更改，而不会影响您从中复制的内容。这就是Python副本发挥作用的地方。然而（幸运的是），Python模块并不具有感知能力，因此我们不必担心基于Linux的机器人起义，但我们必须告诉Python如何有效地复制内容。

__copy__(self)

为类的实例定义copy.copy（）的行为。 copy.copy（）返回对象的浅表副本 - 这意味着，虽然实例本身是一个新实例，但它的所有数据都被引用 - 即，对象本身被复制，但其数据仍被引用（因此，浅拷贝中的数据更改可能会导致原始数据发生变化。

__deepcopy__(self, memodict={})

为类的实例定义copy.deepcopy（）的行为。 copy.deepcopy（）返回对象的深层副本 - 对象及其数据都被复制。 memodict是以前复制的对象的缓存 - 这可以优化复制并在复制递归数据结构时防止无限递归。如果要深度复制单个属性，请使用memodict作为第一个参数调用该属性上的copy.deepcopy（）。这些魔术方法有哪些用例？与往常一样，在任何情况下，您需要比默认行为提供的更细粒度的控制。例如，如果您尝试复制将缓存存储为字典（可能很大）的对象，那么复制缓存也没有意义 - 如果缓存可以在实例之间的内存中共享，那么应该是。

Answer 3

我做了一个快速实验，比较了deepcopy / json / ujson的几个案例，我的结果与@ cherish的某些案例相矛盾，在这里发布了一个小实验：

import ujson
import timeit
import json
import random
import string
import copy
import ujson
import sys


def random_string(N):
    return ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(N))


def random_json(width=5, height=5, levels=1):
    dct = {}
    lst = [random_string(4) for i in range(width)]
    lst2 = [random.randint(0, 10000) for i in range(width)]
    lst3 = [bool(random.randint(0, 1)) for i in range(width)]
    for j in range(height):
        dct[str(j)] = lst
        dct[str(width+j)] = lst2
        dct[str(2*width+j)] = lst3

    for i in range(levels):
        new_dct = {}
        for j in range(height):
            new_dct[str(j)] = dct
        dct = json.loads(json.dumps(new_dct))

    return new_dct

if __name__ == "__main__":
    print(sys.version)
    levels = 3
    for i in range(15):
        dataset = random_json(i, i, levels)
        print("Comparing deepcopy/ujson/json using random dataset({},{},{}), length {}".format(i,i,levels, len(json.dumps(dataset))))
        print(timeit.timeit('copy.deepcopy(dataset)',
                            setup='from __main__ import copy, dataset', number=10))
        print(timeit.timeit('ujson.loads(ujson.dumps(dataset))',
                            setup='from __main__ import ujson, dataset', number=10))
        print(timeit.timeit('json.loads(json.dumps(dataset))',
                            setup='from __main__ import json, dataset', number=10))
        print()

结果将是：

3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)]
Comparing deepcopy/ujson/json using random dataset(0,0,3), length 2
2.6842977659931844e-05
0.00012039864979822371
7.776568527950847e-05

Comparing deepcopy/ujson/json using random dataset(1,1,3), length 63
0.0002731667726569534
3.552747043226263e-05
0.00012987264191349377

Comparing deepcopy/ujson/json using random dataset(2,2,3), length 1106
0.0011858280130946362
0.00034974820892205325
0.0007093651596308467

Comparing deepcopy/ujson/json using random dataset(3,3,3), length 6834
0.0042218477363672215
0.0021178319874343293
0.003378267688436718

Comparing deepcopy/ujson/json using random dataset(4,4,3), length 26572
0.011379054029782284
0.006288757016181971
0.009920059244030693

Comparing deepcopy/ujson/json using random dataset(5,5,3), length 79210
0.028879491215043435
0.027906433274870912
0.029595961868760734

Comparing deepcopy/ujson/json using random dataset(6,6,3), length 183678
0.047142979515255284
0.04682125853300759
0.06791747047568517

Comparing deepcopy/ujson/json using random dataset(7,7,3), length 395528
0.08239215142913198
0.09871347134571351
0.15347433002098887

Comparing deepcopy/ujson/json using random dataset(8,8,3), length 764920
0.1351954464835896
0.19448842613700734
0.3020533693660834

Comparing deepcopy/ujson/json using random dataset(9,9,3), length 1356570
0.24560258734724671
0.44074906118659407
0.5705849913806413

Comparing deepcopy/ujson/json using random dataset(10,10,3), length 2287770
0.3237815755327835
0.61104051671153
0.8698565598118777

Comparing deepcopy/ujson/json using random dataset(11,11,3), length 3598750
0.4958284828467452
0.9472223636741877
1.2514314609961668

Comparing deepcopy/ujson/json using random dataset(12,12,3), length 5636414
0.6261448233909714
1.4066722957969802
1.8636325417418167

Comparing deepcopy/ujson/json using random dataset(13,13,3), length 8220800
0.8396582099444547
2.061675688670409
2.755659427352441

Comparing deepcopy/ujson/json using random dataset(14,14,3), length 12018290
1.0951926990258762
2.96703050743886
4.088875914783021

这个小实验的结论是：

字典是小字典时time(ujson)<time(json)<time(deepcopy)
字典是大字典时time(deepcopy)<time(ujson)<time(json)

因此，这取决于您每秒制作的副本数量以及您正在处理的字典类型，您更愿意在deepcopy或ujson之间切换。

Answer 4

您可以为对象提供自己的复制功能，这样您就不需要深层复制。深层复制检查每个对象以检查需要复制的内容。这是一项昂贵的操作。

Answer 5

基于@BPL的测试程序，并在与我的ARMv6兼容的处理器上添加元组

print(timeit.timeit('marshal.loads(marshal.dumps(dataset))',
       setup='from __main__ import marshal, dataset', number=1))

元帅比ujson快，并且支持set和tuple

2.7.14 (default, Mar  6 2019, 13:27:55)
[GCC 7.3.0]
Comparing deepcopy/marshal/ujson/json using random dataset(0,0,1), length 2
0.000588178634644
0.000134944915771
0.000258922576904
0.00113606452942
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,0,3), length 2
0.000546932220459
0.000134944915771
0.000180006027222
0.00120401382446
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,0,5), length 2
0.000545978546143
0.000128984451294
0.000185966491699
0.00106000900269
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,2,1), length 50
0.00154900550842
0.000281810760498
0.000414848327637
0.00174903869629
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,2,3), length 242
0.00655102729797
0.000789880752563
0.00133085250854
0.00432300567627
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,2,5), length 1010
0.0514280796051
0.0015549659729
0.00413513183594
0.0148711204529
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,4,1), length 172
0.00250005722046
0.000365018844604
0.000761985778809
0.00263404846191
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,4,3), length 2892
0.0329101085663
0.00363397598267
0.0110101699829
0.0262169837952
()
Comparing deepcopy/marshal/ujson/json using random dataset(0,4,5), length 46412
0.616458892822
0.0826110839844
0.189103841782
0.504135131836
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,0,1), length 2
0.000693082809448
0.000132083892822
0.000182867050171
0.00107002258301
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,0,3), length 2
0.000566005706787
0.000132083892822
0.000180959701538
0.00107598304749
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,0,5), length 2
0.000562906265259
0.000128984451294
0.000184059143066
0.00118517875671
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,2,1), length 258
0.00405406951904
0.000534057617188
0.00124287605286
0.00309610366821
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,2,3), length 1058
0.026270866394
0.00180387496948
0.00363302230835
0.0096640586853
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,2,5), length 4338
0.0778729915619
0.00682806968689
0.0151469707489
0.0468928813934
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,4,1), length 716
0.00720596313477
0.00100684165955
0.0215280056
0.0062358379364
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,4,3), length 11468
0.112984895706
0.0238728523254
0.0448131561279
0.0874760150909
()
Comparing deepcopy/marshal/ujson/json using random dataset(2,4,5), length 183628
1.83552503586
0.407335042953
0.617804050446
1.65498495102
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,0,1), length 2
0.000571012496948
0.000132083892822
0.000189781188965
0.00121593475342
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,0,3), length 2
0.000757932662964
0.000131130218506
0.000180959701538
0.00144195556641
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,0,5), length 2
0.00056791305542
0.000132083892822
0.000184059143066
0.00107407569885
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,2,1), length 430
0.00451302528381
0.00053596496582
0.00142502784729
0.00343203544617
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,2,3), length 1730
0.0259549617767
0.00232696533203
0.00387692451477
0.0187470912933
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,2,5), length 7026
0.112207174301
0.0119769573212
0.0211799144745
0.0547370910645
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,4,1), length 1684
0.00609397888184
0.00121903419495
0.00452899932861
0.00959086418152
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,4,3), length 26828
0.19367814064
0.0293428897858
0.0688338279724
0.140627145767
()
Comparing deepcopy/marshal/ujson/json using random dataset(4,4,5), length 433484
3.54843020439
0.590909004211
1.09412097931
2.72070598602

deepcopy（）非常慢

5 个答案: