Question

我有两个表，当post_id

时，我要将testpostmeta.meta_value = testTable.stockcode从一个表复制到另一个

testTable中大约有 2000行，而testpostmeta中有 6.5万行。

代码有效，只需大约1-2分钟即可完成。有什么办法可以加快仓鼠的速度吗？

UPDATE testTable 
  INNER JOIN testpostmeta 
    ON testTable.stockcode = testpostmeta.meta_value
   SET testTable.post_id = testpostmeta.post_id

我尝试添加WHERE testpostmeta.meta_value = testTable.stockcode，但这没用。

Answer 1

确保在testTable和testpostmeta上具有正确的索引

CREATE INDEX my_idx1  ON testTable (stokcode);

CREATE INDEX my_idx2  ON testpostmeta (meta_value , post_id);

Answer 2

尝试向与您的JOIN条件所使用的字段匹配的每个表中添加索引：

ALTER TABLE testTable ADD INDEX stockcode_idx(stockcode);
ALTER TABLE testpostmeta ADD INDEX meta_idx(meta_value);

Answer 3

您可以停止自动提交

python model_main.py --model_dir=training/ --pipeline_config_path=training/faster_rcnn_inception_v2
_pets.config
WARNING:tensorflow:Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x00000155CFDB5C80>) includes params argument, but params are not passed to Estimator.
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
WARNING:root:Variable [SecondStageBoxPredictor/BoxEncodingPredictor/biases] is available in checkpoint, but has an incompatible shape with model variable.
WARNING:root:Variable [SecondStageBoxPredictor/BoxEncodingPredictor/weights] is available in checkpoint, but has an incompatible shape with model variable.
WARNING:root:Variable [SecondStageBoxPredictor/ClassPredictor/biases] is available in checkpoint, but has an incompatible shape with model variable.
WARNING:root:Variable [SecondStageBoxPredictor/ClassPredictor/weights] is available in checkpoint, but has an incompatible shape with model variable.
WARNING:root:Variable [global_step] is not available in checkpoint
D:\ProgramFiles\Anaconda\envs\Eneger\lib\site-packages\tensorflow\python\ops\gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may
consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2018-07-24 05:19:44.209751: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorF
low binary was not compiled to use: AVX AVX2
2018-07-24 05:19:45.534609: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties:

name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
totalMemory: 6.00GiB freeMemory: 4.97GiB
2018-07-24 05:19:45.571588: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0
2018-07-24 05:19:53.604693: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with stren
gth 1 edge matrix:
2018-07-24 05:19:53.615439: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929]      0
2018-07-24 05:19:53.622183: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0:   N
2018-07-24 05:19:53.643331: I C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/re
plica:0/task:0/device:GPU:0 with 4730 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-07-24 05:20:58.481169: E C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\stream_executor\cuda\cuda_event.cc:49] Error polling for event status: failed to query e
vent: **CUDA_ERROR_ILLEGAL_INSTRUCTION**
2018-07-24 05:20:58.511921: F C:\users\nwani\_bazel_nwani\mmtm6wb6\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_event_mgr.cc:208] Unexpected Event status: 1

Answer 4

如果post_id已在目标表中建立索引，这也会减慢更新速度。

尝试在操作之前禁用索引，然后在操作之后启用它。因此，您的数据将被索引一次，而不是在每次后续数据更改时被索引。

ALTER TABLE targetTable DISABLE KEYS;
-- Your UPDATE query
ALTER TABLE targetTable ENABLE KEYS;

如参考文献所述：

一起执行多个更新要比一次锁定表快得多。

这里有一些参考页，可以给您更多有关可以做什么的想法：

复制数据的更好方法？

4 个答案: