使用TestBase.RunBazel执行构建/运行/测试py_test的长时间运行

时间:2019-04-29 09:56:07

标签: bazel

我想使用TDD方法来开发repository_rule,该方法可以处理python sdks的下载
当我描述创建自定义测试工作区的简单测试用例时,我注意到py_test内部的RunBazel指令执行缓慢-每次执行大约需要2分钟,实际上,这不允许我使用它
有一个描述问题的提交-e7d33f7

./bazel-out/k8-fastbuild/bin/src/bazel test //src/test/py/bazel:nested_py_test --test_verbose_timeout_warnings --spawn_strategy=standalone --show_timestamps

(我尝试使用不同的标志)导致输出为

(17:04:24) INFO: Elapsed time: 119.657s, Critical Path: 119.52s
(17:04:24) INFO: 2 processes: 2 local.
(17:04:24) INFO: Build completed, 1 test FAILED, 2 total actions
//src/test/py/bazel:nested_py_test                                       FAILED in 119.4s

我尝试了bazel的主版本(74bf7888b7),并发布了一个(0.24.1)

py_binary进行

测试需要花费 6s 来执行,花费py_test-大约需要 114s

py_test调用的标准输出是

Starting local Bazel server and connecting to it...
(00:17:10) INFO: Current date is 2019-04-29
(00:17:10) Loading:
(00:17:10) Loading: 0 packages loaded
(00:17:11) Analyzing: target @nested_py_workspace//:test (1 packages loaded, 0 targets configured)
(00:17:12) Analyzing: target @nested_py_workspace//:test (16 packages loaded, 78 targets configured)
(00:17:13) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:14) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:15) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:16) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:17) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:18) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:19) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:21) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:22) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:24) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:27) Analyzing: target @nested_py_workspace//:test (17 packages loaded, 148 targets configured)
(00:17:29) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:32) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:36) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:39) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:44) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:49) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:17:55) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:18:02) Analyzing: target @nested_py_workspace//:test (18 packages loaded, 149 targets configured)
(00:18:10) Analyzing: target @nested_py_workspace//:test (19 packages loaded, 185 targets configured)
(00:18:19) Analyzing: target @nested_py_workspace//:test (19 packages loaded, 185 targets configured)
(00:18:29) Analyzing: target @nested_py_workspace//:test (19 packages loaded, 185 targets configured)
(00:18:42) Analyzing: target @nested_py_workspace//:test (19 packages loaded, 185 targets configured)
(00:18:55) Analyzing: target @nested_py_workspace//:test (19 packages loaded, 185 targets configured)
(00:19:02) INFO: Analyzed target @nested_py_workspace//:test (20 packages loaded, 302 targets configured).
(00:19:02) INFO: Found 1 target...
(00:19:02) [0 / 3] [Prepa] Creating source manifest for @nested_py_workspace//:test

by_binary

Starting local Bazel server and connecting to it...
(00:22:07) INFO: Current date is 2019-04-29
(00:22:07) Loading:
(00:22:07) Loading: 0 packages loaded
(00:22:08) Analyzing: target @nested_py_workspace//:test (1 packages loaded, 0 targets configured)
(00:22:09) Analyzing: target @nested_py_workspace//:test (12 packages loaded, 69 targets configured)
(00:22:09) INFO: Analyzed target @nested_py_workspace//:test (15 packages loaded, 129 targets configured).
(00:22:09) INFO: Found 1 target...

py_test分析的个人资料

=== PHASE SUMMARY INFORMATION ===

Total launch phase time         1.082 s    0.85%
Total init phase time            306 ms    0.24%
Total loading phase time         476 ms    0.38%
Total analysis phase time     124.640 s   98.44%
Total preparation phase time    14.6 ms    0.01%
Total execution phase time      70.6 ms    0.06%
Total finish phase time         28.2 ms    0.02%
Total run time                126.618 s  100.00%
...
=== ANALYSIS PHASE INFORMATION ===

Total analysis phase time              124.640 s

Total time (across all threads) spent on:
              Type    Total    Count     Average
              INFO    0.05%       69     3.77 ms
    CREATE_PACKAGE    0.02%       26     3.95 ms
          VFS_STAT    0.00%      607     0.01 ms
           VFS_DIR    0.00%       14     0.07 ms
      VFS_READLINK    0.00%      209     0.03 ms
           VFS_MD5    0.21%       43     25.6 ms
        VFS_DELETE    0.01%       14     1.92 ms
          VFS_OPEN    0.00%      574     0.02 ms
          VFS_READ    0.01%    30058     0.00 ms
         VFS_WRITE    0.25%   557440     0.00 ms
          VFS_GLOB    0.00%       41     0.43 ms
     SKYFRAME_EVAL   23.54%        4    31.093 s
       SKYFUNCTION   38.36%     2034     99.7 ms
   STARLARK_PARSER    0.01%       60     1.28 ms
  STARLARK_USER_FN   37.48%     1075      184 ms
STARLARK_BUILTIN_FN    0.06%     1720     0.17 ms
...

所以我想知道:

  1. 为什么分析如此缓慢,我该如何加快测试速度,也许我根本不应该使用py_test指令
  2. 有什么方法可以缓存上面完成的步骤吗?

0 个答案:

没有答案