Question

我用Jmeter编写了一些API测试，并通过我们的CI工具TeamCity运行了它们。问题是，尽管我在Jmeter报告中看到测试中的某些断言失败，但TeamCity始终将构建显示为“成功”。我想要的是，在这种情况下，TeamCity相应地将构建显示为“失败”。

TeamCity配置使用命令行作为构建步骤，该步骤触发以下命令：

jmeter -n -t jMeterTeamCityExample.jmx -l log\api_tests_log.jmx -e -o api-tests-results

这是我为测试目的制作的示例.jmx文件：

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="3.2" jmeter="3.3 r1808647">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="jMeter + TeamCity example" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">stopthread</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1530777613000</longProp>
        <longProp name="ThreadGroup.end_time">1530777613000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="GET post" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">jsonplaceholder.typicode.com</stringProp>
          <stringProp name="HTTPSampler.port"></stringProp>
          <stringProp name="HTTPSampler.protocol"></stringProp>
          <stringProp name="HTTPSampler.contentEncoding"></stringProp>
          <stringProp name="HTTPSampler.path">posts/1</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
          <stringProp name="HTTPSampler.connect_timeout"></stringProp>
          <stringProp name="HTTPSampler.response_timeout"></stringProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Response Assertion" enabled="true">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="51508">400</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_code</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">8</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
      </hashTree>
    </hashTree>
    <WorkBench guiclass="WorkBenchGui" testclass="WorkBench" testname="WorkBench" enabled="true">
      <boolProp name="WorkBench.save">true</boolProp>
    </WorkBench>
    <hashTree/>
  </hashTree>
</jmeterTestPlan>

在文件中，我故意在响应声明中添加了无效的响应代码（400），以导致测试失败。

结论和问题：当我使用上面提供的命令通过TeamCity运行此测试时，TeamCity显示成功，尽管测试本身在响应断言时失败。我希望如果Jmeter测试计划的任何响应断言失败，TeamCity将显示失败。有任何想法吗？

注意：我了解一些TeamCity的Jmeter插件，但是我想仅通过使用CommandLine构建步骤来找到解决方案。

Answer 1

我发现了一个不太优雅的解决方案。

添加构建步骤以将jtl文件转储到团队城市构建日志：

我在Windows上使用以下命令行创建了命令行构建步骤：powershell -nologo "& "Get-Content <location of your .jtl file> -Tail 100"

在Linux上，您可以这样做：cat <location of your .jtl file>

我添加了构建失败条件以在构建日志中查找此文本 “文字错误”

这是构建日志的一部分，其中显示了构建日志中我失败的电表声明：

[18:00:31]Step 5/5: check file for failed assertions (Command Line)
[18:00:31][Step 5/5] Starting: C:\BuildAgent\temp\agentTmp\custom_script6595741193421826142.cmd
[18:00:31][Step 5/5] in directory: C:\apache-jmeter-5.1.1\JmeterREsults\Assertionfailureexample\106
[18:00:31][Step 5/5] 1562598029549  58  HTTP Request API missioncontrol May2019 200 OK  Thread Group 1-1    text    false   Value expected to match regexp '114.13295998454568', but it did not match: '[112.13295998454568,113.30262810285055,82.08123404643247]'  16425   741 1   1   https://xxxxx-api.yyyyy
.com/api/page_mission_control/  58  0   0
[18:00:31][Step 5/5] Process exited with code 0

尽管我的Jmeter测试在响应断言时失败，但TeamCity成功

1 个答案: