Java 空手道考试通过与否不一致
我在Gradle中使用空手道,有8个功能文件测试SpringBootAPI的读取/获取功能 我看到这些测试以一种随机的方式失败。 这些失败在某种程度上与授权有关,但我看不出表面上有什么错 举个例子 这失败了Java 空手道考试通过与否不一致,java,integration-testing,karate,Java,Integration Testing,Karate,我在Gradle中使用空手道,有8个功能文件测试SpringBootAPI的读取/获取功能 我看到这些测试以一种随机的方式失败。 这些失败在某种程度上与授权有关,但我看不出表面上有什么错 举个例子 这失败了 Feature: Get Objectives Background: * url baseUrl * def objectiveEndpoint = '/objectives' * header Authorization = token Scenari
Feature: Get Objectives
Background:
* url baseUrl
* def objectiveEndpoint = '/objectives'
* header Authorization = token
Scenario: Get an Objective that exists
Given path objectiveEndpoint + '/37376564-3139-6232-2d66-6631392d3466'
When method GET
Then status 200
And match response contains { objectiveId: '37376564-3139-6232-2d66-6631392d3466' }
这就过去了
Feature: Get Assessments
Background:
* url baseUrl
* def assessmentEndpoint = '/assessments'
* header Authorization = token
Scenario: Get an assessment that exists
Given path assessmentEndpoint + '/2900b695-d344-4bec-b25d-524f6b22a93a'
When method GET
Then status 200
And match response contains { odiAssessmentId: '2900b695-d344-4bec-b25d-524f6b22a93a' }
目标测试失败,原因是401出现以下消息:
com.intuit.karate.exception.KarateException: objectives-read.feature:12 - status code was: 401, expected: 200, response time: 74, url: http://localhost:8080/objectives/37376564-3139-6232-2d66-6631392d3466, response: {"path":"/objectives/37376564-3139-6232-2d66-6631392d3466","error":"Unauthorized","message":"Unauthorized","timestamp":"2020-06-02T08:04:57.231+0000","status":401}
考试通过了
我通过运行授权功能获取令牌,并将其结果存储到令牌变量中
身份验证功能是:
Feature: Log In
Background:
* url 'https://$URL/oauth/token'
* def localSecret = secret
* def localClientId = clientId
Scenario: Get token
Given url 'https://$URL/oauth/token'
And form field grant_type = 'client_credentials'
And form field client_id = localClientId
And form field client_secret = localSecret
And form field audience = 'http://localhost:8080'
When method post
Then status 200
And match response contains { access_token: '#string' }
And def access_token = $.access_token
然后,我将令牌添加到config中,并将其传递到每个测试中,如下所示:
var result = karate.callSingle('classpath:login.feature', config);
config.token = 'Bearer ' + result.access_token;
@Karate.Test
Karate testAssessments() {
return Karate.run().relativeTo(AssessmentsRunner.class);
}
public class SpecTestParallel {
public static void generateReport(String karateOutputPath) {
Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[]{"json"}, true);
List<String> jsonPaths = new ArrayList(jsonFiles.size());
jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
Configuration config = new Configuration(new File("target"), "Test API");
ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
reportBuilder.generateReports();
}
@Test
void testParallel() {
Results results = Runner.path("classpath:specTests").tags("~@ignore").parallel(5);
generateReport(results.getReportDir());
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
}
我已经确认上述两个功能中使用的令牌是有效的。我已经用失败测试的输出中打印的令牌手动测试了我的API,当我在Postman中重新创建它们时,上述两个测试都可以正常工作。对于我的API来说,这并不是一个问题,因为如果我重新运行测试套件,失败的测试会有所不同,在我的CI上,这些测试的生成是绿色的
我在单独运行测试套件时遇到了以下两个问题:
var result = karate.callSingle('classpath:login.feature', config);
config.token = 'Bearer ' + result.access_token;
@Karate.Test
Karate testAssessments() {
return Karate.run().relativeTo(AssessmentsRunner.class);
}
public class SpecTestParallel {
public static void generateReport(String karateOutputPath) {
Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[]{"json"}, true);
List<String> jsonPaths = new ArrayList(jsonFiles.size());
jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
Configuration config = new Configuration(new File("target"), "Test API");
ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
reportBuilder.generateReports();
}
@Test
void testParallel() {
Results results = Runner.path("classpath:specTests").tags("~@ignore").parallel(5);
generateReport(results.getReportDir());
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
}
当我像这样并行运行所有测试时:
var result = karate.callSingle('classpath:login.feature', config);
config.token = 'Bearer ' + result.access_token;
@Karate.Test
Karate testAssessments() {
return Karate.run().relativeTo(AssessmentsRunner.class);
}
public class SpecTestParallel {
public static void generateReport(String karateOutputPath) {
Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[]{"json"}, true);
List<String> jsonPaths = new ArrayList(jsonFiles.size());
jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
Configuration config = new Configuration(new File("target"), "Test API");
ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
reportBuilder.generateReports();
}
@Test
void testParallel() {
Results results = Runner.path("classpath:specTests").tags("~@ignore").parallel(5);
generateReport(results.getReportDir());
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
}
公共类SpecTestParallel{
公共静态void生成器报告(字符串karateOutputPath){
Collection jsonFiles=FileUtils.listFiles(新文件(karateOutputPath),新字符串[]{“json”},true);
List jsonpath=newarraylist(jsonFiles.size());
forEach(文件->jsonpath.add(文件.getAbsolutePath());
配置配置=新配置(新文件(“目标”),“测试API”);
ReportBuilder ReportBuilder=newreportbuilder(jsonpath,config);
reportBuilder.generateReports();
}
@试验
void testParallel(){
Results=Runner.path(“classpath:specTests”).tags(~@ignore”).parallel(5);
generateReport(results.getReportDir());
assertEquals(0,results.getFailCount(),results.getErrorMessages());
}
}
以前有人有过类似的问题吗 事实证明,这是IntelliJ中的意外行为 所以空手道似乎有一个重试策略。如果测试失败,它将重试几次 当我使用IntelliJ中的test runner函数运行测试时,每次测试失败时,IntelliJ都会在test runner窗口中记录失败的测试,但Karate会继续运行、重试,测试通过。我现在可以在报告中看到,但是IntelliJ测试运行程序没有更新 这导致了一种混乱的情况,即测试通过,但似乎在本地失败,但测试通过CI 以下是本地测试的示例: 在CI中传递相同的提交: 我不确定这里有什么解决办法,如果有的话。如果IntelliJ意识到这种行为就好了,或者空手道只能在处理完所有重试后报告测试结果——现在看起来空手道会在第一次测试运行完成后立即报告
这几天令人困惑。不,一定是你做错了什么。可能您使用的是此处未显示的Java代码,它不是线程安全的。任何人都无法从查看此代码中获得任何解决方案(或帮助我们缩小错误修复范围),请遵循以下流程:您是否在JUnit 5上,是否可以升级到0.9.6.RC3并尝试,这有(轻微)的可能性通过以下方式修复: