Linkedin Where How，Where How常见构建错误（创建的作业太多）

根据此处的文档（）尝试在Centos7上安装Linkedin的

WhereHows

，并遇到与WhereHows common相关的生成错误：使用gradle生成时进行测试

> Task :wherehows-common:test
Putting task artifact state for task ':wherehows-common:test' into context took 0.001 secs.
Executing task ':wherehows-common:test' (up-to-date check took 0.041 secs) due to:
  No history is available.
Starting process 'Gradle Test Executor 2'. Working directory: /home/me/WhereHows/wherehows-common Command: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.7.1.201405082137.jar_r1v2o0qii4g3mwygeto49hza/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,dumponexit=true,output=file,jmx=false -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/me/.gradle/caches/4.0.2/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 2'
Successfully started process 'Gradle Test Executor 2'

Gradle suite > Gradle test > FileWriterTest. STANDARD_OUT
    hdfs:///a/b/cnull{"sample": [aaa, bbb, ccc] }

Gradle suite > Gradle test > wherehows.common.utils.JobsUtilTest.testGetEnabledJobs FAILED
    java.lang.AssertionError: expected [2] but found [47]
        at org.testng.Assert.fail(Assert.java:94)
        at org.testng.Assert.failNotEquals(Assert.java:513)
        at org.testng.Assert.assertEqualsImpl(Assert.java:135)
        at org.testng.Assert.assertEquals(Assert.java:116)
        at org.testng.Assert.assertEquals(Assert.java:389)
        at org.testng.Assert.assertEquals(Assert.java:399)
        at wherehows.common.utils.JobsUtilTest.testGetEnabledJobs(JobsUtilTest.java:91)
.....

将引用的JobsUtilTest.java（）中的testGetEnabledJobs（）函数更改为如下所示（注意，我添加了一些print语句，以查看运行测试时jobs字典实际看到的内容）：

所以应该只有两份工作，但是有一堆重复的工作。这里有什么问题吗

其他使用此工具的人知道这里可能存在什么问题吗

**有用的系统和环境。资料:

[me@mapr07 WhereHows]$ tree -L 2 wherehows-etl/extralibs/
wherehows-etl/extralibs/
├── dist
│   ├── changes.txt
│   ├── compile_run_demo.txt
│   ├── demos
│   ├── GeneralSQLParser_UserGuide.pdf
│   ├── gsp.jar
│   ├── javadoc
│   └── test
├── gsp.jar
├── gsp_java_trial_1_9_2_3.zip
├── ojdbc7.jar
├── orai18n.jar
├── README.md
├── readme.txt
├── tdgssconfig.jar
├── terajdbc4.jar
└── TeraJDBC__indep_indep.16.20.00.06.tar

4 directories, 13 files

[me@mapr07 WhereHows]$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

[me@mapr07 WhereHows]$ javac -version
javac 1.8.0_171

[me@mapr07 WhereHows]$ echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/jre

[me@mapr07 WhereHows]$ cat /proc/version
Linux version 3.10.0-693.21.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Wed Mar 7 19:03:37 UTC 2018

解决了这个问题。 在测试函数的代码中添加一些打印语句，如下所示：

 @Test
  public void testGetEnabledJobs() throws IOException, ConfigurationException {
    String propertyStr1 = "job.class=test\n" + "job.cron.expr=0 0 1 * * ? *\n" + "#job.disabled=1\n" + "job.type=TEST1";
    String propertyStr2 = "job.class=test\n" + "#job.disabled=1\n";

    Path path1 = createPropertiesFile(propertyStr1);
    Path path2 = createPropertiesFile(propertyStr2);

    String filename1 = jobNameFromPath(path1);
    String filename2 = jobNameFromPath(path2);
    System.out.println("filenames: " + filename1 + "\n" + filename2);

    String dir = path1.getParent().toString();
    System.out.println("dir="+dir);

    Map<String, Properties> jobs = getEnabledJobs(dir);

    System.out.println("Displaying jobs map");
    jobs.forEach((key, value) -> System.out.println(key + ":" + value));

    Assert.assertEquals(jobs.size(), 2);
    Assert.assertEquals(jobs.get(filename1).getProperty("job.class"), "test");
    Assert.assertEquals(jobs.get(filename1).getProperty("job.disabled", ""), "");
    Assert.assertEquals(jobs.get(filename2).getProperty("job.class"), "test");

    Files.deleteIfExists(path1);
    Files.deleteIfExists(path2);
  }

查看/tmp目录。我看到了一堆旧的临时工作文件。删除这些并重新运行似乎已允许我成功构建

我认为问题是这样的：

最初运行gradlew build命令，由于

JAVA_HOME

未设置，该命令出错并退出。此时，我认为testGetEnabledJobs（）已经创建了它的断言所期望并成功传递的2个temp*.job文件

然后设置

JAVA\u HOME

并再次尝试构建。这次testGetEnabledJobs（）在/tmp中创建了更多的temp*.job文件（在上一次构建尝试中未删除的文件的基础上），并得到testGetEnabledJobs（）错误，因为现在/tmp中的temp*.job文件比预期的多，导致错误并退出

此时，构建的每次重试都会显示断言计数越来越超出预期值

在任何情况下，我认为应该更改此测试以删除它在相关断言中出错时创建的temp*.job文件，否则我们将在/tmp dir中得到复合问题。我刚修好的

[me@mapr07 WhereHows]$ tree -L 2 wherehows-etl/extralibs/
wherehows-etl/extralibs/
├── dist
│   ├── changes.txt
│   ├── compile_run_demo.txt
│   ├── demos
│   ├── GeneralSQLParser_UserGuide.pdf
│   ├── gsp.jar
│   ├── javadoc
│   └── test
├── gsp.jar
├── gsp_java_trial_1_9_2_3.zip
├── ojdbc7.jar
├── orai18n.jar
├── README.md
├── readme.txt
├── tdgssconfig.jar
├── terajdbc4.jar
└── TeraJDBC__indep_indep.16.20.00.06.tar

4 directories, 13 files

[me@mapr07 WhereHows]$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

[me@mapr07 WhereHows]$ javac -version
javac 1.8.0_171

[me@mapr07 WhereHows]$ echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/jre

[me@mapr07 WhereHows]$ cat /proc/version
Linux version 3.10.0-693.21.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Wed Mar 7 19:03:37 UTC 2018

 @Test
  public void testGetEnabledJobs() throws IOException, ConfigurationException {
    String propertyStr1 = "job.class=test\n" + "job.cron.expr=0 0 1 * * ? *\n" + "#job.disabled=1\n" + "job.type=TEST1";
    String propertyStr2 = "job.class=test\n" + "#job.disabled=1\n";

    Path path1 = createPropertiesFile(propertyStr1);
    Path path2 = createPropertiesFile(propertyStr2);

    String filename1 = jobNameFromPath(path1);
    String filename2 = jobNameFromPath(path2);
    System.out.println("filenames: " + filename1 + "\n" + filename2);

    String dir = path1.getParent().toString();
    System.out.println("dir="+dir);

    Map<String, Properties> jobs = getEnabledJobs(dir);

    System.out.println("Displaying jobs map");
    jobs.forEach((key, value) -> System.out.println(key + ":" + value));

    Assert.assertEquals(jobs.size(), 2);
    Assert.assertEquals(jobs.get(filename1).getProperty("job.class"), "test");
    Assert.assertEquals(jobs.get(filename1).getProperty("job.disabled", ""), "");
    Assert.assertEquals(jobs.get(filename2).getProperty("job.class"), "test");

    Files.deleteIfExists(path1);
    Files.deleteIfExists(path2);
  }

> Task :wherehows-common:test
Putting task artifact state for task ':wherehows-common:test' into context took 0.0 secs.
Executing task ':wherehows-common:test' (up-to-date check took 0.02 secs) due to:
  No history is available.
Starting process 'Gradle Test Executor 3'. Working directory: /home/rvillanueva/WhereHows/wherehows-common Command: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.7.1.201405082137.jar_r1v2o0qii4g3mwygeto49hza/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,dumponexit=true,output=file,jmx=false -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/rvillanueva/.gradle/caches/4.0.2/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 3'
Successfully started process 'Gradle Test Executor 3'

Gradle suite > Gradle test > FileWriterTest. STANDARD_OUT
    hdfs:///a/b/cnull{"sample": [aaa, bbb, ccc] }

Gradle suite > Gradle test > wherehows.common.utils.JobsUtilTest.testGetEnabledJobs STANDARD_OUT
    filenames: temp7961565693796243498
    temp1135071152526160059
    dir=/tmp
    Displaying jobs map
    temp7961565693796243498:{job.class=test, job.type=TEST1, job.cron.expr=0 0 1 * * ? *}
    temp1135071152526160059:{job.class=test}
    ......