Warning: file_get_contents(/data/phpspider/zhask/data//catemap/1/amazon-web-services/14.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
AWS粘合失败,exitCode:15,用户类引发异常:java.lang.RuntimeException:未使用外部依赖项初始化_Java_Amazon Web Services_Apache Spark_Amazon S3_Aws Glue - Fatal编程技术网

AWS粘合失败,exitCode:15,用户类引发异常:java.lang.RuntimeException:未使用外部依赖项初始化

AWS粘合失败,exitCode:15,用户类引发异常:java.lang.RuntimeException:未使用外部依赖项初始化,java,amazon-web-services,apache-spark,amazon-s3,aws-glue,Java,Amazon Web Services,Apache Spark,Amazon S3,Aws Glue,我正在尝试使用aws glue Scala脚本处理aws s3文件夹中的几个文件。在此之前,我需要检查文件数量和名称。为此,我编写了一个java类,它包含检查文件数量和名称的所有操作,并打包到一个jar文件中。我在Glue配置中将这个jar文件作为一个依赖jars路径提供。 当我在本地运行这个java类时,它运行得很好。但在胶水作业中使用时,其刮平误差如下所示 ERROR ApplicationMaster: User class threw exception: java.lang.Runti

我正在尝试使用aws glue Scala脚本处理aws s3文件夹中的几个文件。在此之前,我需要检查文件数量和名称。为此,我编写了一个java类,它包含检查文件数量和名称的所有操作,并打包到一个jar文件中。我在Glue配置中将这个jar文件作为一个依赖jars路径提供。 当我在本地运行这个java类时,它运行得很好。但在胶水作业中使用时,其刮平误差如下所示

ERROR ApplicationMaster: User class threw exception: java.lang.RuntimeException: Not initialized
java.lang.RuntimeException: Not initialized
    at com.amazonaws.services.glue.util.JobBookmark$.preCheck(JobBookmarkUtils.scala:22)
    at com.amazonaws.services.glue.util.JobBookmark$.commit(JobBookmarkUtils.scala:84)
    at com.amazonaws.services.glue.util.Job$.commit(Job.scala:94)
    at GlueApp$.runJob(script_2020-10-04-04-10-24.scala:117)
    at GlueApp$.main(script_2020-10-04-04-10-24.scala:62)
我的Java Jar文件pom.xml依赖项

<dependencies>
         <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-s3</artifactId>
            <version>1.11.875</version>
        </dependency>
 
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-core</artifactId>
            <version>1.11.875</version>
        </dependency>
    </dependencies>
提前谢谢

public ArrayList<String> getFiles() {
        BasicAWSCredentials credit = new BasicAWSCredentials("key",
                "secreat");
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(Regions.US_EAST_2)
                .withCredentials(new AWSStaticCredentialsProvider(credit)).build();
        ArrayList<String> records = new ArrayList<>();
        ListObjectsV2Request req = new ListObjectsV2Request().withBucketName("my-inputpath")
                .withPrefix("monthly/data/");

        ListObjectsV2Result listing = s3Client.listObjectsV2(req);  

        for (S3ObjectSummary summary : listing.getObjectSummaries()) {
           
                records.add(summary.getKey().replace("monthly/", "/"));
        }
       
        return records;
    }
INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.RuntimeException: Not initialized)